Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Naming vec_distance column and then referring to it in order by does not use the index #8829

Open
timsehn opened this issue Feb 6, 2025 · 0 comments

Comments

@timsehn
Copy link
Contributor

timsehn commented Feb 6, 2025

Repro:

  1. Clone: https://www.dolthub.com/repositories/timsehn/vector_example
  2. Checkout all-MiniLM-L6-v2 branch
  3. This query uses the index:
$ dolt sql -q "set @query = (select embeddings from yelp where id = 1); explain plan select id, review, vec_distance(embeddings, @query) as distance from yelp order by vec_distance(embeddings, @query) limit 3;"
+-------------------------------------------------------------------------------------------------------+
| plan                                                                                                  |
+-------------------------------------------------------------------------------------------------------+
| Limit(3)                                                                                              |
|  └─ Project                                                                                           |
|      ├─ columns: [yelp.id, yelp.review, VEC_DISTANCE_L2_SQUARED(yelp.embeddings, @query) as distance] |
|      └─ IndexedTableAccess(yelp)                                                                      |
|          ├─ index: [yelp.embeddings]                                                                  |
|          ├─ order: VEC_DISTANCE_L2_SQUARED(yelp.embeddings, @query) LIMIT 3 (bigint)                  |
|          └─ columns: [review id embeddings]                                                           |
+-------------------------------------------------------------------------------------------------------+

This query does not:

$ dolt sql -q "set @query = (select embeddings from yelp where id = 1); explain plan select id, review, vec_distance(embeddings, @query) as distance from yelp order by distance limit 3;"
+--------------------------------------------------------------------------------------------------------------------------------------------+
| plan                                                                                                                                       |
+--------------------------------------------------------------------------------------------------------------------------------------------+
| Limit(3)                                                                                                                                   |
|  └─ Project                                                                                                                                |
|      ├─ columns: [yelp.id, yelp.review, VEC_DISTANCE_L2_SQUARED(yelp.embeddings, @query) as distance]                                      |
|      └─ Sort(VEC_DISTANCE_L2_SQUARED(yelp.embeddings, @query) as distance ASC)                                                             |
|          └─ Project                                                                                                                        |
|              ├─ columns: [yelp.label, yelp.review, yelp.id, yelp.embeddings, VEC_DISTANCE_L2_SQUARED(yelp.embeddings, @query) as distance] |
|              └─ Table                                                                                                                      |
|                  ├─ name: yelp                                                                                                             |
|                  └─ columns: [label review id embeddings]                                                                                  |
+--------------------------------------------------------------------------------------------------------------------------------------------+

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants