how do i filter a metadata value between a range

[dansharpe83]

hi,

we currently have price in the metdata, and im trying to write a filter so i can query for vectors where the price is between 2 values, but i'm not sure how to write it using this library,
i've tried:

map.Add("$price", new MetadataMap
                {
                    
                        ["$gte"] = minPrice.Value,
                        ["&lte"] = maxPrice.Value
                });

i've also tried:

map.Add("$price", new MetadataMap
                {
                    ["$and"] = new MetadataMap{
                        ["$gte"] = minPrice.Value,
                        ["&lte"] = maxPrice.Value
                    }
                });

but that gives an error from Pinecone, any help would be greatly appreciated

from my understanding it would need to be something along the lines of:

map.Add("$and", new MetadataMap[]
                {
                    ["Price"], new MetadataMap()
                {
                    ["$gte"] = minPrice.Value
                }, 
 ["Price"], new MetadataMap()
                {
                    ["$lte"] = minPrice.Value
                }, 
                });

but i cannot add price twice to the metdata data map, nor can i use a metadata map array

[neon-sunset]

Hi Dan,

Could you try the following snippet?

var priceRange = new MetadataMap
{
    ["price"] = new MetadataMap
    {
        ["$gte"] = 75,
        ["$lte"] = 125
    }
};

Here's the full code to double-check that the query has the desired behavior:

var key = "your-key";
var env = "your-env";

using var pinecone = new PineconeClient(key, env);
using var index = await pinecone.GetIndex(
    (await pinecone.ListIndexes())[0]);

var vec1 = new Vector { Id = "first", Values = new float[1536], Metadata = new() { ["price"] = 50 } };
var vec2 = new Vector { Id = "second", Values = new float[1536], Metadata = new() { ["price"] = 100 } };
var vec3 = new Vector { Id = "third", Values = new float[1536], Metadata = new() { ["price"] = 150 } };

await index.Upsert(new[] { vec1, vec2, vec3 });

var priceRange = new MetadataMap
{
    ["price"] = new MetadataMap
    {
        ["$gte"] = 75,
        ["$lte"] = 125
    }
};

var results = await index.Query(
    new float[1536],
    topK: 3,
    filter: priceRange,
    includeMetadata: true);

Console.WriteLine(string.Join('\n', results.SelectMany(v => v.Metadata!)));

More examples on mapping Pinecone documentation syntax to metadata types:

• { "word": "value", "number": 1337 } -> new MetadataMap { ["word"] = "value", ["number"] = 1337 }
• [ "first", "second", "third"] -> new MetadataValue[] { "first", "second", "third" }
• { "$or": [{ "genre": { "$eq": "drama" } }, { "year": { "$gte": 2020 } }] } -> (ouch!)

new MetadataMap
{
   ["$or"] = new MetadataValue[]
   {
       new MetadataMap { ["genre"] = new MetadataMap { ["$eq"] = "drama" } }, 
       new MetadataMap { ["year"] = new MetadataMap { ["$gte"] = 2020 } },
}

As you can see, it is possible to mix and match so you could probably avoid structuring queries in ways which are migraine-inducing to write in C#.

In general, metadata filter query building is a user experience shortcoming of this library due to statically typed nature of C# and tends to get ugly fast. I've been considering implementing a query parser to allow writing filters as a string and then map it to MetadataMap internally but this seemed to be a stretch goal with questionable value at the time (better syntax might be enabled by newer lang. features for example).

[neon-sunset]

@dansharpe83 Let me know if the above solution works for you.