feat: support string functions [LINQ] #500

jogibear9988 · 2023-04-03T15:15:28Z

Closes #499

Proposed Changes

Support string functions from linq

Checklist

CHANGELOG.md updated
Rebased/mergeable
A test has been added if appropriate
dotnet test completes successfully
Commit messages are conventional
Sign CLA (if not already signed)

jogibear9988 · 2023-04-04T13:38:13Z

@powersj could you review?

jogibear9988 · 2023-04-27T11:37:21Z

@bednar could you review this?

bednar · 2023-05-04T06:34:41Z

@bednar could you review this?

Yes, I can. Can you rebase sources to trigger checks?

Regards

jogibear9988 · 2023-05-04T08:06:09Z

rebase done

jogibear9988 · 2023-07-13T07:15:54Z

@bednar an chance to get this merged?

bednar · 2023-07-13T07:28:22Z

Hi @jogibear9988,

I plan to review your PR either today or tomorrow. In the meantime, could you please correct the formatting of your code? You can do this by running ./Scripts/reformat-code.sh in the root of the project.

Best Regards

bednar

Thanks for your PR 👍

There are a few requirements that must be be satisfy before we accept the PR:

Please update CHANGELOG.md file

and also:

Client.Linq/Internal/QueryExpressionTreeVisitor.cs

Client.Linq/Internal/QueryAggregator.cs

jogibear9988 · 2023-07-13T20:34:38Z

I've now created the classes, but with the name of the C# functions. Is this okay? I could also use the name of Influx, but then also "ToString" should be named "String"

Client.Linq/Internal/Expressions/String/Contains.cs

Client.Linq/Internal/Expressions/String/EndsWith.cs

Client.Linq/Internal/Expressions/String/Replace.cs

Client.Linq/Internal/Expressions/String/StartsWith.cs

Client.Linq/Internal/Expressions/String/ToLower.cs

Client.Linq/Internal/Expressions/String/ToString.cs

Client.Linq/Internal/Expressions/String/ToUpper.cs

CHANGELOG.md

Client.Linq/Internal/Expressions/String/Contains.cs

Client.Linq/Internal/Expressions/String/EndsWith.cs

Client.Linq/Internal/Expressions/String/StartsWith.cs

jogibear9988 · 2023-07-17T07:51:22Z

Should I change QueryAggregator, to use Lazy initalized Fields for the other parts as well? So only memory is used when for example filters, sortings,... are used?

…nfluxdata#499

bednar · 2023-07-17T12:47:04Z

I've now created the classes, but with the name of the C# functions. Is this okay? I could also use the name of Influx, but then also "ToString" should be named "String"

It is ok. Thanks

Should I change QueryAggregator, to use Lazy initalized Fields for the other parts as well? So only memory is used when for example filters, sortings,... are used?

If you have the time, yes, please make the change. However, if you're unable to do so, you may leave the code as is; it won't block the approval process.

bednar

I believe we're almost ready to merge this PR into our master branch. Please satisfy the following requirements for the tests:

bednar · 2023-07-17T12:51:37Z

Client.Linq.Test/InfluxDBQueryVisitorTest.cs

+        [Test]
+        public void StringFunctionsQuery()
+        {
+            var query = from s in InfluxDBQueryable<Sensor>.Queryable("my-bucket", "my-org", _queryApi)
+                where s.SensorId.ToLower().Contains("aaa")
+                select s;
+            var visitor = BuildQueryVisitor(query);
+
+            const string expected =
+                "import \"strings\"\nstart_shifted = int(v: time(v: p2))\n\nfrom(bucket: p1) |> range(start: time(v: start_shifted)) |> filter(fn: (r) => strings.containsStr(v: strings.toLower(v: r[\"sensor_id\"]), substr: p3)) |> pivot(rowKey:[\"_time\"], columnKey: [\"_field\"], valueColumn: \"_value\") |> drop(columns: [\"_start\", \"_stop\", \"_measurement\"]) |> filter(fn: (r) => strings.containsStr(v: strings.toLower(v: r[\"sensor_id\"]), substr: p3))";
+            var qry = visitor.BuildFluxQuery();
+            Assert.AreEqual(expected, visitor.BuildFluxQuery());
+        }
+
+        [Test]
+        public void ToStringFunctionQuery()
+        {
+            var query = from s in InfluxDBQueryable<Sensor>.Queryable("my-bucket", "my-org", _queryApi)
+                where s.Value.ToString() == "3"
+                select s;
+            var visitor = BuildQueryVisitor(query);
+
+            const string expected =
+                "start_shifted = int(v: time(v: p2))\n\nfrom(bucket: p1) |> range(start: time(v: start_shifted)) |> filter(fn: (r) => (string(v: r[\"data\"]) == p3)) |> pivot(rowKey:[\"_time\"], columnKey: [\"_field\"], valueColumn: \"_value\") |> drop(columns: [\"_start\", \"_stop\", \"_measurement\"]) |> filter(fn: (r) => (string(v: r[\"data\"]) == p3))";
+            var qry = visitor.BuildFluxQuery();
+            Assert.AreEqual(expected, visitor.BuildFluxQuery());
+        }
+
+        [Test]
+        public void ReplaceAllFunctionQuery()
+        {
+            var query = from s in InfluxDBQueryable<Sensor>.Queryable("my-bucket", "my-org", _queryApi)
+                where s.SensorId.ToLower().Replace("a", "b") == "b"
+                select s;
+            var visitor = BuildQueryVisitor(query);
+
+            const string expected =
+                "import \"strings\"\nstart_shifted = int(v: time(v: p2))\n\nfrom(bucket: p1) |> range(start: time(v: start_shifted)) |> filter(fn: (r) => (strings.replaceAll(v: strings.toLower(v: r[\"sensor_id\"]), t: p3, u: p4) == p5)) |> pivot(rowKey:[\"_time\"], columnKey: [\"_field\"], valueColumn: \"_value\") |> drop(columns: [\"_start\", \"_stop\", \"_measurement\"]) |> filter(fn: (r) => (strings.replaceAll(v: strings.toLower(v: r[\"sensor_id\"]), t: p3, u: p4) == p5))";
+            var qry = visitor.BuildFluxQuery();
+            Assert.AreEqual(expected, visitor.BuildFluxQuery());
+        }
+


Please include tests for all new string functions, and also evaluate the parameters of the Flux query. Here's an example of what we're looking for:

influxdb-client-csharp/Client.Linq.Test/InfluxDBQueryVisitorTest.cs

Line 339 in 4c6f9f2

Assert.AreEqual("4", GetLiteral<IntegerLiteral>(ast, 2).Value);

jogibear9988 · 2023-07-29T16:49:27Z

hopefully i can fix the rest next week

bednar · 2023-07-31T03:21:06Z

@jogibear9988 thanks, awesome news 👍

jogibear9988 · 2023-10-31T13:40:06Z

@bednar
hey, I think I'll have no time to fix this in the next time. all the functions work, but as influx will not support flux official in the next version, I'll think I'll leave it as is.

bednar · 2023-11-01T11:46:02Z

This PR has been closed because it has not had recent activity. Please reopen if this PR is still important to you and you want to continue with them.

jogibear9988 force-pushed the influxString branch from 09b7387 to 817a9f0 Compare April 3, 2023 19:28

7evenk approved these changes Apr 3, 2023

View reviewed changes

bednar self-requested a review May 4, 2023 06:34

jogibear9988 force-pushed the influxString branch from 817a9f0 to 3ca80c3 Compare May 4, 2023 08:05

jogibear9988 force-pushed the influxString branch from 1743608 to 699ce13 Compare July 13, 2023 07:15

jogibear9988 force-pushed the influxString branch from 699ce13 to 971245d Compare July 13, 2023 13:46

bednar changed the title ~~first draft of influx string functions, fixes #499~~ feat: support string functions [LINQ] Jul 13, 2023

bednar requested changes Jul 13, 2023

View reviewed changes

jogibear9988 force-pushed the influxString branch from 118a69c to 4c6f9f2 Compare July 14, 2023 08:22

bednar requested changes Jul 16, 2023

View reviewed changes

jogibear9988 force-pushed the influxString branch from 4c6f9f2 to aa19ce1 Compare July 17, 2023 07:40

jogibear9988 force-pushed the influxString branch from aa19ce1 to fb8aeff Compare July 17, 2023 08:31

feat: support some linq string functions in where conditions, fixes i…

8dc5afa

…nfluxdata#499

jogibear9988 force-pushed the influxString branch from fb8aeff to 8dc5afa Compare July 17, 2023 08:35

bednar requested changes Jul 17, 2023

View reviewed changes

bednar closed this Nov 1, 2023

bednar added this to the 4.14.0 milestone Nov 1, 2023

bednar added invalid This doesn't seem right wontfix This will not be worked on labels Nov 1, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: support string functions [LINQ] #500

feat: support string functions [LINQ] #500

jogibear9988 commented Apr 3, 2023 •

edited

Loading

jogibear9988 commented Apr 4, 2023

jogibear9988 commented Apr 27, 2023

bednar commented May 4, 2023

jogibear9988 commented May 4, 2023

jogibear9988 commented Jul 13, 2023

bednar commented Jul 13, 2023

bednar left a comment

jogibear9988 commented Jul 13, 2023

jogibear9988 commented Jul 17, 2023

bednar commented Jul 17, 2023

bednar left a comment

bednar Jul 17, 2023

jogibear9988 commented Jul 29, 2023

bednar commented Jul 31, 2023

jogibear9988 commented Oct 31, 2023

bednar commented Nov 1, 2023

feat: support string functions [LINQ] #500

feat: support string functions [LINQ] #500

Conversation

jogibear9988 commented Apr 3, 2023 • edited Loading

Proposed Changes

Checklist

jogibear9988 commented Apr 4, 2023

jogibear9988 commented Apr 27, 2023

bednar commented May 4, 2023

jogibear9988 commented May 4, 2023

jogibear9988 commented Jul 13, 2023

bednar commented Jul 13, 2023

bednar left a comment

Choose a reason for hiding this comment

jogibear9988 commented Jul 13, 2023

jogibear9988 commented Jul 17, 2023

bednar commented Jul 17, 2023

bednar left a comment

Choose a reason for hiding this comment

bednar Jul 17, 2023

Choose a reason for hiding this comment

jogibear9988 commented Jul 29, 2023

bednar commented Jul 31, 2023

jogibear9988 commented Oct 31, 2023

bednar commented Nov 1, 2023

jogibear9988 commented Apr 3, 2023 •

edited

Loading