From a20d7aeb1ac41a96a2dac9d354622309e8c11c2e Mon Sep 17 00:00:00 2001 From: Wan Kai Date: Fri, 9 Aug 2024 06:44:04 +0800 Subject: [PATCH] Add bydbctl and web-ui interacting guide. (#507) * Add bydbctl and web-ui interacting guide. --- CHANGES.md | 2 + docs/concept/data-model.md | 2 +- docs/crud/measure/query.md | 65 ----- docs/crud/stream/query.md | 61 ----- docs/interacting/bydbctl/bydbctl.md | 20 ++ .../{crud => interacting/bydbctl}/property.md | 7 +- .../bydbctl/query/filter-operation.md | 92 ++++++++ docs/interacting/bydbctl/query/measure.md | 223 ++++++++++++++++++ docs/interacting/bydbctl/query/stream.md | 205 ++++++++++++++++ .../bydbctl/schema}/group.md | 6 +- .../bydbctl/schema/index-rule-binding.md} | 6 +- .../bydbctl/schema/index-rule.md} | 20 +- .../bydbctl/schema/measure.md} | 6 +- .../bydbctl/schema/stream.md} | 6 +- .../bydbctl/schema/top-n-aggregation.md | 20 ++ docs/interacting/java-client.md | 3 + docs/interacting/web-ui/dashboard.md | 9 + docs/interacting/web-ui/property.md | 11 + docs/interacting/web-ui/query/measure.md | 17 ++ docs/interacting/web-ui/query/stream.md | 13 + docs/interacting/web-ui/schema/group.md | 18 ++ .../web-ui/schema/index-rule-binding.md | 17 ++ docs/interacting/web-ui/schema/index-rule.md | 17 ++ docs/interacting/web-ui/schema/measure.md | 20 ++ docs/interacting/web-ui/schema/stream.md | 20 ++ .../web-ui/schema/top-n-aggregation.md | 16 ++ docs/menu.yml | 118 +++++---- 27 files changed, 830 insertions(+), 190 deletions(-) delete mode 100644 docs/crud/measure/query.md delete mode 100644 docs/crud/stream/query.md create mode 100644 docs/interacting/bydbctl/bydbctl.md rename docs/{crud => interacting/bydbctl}/property.md (94%) create mode 100644 docs/interacting/bydbctl/query/filter-operation.md create mode 100644 docs/interacting/bydbctl/query/measure.md create mode 100644 docs/interacting/bydbctl/query/stream.md rename docs/{crud => interacting/bydbctl/schema}/group.md (88%) rename docs/{crud/index_rule_binding.md => interacting/bydbctl/schema/index-rule-binding.md} (92%) rename docs/{crud/index_rule.md => interacting/bydbctl/schema/index-rule.md} (76%) rename docs/{crud/measure/schema.md => interacting/bydbctl/schema/measure.md} (91%) rename docs/{crud/stream/schema.md => interacting/bydbctl/schema/stream.md} (90%) create mode 100644 docs/interacting/bydbctl/schema/top-n-aggregation.md create mode 100644 docs/interacting/java-client.md create mode 100644 docs/interacting/web-ui/dashboard.md create mode 100644 docs/interacting/web-ui/property.md create mode 100644 docs/interacting/web-ui/query/measure.md create mode 100644 docs/interacting/web-ui/query/stream.md create mode 100644 docs/interacting/web-ui/schema/group.md create mode 100644 docs/interacting/web-ui/schema/index-rule-binding.md create mode 100644 docs/interacting/web-ui/schema/index-rule.md create mode 100644 docs/interacting/web-ui/schema/measure.md create mode 100644 docs/interacting/web-ui/schema/stream.md create mode 100644 docs/interacting/web-ui/schema/top-n-aggregation.md diff --git a/CHANGES.md b/CHANGES.md index 7389aaae3..255c30dd0 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -43,6 +43,8 @@ Release Notes. - Introduce new doc menu structure. - Add installation on Docker and Kubernetes. - Add quick-start guide. +- Add web-ui interacting guide. +- Add bydbctl interacting guide. ### Chores diff --git a/docs/concept/data-model.md b/docs/concept/data-model.md index 732119f7d..06726eb3e 100644 --- a/docs/concept/data-model.md +++ b/docs/concept/data-model.md @@ -6,7 +6,7 @@ This chapter introduces BanyanDB's data models and covers the following: * data model * data retrieval -You can also find [examples](../crud/) of how to interact with BanyanDB using [bydbctl](../clients.md#command-line), how to create and drop groups, or how to create, read, update and drop streams/measures. +You can also find [examples](../interacting/bydbctl/schema) of how to interact with BanyanDB using [bydbctl](../interacting/bydbctl/bydbctl.md), how to create and drop groups, or how to create, read, update and drop streams/measures. ## Structure of BanyanDB diff --git a/docs/crud/measure/query.md b/docs/crud/measure/query.md deleted file mode 100644 index e4a5dcc59..000000000 --- a/docs/crud/measure/query.md +++ /dev/null @@ -1,65 +0,0 @@ -# Query Measures - -Query operation queries the data in a measure. - -[`bydbctl`](../../clients.md#command-line) is the command line tool in examples. - -The input contains two parts: - -* Request: a YAML-based text which is defined by the [API](#api-reference) -* Time Range: YAML and CLI's flags both support it. - -## Time Range - -The query specification contains `time_range` field. The request should set absolute times to it. -`bydbctl` also provides `start` and `end` flags to support passing absolute and relative times. - -"start" and "end" specify a time range during which the query is performed, they can be an absolute time like ["2006-01-02T15:04:05Z07:00"](https://www.rfc-editor.org/rfc/rfc3339), -or relative time (to the current time) like "-30m", or "30m". -They are both optional and their default values follow the rules below: - -* when "start" and "end" are both absent, "start = now - 30 minutes" and "end = now", -namely past 30 minutes; -* when "start" is absent and "end" is present, this command calculates "start" (minus 30 units), -e.g. "end = 2022-11-09T12:34:00Z", so "start = end - 30 minutes = 2022-11-09T12:04:00Z"; -* when "start" is present and "end" is absent, this command calculates "end" (plus 30 units), -e.g. "start = 2022-11-09T12:04:00Z", so "end = start + 30 minutes = 2022-11-09T12:34:00Z". - -## Examples - -To retrieve a series of data points between `2022-10-15T22:32:48Z` and `2022-10-15T23:32:48Z` could use the below command. These data points contain tags: `id` and `entity_id` that belong to a family `default`. They also choose fields: `total` and `value`. - -```shell -bydbctl measure query -f - < more ~/.bydbctl.yaml +addr: http://127.0.0.1:17913 +group: "" +``` + +`bydbctl` leverages HTTP endpoints to retrieve data instead of gRPC. + +## HTTP client + +Users could select any HTTP client to access the HTTP based endpoints. The default address is `localhost:17913/api` \ No newline at end of file diff --git a/docs/crud/property.md b/docs/interacting/bydbctl/property.md similarity index 94% rename from docs/crud/property.md rename to docs/interacting/bydbctl/property.md index 2b2d5804c..2aec93f55 100644 --- a/docs/crud/property.md +++ b/docs/interacting/bydbctl/property.md @@ -1,10 +1,10 @@ -# CRUD Property +# CRUD [Property](../../concept/data-model.md#properties) CRUD operations create/update, read and delete property. Property stores the user defined data. -[`bydbctl`](../clients.md#command-line) is the command line tool in examples. +[bydbctl](bydbctl.md) is the command line tool in examples. ## Apply (Create/Update) operation @@ -76,6 +76,7 @@ tags: str: value: "failed" ttl: "1h" +EOF ``` ## Get operation @@ -169,4 +170,4 @@ bydbctl property keepalive --lease_id 1 ## API Reference -[MeasureService v1](../api-reference.md#PropertyService) +[PropertyService v1](../../api-reference.md#propertyservice) diff --git a/docs/interacting/bydbctl/query/filter-operation.md b/docs/interacting/bydbctl/query/filter-operation.md new file mode 100644 index 000000000..7283c7e52 --- /dev/null +++ b/docs/interacting/bydbctl/query/filter-operation.md @@ -0,0 +1,92 @@ +# Filter Operation + +Filter operation is a part of the query configuration. It is used to filter the data based on the given condition for [Stream](stream.md) and [Measure](measure.md) queries. + +The condition is a combination of the tag name, operation, and value. +The operation's root is Criteria which is defined in the [API Reference](../../../api-reference.md#criteria). + +The following are the examples of filter operations: + +## [Condition.BinaryOp](../../../api-reference.md#conditionbinaryop) + +### EQ, NE, LT, GT, LE and GE +EQ, NE, LT, GT, LE and GE, only one operand should be given, i.e. one-to-one relationship. + +```shell +criteria: + condition: + name: "entity_id" + op: "BINARY_OP_EQ" + value: + str: + value: "entity_1" +``` + +### IN and NOT_IN +HAVING and NOT_HAVING allow multi-value to be the operand such as array/vector, i.e. one-to-many relationship. + +```shell +criteria: + condition: + name: "entity_id" + op: "BINARY_OP_IN" + value: + str_array: + value: ["entity_1", "entity_2", "unknown"] +``` + +### HAVING and NOT_HAVING +HAVING and NOT_HAVING allow multi-value to be the operand such as array/vector, i.e. one-to-many relationship. For example, "keyA" contains "valueA" and "valueB" + +```shell +criteria: + condition: + name: "extended_tags" + op: "BINARY_OP_HAVING" + value: + strArray: + value: ["c", "b"] +``` + +### MATCH +MATCH performances a full-text search if the tag is analyzed. +The string value applies to the same analyzer as the tag, but string array value does not. +Each item in a string array is seen as a token instead of a query expression. + +How to set the analyzer for a tag can find in the [IndexRules](../schema/index-rule.md). + +```shell +criteria: + condition: + name: "name" + op: "BINARY_OP_MATCH" + value: + str: + value: "us" +``` + +## [LogicalExpression.LogicalOp](../../../api-reference.md#logicalexpressionlogicalop) +Logical operation is used to combine multiple conditions. + +### AND, OR +The following example queries the data where the `id` is `1` and the `service_id` is `service_1` + +```shell +criteria: + le: + op: "LOGICAL_OP_AND" + right: + condition: + name: "id" + op: "BINARY_OP_EQ" + value: + str: + value: "1" + left: + condition: + name: "service_id" + op: "BINARY_OP_EQ" + value: + str: + value: "service_1" +``` diff --git a/docs/interacting/bydbctl/query/measure.md b/docs/interacting/bydbctl/query/measure.md new file mode 100644 index 000000000..5baa76ba5 --- /dev/null +++ b/docs/interacting/bydbctl/query/measure.md @@ -0,0 +1,223 @@ +# Query [Measures](../../../concept/data-model.md#measures) + +Query operation queries the data in a measure. + +[bydbctl](../bydbctl.md) is the command line tool in examples. + +The input contains two parts: + +* Request: a YAML-based text which is defined by the [API](#api-reference) +* Time Range: YAML and CLI's flags both support it. + +## Time Range + +The query specification contains `time_range` field. The request should set absolute times to it. +`bydbctl` also provides `start` and `end` flags to support passing absolute and relative times. + +"start" and "end" specify a time range during which the query is performed, they can be an absolute time like ["2006-01-02T15:04:05Z07:00"](https://www.rfc-editor.org/rfc/rfc3339), +or relative time (to the current time) like "-30m", or "30m". +They are both optional and their default values follow the rules below: + +* when "start" and "end" are both absent, "start = now - 30 minutes" and "end = now", +namely past 30 minutes; +* when "start" is absent and "end" is present, this command calculates "start" (minus 30 units), +e.g. "end = 2022-11-09T12:34:00Z", so "start = end - 30 minutes = 2022-11-09T12:04:00Z"; +* when "start" is present and "end" is absent, this command calculates "end" (plus 30 units), +e.g. "start = 2022-11-09T12:04:00Z", so "end = start + 30 minutes = 2022-11-09T12:34:00Z". + +## Understand the schema you are querying +Before querying the data, you need to know the measure name and the tag families and fields in the measure. You can use the `bydbctl measure get` command to get the measure schema. +If you want to get the schema of a measure named `service_cpm_minute` in the group `measure-minute`, you can use the below command: +```shell +bydbctl measure get -g measure-minute -n service_cpm_minute +``` +```shell +measure: + entity: + tagNames: + - entity_id + fields: + - compressionMethod: COMPRESSION_METHOD_ZSTD + encodingMethod: ENCODING_METHOD_GORILLA + fieldType: FIELD_TYPE_INT + name: value + - compressionMethod: COMPRESSION_METHOD_ZSTD + encodingMethod: ENCODING_METHOD_GORILLA + fieldType: FIELD_TYPE_INT + name: total + interval: 1m + metadata: + createRevision: "206" + group: measure-minute + id: 0 + modRevision: "206" + name: service_cpm_minute + tagFamilies: + - name: storage-only + tags: + - indexedOnly: false + name: entity_id + type: TAG_TYPE_STRING + updatedAt: null +``` + +## Examples +The following examples use above schema to show how to query data in a measure and cover some common use cases: + +### Query between specific time range +To retrieve a series of data points between `2022-10-15T22:32:48Z` and `2022-10-15T23:32:48Z` could use the below command. These data points contain tags: `id` and `entity_id` that belong to a family `default`. They also choose fields: `total` and `value`. + +```shell +bydbctl measure query -f - <