Release v0.8.0-nightly-20240318

What's Changed

• feat: acquire all locks in procedure by @WenyXu in #3514
• docs: revise README file by @tisonkun in #3522
• feat: implement the drop database parser by @WenyXu in #3521
• feat(mito): Checks whether a region should flush periodically by @evenyag in #3459

Full Changelog: v0.7.1...v0.8.0-nightly-20240318

Release v0.7.1

What's Changed

This release contains an important bug fix while decoding the Prometheus remote write proto #3505.

It's highly recommended to upgrade to this version if you're using v0.7.

Features

• feat: impl some "set"s to adapt to some client apps by @MichaelScofield in #3443
• feat: update dashboard to v0.4.8 by @ZonaHex in #3450
• feat: max-txn-ops option by @fengjiachun in #3458
• feat(influxdb): add db query param support for v2 write api by @etolbakov in #3445
• feat: support first_value/last_value in range query by @Taylor-lagrange in #3448
• feat: clamp function by @waynexia in #3465
• feat(fuzz): validate columns by @WenyXu in #3485
• feat: to_timezone function by @tisonkun in #3470
• feat(flow): accumlator for aggr func by @discord9 in #3396
• feat(flow): plan def by @discord9 in #3490
• feat: improve prom write requests decode performance by @v0y4g3r in #3478
• feat(fuzz): add insert target by @zhongzc in #3499
• feat(fuzz): add alter table target by @WenyXu in #3503
• feat: support decode gzip if influxdb write specify it by @tisonkun in #3494
• feat: Partition memtables by time if compaction window is provided by @evenyag in #3501

Fixes

• fix: add support for influxdb basic auth by @shuiyisong in #3437
• fix: fix incorrect COM_STMT_PREPARE reply by @WenyXu in #3463
• fix: impl RecordBatchStream method explicitly by @shuiyisong in #3482
• fix: make max-txn-ops limit valid by @fengjiachun in #3481
• fix: fix f64 has no sufficient precision during parsing by @WenyXu in #3483
• fix: freeze data buffer in shard by @evenyag in #3468
• fix: allow passing extra table options by @evenyag in #3484
• fix(common-time): allow building nanos timestamp from parts split from i64::MIN by @zhongzc in #3493
• fix: adjust fill behavior of range query by @Taylor-lagrange in #3489
• fix: decoding prometheus remote write proto doesn't reset the value by @waynexia in #3505
• fix: correctly generate sequences when the value is pre-existed by @MichaelScofield in #3502

Breaking Changes

• fix!: remove error message from http header to avoid panic by @sunng87 in #3506

Others

• build(deps): upgrade opendal to 0.45.1 by @tisonkun in #3432
• chore: add bin opt to build cmd by @WenyXu in #3440
• chore: specify binary name by @WenyXu in #3449
• test: add fuzz test for create table by @WenyXu in #3441
• refactor: separate the quote char and value by @WenyXu in #3455
• perf: Reduce decode overhead during pruning keys in the memtable by @evenyag in #3415
• chore: remove repetitive words by @gcmutator in #3469
• refactor: introduce new Output with OutputMeta by @shuiyisong in #3466
• refactor: make http api returns non-200 status code by @crwen in #3473
• ci: use another mirror for etcd image by @waynexia in #3486
• perf: more benchmarks for memtables by @evenyag in #3491
• refactor: validate constraints eagerly by @tisonkun in #3472
• ci: attempt to setup docker cache for etcd by @sunng87 in #3488
• chore: bump version to v0.7.1 by @evenyag in #3510
• docs: add v0.7 TSBS benchmark result by @evenyag in #3512
• ci: add bin options to reduce build burden by @WenyXu in #3518

New Contributors

• @gcmutator made their first contribution in #3469
• @crwen made their first contribution in #3473

Full Changelog: v0.7.0...v0.7.1

Release v0.7.0

What's Changed

Highlights

• Metric Engine: handles a large number of small tables, making it particularly suitable for cloud-native monitoring.
• Region Migration: enhances the user experience and simplifies region migrations with straightforward SQL commands.
• Inverted Index: dramatically improves the efficiency of locating data segments relevant to user queries, significantly reducing the IO operations needed for scanning data files and thus accelerating the query process.

Features:

• feat: add modulo function by @etolbakov in #3147
• feat: support HTTP&gRPC&pg set timezone by @Taylor-lagrange in #3125
• feat(mito): enable inverted index by @zhongzc in #3158
• feat: upgrade pgwire to 0.19 by @sunng87 in #3157
• feat: let tables API return a stream by @fengjiachun in #3170
• feat: add tests-fuzz crate by @WenyXu in #3173
• feat: tables stream with CatalogManager by @fengjiachun in #3180
• feat: adds date_format function by @killme2008 in #3167
• feat: auto config cache size according to memory size by @QuenKar in #3165
• feat: precise filter for mito parquet reader by @waynexia in #3178
• feat: adds parse options for SQL parser by @killme2008 in #3193
• feat(tests-fuzz): add CreateTableExprGenerator & AlterTableExprGenerator by @WenyXu in #3182
• feat: make procedure able to return output by @WenyXu in #3201
• feat: copy database from by @v0y4g3r in #3164
• feat: introduce information schema provider cache by @WenyXu in #3208
• feat: lazy initialize vector builder on write by @waynexia in #3210
• feat: make query be aware of timezone setting by @killme2008 in #3175
• feat: add create alter table expr translator by @WenyXu in #3203
• feat: read column and region info from state cache by @waynexia in #3222
• feat: enable concurrent write by @WenyXu in #3214
• feat: add Arrow IPC output format for http rest api by @sunng87 in #3177
• feat: change Range Query’s default align behavior aware of timezone by @killme2008 in #3219
• feat: update dashboard to v0.4.7 by @ZonaHex in #3229
• feat: http sql api return schema on empty resultset by @sunng87 in #3237
• feat: add pg create alter table expr translator by @WenyXu in #3206
• feat: read metadata from write cache by @QuenKar in #3224
• feat: batch create ddl by @fengjiachun in #3194
• feat: add insert/select generator & translator by @WenyXu in #3240
• feat: return request outdated error on handling alter by @waynexia in #3239
• feat: don't map semantic type in metric engine by @waynexia in #3243
• feat: create tables in batch on prom write by @fengjiachun in #3246
• feat: Only allow inserts and deletes operations to be executed in parallel by @fengjiachun in #3257
• feat: basic types for Dataflow Framework by @discord9 in #3186
• feat: initial configuration for grafana dashboard by @waynexia in #3263
• feat: support fraction part in timestamp by @discord9 in #3272
• feat: use simple filter to prune memtable by @waynexia in #3269
• feat: Basic Definitions for Expression&Functions for Dataflow by @discord9 in #3267
• feat: support cache for batch_get in CachedMetaKvBackend by @fengys1996 in #3277
• feat: put all filter exprs in a filter plan separately by @waynexia in #3288
• feat: administration functions by @killme2008 in #3236
• feat(mito): adjust seg size of inverted index to finer granularity instead of row group level by @zhongzc in #3289
• feat(mito): add options to ignore building index for specific column ids by @zhongzc in #3295
• feat: organize tracing on query path by @waynexia in #3310
• feat: impl partitions and region_peers information schema by @killme2008 in #3278
• feat: batch get physical table routes by @fengjiachun in #3319
• feat: Defines structs in the merge tree memtable by @evenyag in #3326
• feat(metric-engine): set index options for data region by @zhongzc in #3330
• feat: data buffer and related structs by @v0y4g3r in #3329
• feat: Implement KeyDictBuilder for the merge tree memtable by @evenyag in #3334
• feat: replace pk index with pk_weight during freeze by @v0y4g3r in #3343
• feat: merge tree data parts by @v0y4g3r in #3346
• feat: Defines more structs and methods for a partitioned merge tree by @evenyag in #3348
• feat(flow): impl ScalarExpr&Scalar Function by @discord9 in #3283
• feat: impl migrate_region and procedure_state SQL function by @killme2008 in #3325
• feat: add isnull function by @KKould in #3360
• feat: skip filling NULL for put and delete requests by @waynexia in #3364
• feat: impl merge reader for DataParts by @v0y4g3r in #3361
• feat: Implement write and fork for the new memtable by @evenyag in #3357
• feat: distinguish between different read paths by @v0y4g3r in #3369
• feat: Add freeze and fork method to the memtable by @evenyag in #3374
• feat: merge tree dedup reader by @v0y4g3r in #3375
• feat: Implement iter for the new memtable by @evenyag in #3373
• feat: Implement dedup for the new memtable and expose the config by @evenyag in #3377
• feat: change how region id maps to region worker by @waynexia in #3384
• feat: make tls certificates/keys reloadable (part 1) by @sunng87 in #3335
• feat(grafana): enable shared tooltip, add raft engine throughput by @waynexia in #3387
• feat: Implement partition eviction and only add value size to write buffer size by @evenyag in #3393
• feat: enable zstd compression and encodings in merge tree data part by @v0y4g3r in #3380
• feat(flow): impl for MapFilterProject by @discord9 in #3359
• feat: flush or compact table and region functions by @killme2008 in #3363
• feat: Use a partition level map to look up pk index by @evenyag in #3400
• feat(index): measure memory usage in global instead of single-column and add metrics by @zhongzc in #3383
• feat: enable ArrowFlight compression by @tisonkun in #3403
• feat: zero copy on split rows by @fengjiachun in #3407
• feat: Support automatic DNS lookup for kafka bootstrap servers by @J0HN50N133 in #3379
• feat: add configuration for tls watch option by @sunng87 in #3395
• feat: employ sparse key encoding for shard lookup by @v0y4g3r in #3410
• feat: support Create Table ... Like by @KKould in #3372
• feat: tableref cache by @fengjiachun in #3420
• feat: add verbose support for tql explain/an...

Release v0.7.0-nightly-20240304

What's Changed

• refactor: change the receivers of merge tree components by @v0y4g3r in #3378
• fix: resets dict builder keys counter and avoid unnecessary pruning by @evenyag in #3386
• feat: change how region id maps to region worker by @waynexia in #3384
• chore: skip reorder workspace tables in taplo by @waynexia in #3388
• feat: make tls certificates/keys reloadable (part 1) by @sunng87 in #3335
• feat(grafana): enable shared tooltip, add raft engine throughput by @waynexia in #3387
• test: range fix in modulo function tests by @dimbtp in #3389
• ci: add builder result outputs in release action by @daviderli614 in #3381
• fix: throw errors instead of panic by @WenyXu in #3391
• feat: Implement partition eviction and only add value size to write buffer size by @evenyag in #3393
• feat: enable zstd compression and encodings in merge tree data part by @v0y4g3r in #3380
• feat!: new partition grammar - parser part by @waynexia in #3347
• feat(flow): impl for MapFilterProject by @discord9 in #3359
• feat: flush or compact table and region functions by @killme2008 in #3363
• chore: retry fetch dashboard assets by @tisonkun in #3394
• feat: Use a partition level map to look up pk index by @evenyag in #3400
• chore: add metris for memtable read path by @v0y4g3r in #3397
• build: do not retry for connrefused by @tisonkun in #3402
• refactor: refactor TableRouteManager by @WenyXu in #3392
• feat(index): measure memory usage in global instead of single-column and add metrics by @zhongzc in #3383
• fix: some read metrics by @v0y4g3r in #3404
• feat: enable ArrowFlight compression by @tisonkun in #3403
• refactor: move some costly methods in DataBuffer::read out of read lock by @v0y4g3r in #3406
• feat: zero copy on split rows by @fengjiachun in #3407
• feat: Support automatic DNS lookup for kafka bootstrap servers by @J0HN50N133 in #3379
• fix: partition region id by @fengjiachun in #3414
• fix: show table names not complete from information_schema by @killme2008 in #3417
• feat: add configuration for tls watch option by @sunng87 in #3395
• feat: employ sparse key encoding for shard lookup by @v0y4g3r in #3410
• fix: mitigate memory spike during startup by @niebayes in #3418
• feat: support Create Table ... Like by @KKould in #3372
• feat: tableref cache by @fengjiachun in #3420
• feat: add verbose support for tql explain/analyze by @etolbakov in #3390
• feat: reduce a clone of string by @fengjiachun in #3422
• docs: update pull_request_template.md by @tisonkun in #3421

New Contributors

• @J0HN50N133 made their first contribution in #3379

Full Changelog: v0.7.0-nightly-20240226...v0.7.0-nightly-20240304

Release v0.7.0-nightly-20240226

What's Changed

• refactor: make grpc service able to be added dynamically by @MichaelScofield in #3160
• feat: support HTTP&gRPC&pg set timezone by @Taylor-lagrange in #3125
• feat(mito): enable inverted index by @zhongzc in #3158
• feat: upgrade pgwire to 0.19 by @sunng87 in #3157
• test: engine with write cache by @QuenKar in #3163
• docs: add tracking issue for inverted-index RFC by @tisonkun in #3169
• docs: Update README.md by @tisonkun in #3168
• feat: let tables API return a stream by @fengjiachun in #3170
• feat: add tests-fuzz crate by @WenyXu in #3173
• feat: tables stream with CatalogManager by @fengjiachun in #3180
• chore: expose promql test to distributed instance by @niebayes in #3176
• feat: adds date_format function by @killme2008 in #3167
• refactor: remove TableEngine by @waynexia in #3181
• fix: change back GREPTIME_DB_HEADER_NAME header key name by @shuiyisong in #3184
• chore: carry metrics in flight metadata from datanode to frontend by @shuiyisong in #3113
• refactor: expose region edit in mito engine by @MichaelScofield in #3179
• feat: auto config cache size according to memory size by @QuenKar in #3165
• refactor: introduce common-wal to aggregate wal stuff by @niebayes in #3171
• feat: precise filter for mito parquet reader by @waynexia in #3178
• fix(index): S3 EntityTooSmall error by @zhongzc in #3192
• docs: update SDK links in README.md by @nicecui in #3156
• feat: adds parse options for SQL parser by @killme2008 in #3193
• feat(tests-fuzz): add CreateTableExprGenerator & AlterTableExprGenerator by @WenyXu in #3182
• refactor: reduce number of parquet metadata reads and enable reader buffer by @WenyXu in #3197
• feat: make procedure able to return output by @WenyXu in #3201
• refactor: read parquet metadata by @QuenKar in #3199
• chore: bump opendal to v0.44.2 by @WenyXu in #3209
• feat: copy database from by @v0y4g3r in #3164
• feat: introduce information schema provider cache by @WenyXu in #3208
• feat: lazy initialize vector builder on write by @waynexia in #3210
• fix: remove name matcher from processed matcher list by @waynexia in #3213
• chore: change default factor to compute memory size by @shuiyisong in #3211
• feat: make query be aware of timezone setting by @killme2008 in #3175
• feat!: switch prom remote write to metric engine by @waynexia in #3198
• feat: add create alter table expr translator by @WenyXu in #3203
• fix: fix default value cannot accept negative number by @WenyXu in #3217
• refactor!: rename initialize_region_in_background to init_regions_in_background by @WenyXu in #3216
• feat: read column and region info from state cache by @waynexia in #3222
• feat: enable concurrent write by @WenyXu in #3214
• fix: only register region keeper while creating physical table by @WenyXu in #3223
• refactor: make http server built flexibly by @MichaelScofield in #3225
• feat: add Arrow IPC output format for http rest api by @sunng87 in #3177
• fix: fix MockInstance rebuild issue by @WenyXu in #3218
• feat: change Range Query’s default align behavior aware of timezone by @killme2008 in #3219
• feat: update dashboard to v0.4.7 by @ZonaHex in #3229
• fix: security update for shlex and h2 by @sunng87 in #3227
• feat: http sql api return schema on empty resultset by @sunng87 in #3237
• ci: make git's "safe.directory" accept all by @MichaelScofield in #3234
• feat: add pg create alter table expr translator by @WenyXu in #3206
• ci: adding DOCKER_BUILD_ROOT docker arg for dev build by @MichaelScofield in #3241
• feat: read metadata from write cache by @QuenKar in #3224
• fix: fix create table ddl return incorrect table id by @WenyXu in #3232
• fix: init parquet reader metrics twice by @evenyag in #3242
• feat: batch create ddl by @fengjiachun in #3194
• feat: add insert/select generator & translator by @WenyXu in #3240
• feat: return request outdated error on handling alter by @waynexia in #3239
• feat: don't map semantic type in metric engine by @waynexia in #3243
• docs: RFC of Dataflow Framework by @discord9 in #3185
• test: align chrono and some forked test deps to upstream by @sunng87 in #3245
• fix: IntermediateWriter closes underlying writer twice by @waynexia in #3248
• feat: create tables in batch on prom write by @fengjiachun in #3246
• chore: adjust MySQL connection log by @shuiyisong in #3251
• ci: merge doc label actor and checker tasks by @sunng87 in #3252
• build: make binary name a Dockerfile "ARG" by @MichaelScofield in #3254
• fix: decouple columns in projection and prune by @waynexia in #3253
• chore: switch to free machine by @WenyXu in #3256
• feat: Only allow inserts and deletes operations to be executed in parallel by @fengjiachun in #3257
• chore: adjust storage engine related metrics by @waynexia in #3261
• fix(Copy From): fix incorrect type casts by @WenyXu in #3264
• feat: basic types for Dataflow Framework by @discord9 in #3186
• feat: initial configuration for grafana dashboard by @waynexia in #3263
• fix: SQL insertion and column default constraint aware of timezone by @killme2008 in #3266
• fix: cli export database default value by @tisonkun in #3259
• ci: build GreptimeDB binary for later use by @MichaelScofield in #3244
• refactor: add same SST files by @MichaelScofield in #3270
• feat: support fraction part in timestamp by @discord9 in #3272
• fix: use fe_opts after setup_frontend_plugins in standalone by @shuiyisong in #3275
• ci: fix nightly build by @MichaelScofield in #3276
• feat: use simple filter to prune memtable by @waynexia in #3269
• feat: Basic Definitions for Expression&Functions for Dataflow by @discord9 in #3267
• fix: fix incorrect StatusCode parsing by @WenyXu in #3281
• chore: share cache corss jobs by @WenyXu in #3284
• chore: start plugins during standalone startup & comply with current catalog while changing database by @shuiyisong in #3282
• feat: support cache for batch_get in CachedMetaKvBackend by @fengys1996 in #3277
• feat: put all filter exprs in a filter plan separately by @waynexia in https://github.com/GreptimeTeam/gr...

Release v0.7.0-nightly-20240115

What's Changed

• feat: add modulo function by @etolbakov in #3147

Full Changelog: v0.6.0...v0.7.0-nightly-20240115

Release v0.6.0

What's Changed

Highlights

• Region migration is now available, allowing users to move regions between the datanodes.

Features:

• feat: support PromQL function vector by @waynexia in #3036
• feat(mito): Init the write cache in datanode by @evenyag in #3100
• feat: support grpc for otlp trace and metrics by @fengys1996 in #3105
• feat: add options to enable log recycle and periodical fsync by @v0y4g3r in #3114
• feat: add parquet metadata to cache by @QuenKar in #3097
• feat(python): Print Python interpreter version by @tisonkun in #3118
• feat(inverted_index): introduce SstIndexCreator by @zhongzc in #3107
• feat: update our cross-schema check to cross catalog by @sunng87 in #3123
• feat: expose the region migration replay_timeout argument by @WenyXu in #3129
• feat: adds runtime_metrics by @killme2008 in #3127
• feat: write and upload sst by @QuenKar in #3106
• feat(parquet): introduce inverted index applier to reader by @zhongzc in #3130
• feat: handle drop request for metric table by @waynexia in #3136
• feat: manager kafka cluster in sqlness runner by @waynexia in #3143
• feat: read data from write cache by @QuenKar in #3128
• feat: support PromQL operations over the same metric by @waynexia in #3124
• feat(mito): support write cache for index file by @zhongzc in #3144

Fixes:

• fix: expose unsupported datatype error on MySQL protocol by @waynexia in #3121
• fix: unstable time record test by @zhongzc in #3131
• fix: incorrect parsing broker_endpoints env variable by @WenyXu in #3135
• fix: remove incorrect wal comments in the config file by @niebayes in #3142
• fix: save code in debug_assert! by @WenyXu in #3137
• fix(kafka): overwrite the EntryId with Offset while consuming records by @WenyXu in #3148
• fix: print a detailed error by @WenyXu in #3146
• fix: fix store all wal options by @WenyXu in #3149
• fix: handle non-identical time index and field column in PromQL set operation by @waynexia in #3145
• fix: fix tests failed on Windows by @WenyXu in #3155

Others/Chores:

• chore: add bound check for raft-engine logstore by @v0y4g3r in #3073
• chore(remote_wal): remove topic alias by @niebayes in #3120
• chore: change the default doc checkbox to no need by @waynexia in #3122
• chore: stop exposing num_partitions by @niebayes in #3132
• chore: impl KvBackend for MetaPeerClient by @lyang24 in #3076
• chore: bump version to 0.6.0 by @WenyXu in #3154

Full Changelog: v0.5.1...v0.6.0

Release v0.5.1

Highlights

• Enhancing the information_schema with new additions like character sets, collations, events, and more.
• Efforts in Kafka integration show a move towards better support for distributed systems.
• Continuous improvements in WAL handling demonstrate a commitment to data integrity and performance.

Features

• Database Timezone Support: Support set timezone in db by @Taylor-lagrange (#2992)
• Statement Caching: Upgrade pgwire to 0.18 for corrected statement caching by @sunng87 (#3010)
• Region Migration Management: Implement region migration manager and related task handling by @WenyXu (#3014, #3015)
• PromQL Enhancements: Implement OR for PromQL by @waynexia (#3024)
• Information Schema Extensions: Adds character_sets, collations, events, and additional tables to information_schema by @killme2008 and @dimbtp (#3017, #3051, #3054, #3057, #3060)
• Kafka Integration: Enhance Kafka integration with utilities and tests by @WenyXu (#3065, #3069, #3071)

Fixes

• CI Repository Targeting: Fix CI target repo to the origin one by @waynexia (#3011)
• Decimal128 Scalar Conversion: Fix decimal128 ScalarValue to Value by @QuenKar (#3019)
• Write-Ahead Logging (WAL): Replay memtable should start from flushed_entry_id + 1 and fix some known issues by @WenyXu and @niebayes (#3038, #3052)

Performance Improvements

• Dashboard Update: Update dashboard to v0.4.5 for improved performance by @ZonaHex (#3033)
• File Cache Implementation: Implement file cache by @evenyag (#3022)

Others

• Documentation and Refactoring: Various documentation updates, refactoring efforts, and CI process enhancements by multiple contributors (@MichaelScofield, @zyy17, @sunng87, and others)

New Contributors

• @AntiTopQuark made their first contribution in #3031
• @dimbtp made their first contribution in #3054
• @ClSlaid made their first contribution in #3084

Full Changelog: v0.5.0...v0.5.1

Release v0.6.0-nightly-20240101

What's Changed

• fix: change CI target repo to the origin one by @waynexia in #3011
• chore: try to fix size labeller by @sunng87 in #3012
• chore: "fix: revert unfinished route table change" by @MichaelScofield in #3009
• docs: RFC of enclosing column id by @waynexia in #2983
• feat: upgrade pgwire to 0.18 for corrected statement caching by @sunng87 in #3010
• feat: support set timezone in db by @Taylor-lagrange in #2992
• fix: decimal128 ScalarValue to Value by @QuenKar in #3019
• feat: implement param parsing of SubmitRegionMigrationTaskHandler by @WenyXu in #3015
• feat: implement region migration manager by @WenyXu in #3014
• refactor: use string type instead of Option type for '--store-key-prefix' by @zyy17 in #3018
• chore: do not send message for xlarge PR by @waynexia in #3020
• feat: implement handle upgrade region instruction by @WenyXu in #3013
• feat: adds character_sets, collations and events etc. by @killme2008 in #3017
• chore: change all reference from develop to main by @waynexia in #3026
• chore: Update CI badge in README.md by @evenyag in #3028
• feat: Implement OR for PromQL by @waynexia in #3024
• feat: update dashboard to v0.4.5 by @ZonaHex in #3033
• fix(wal): replay memtable should from flushed_entry_id + 1 by @WenyXu in #3038
• feat: expose region migration http endpoint by @WenyXu in #3032
• feat(remote_wal): append a noop record after kafka topic initialization by @WenyXu in #3040
• test(remote_wal): add sqlness with kafka wal by @WenyXu in #3027
• refactor: merge standalone and metasrv table metadata allocators by @MichaelScofield in #3035
• feat: adds information_schema.schemata by @killme2008 in #3051
• chore: attempt to add doc issue in label task by @sunng87 in #3021
• ci: fix duplicatd doc issue by @sunng87 in #3056
• feat(inverted_index): Add applier builder to convert Expr to Predicates (Part 1) by @zhongzc in #3034
• feat(TableRouteValue): add panic notes and type checks by @AntiTopQuark in #3031
• fix(remote_wal): some known issues by @niebayes in #3052
• feat: add information_schema.files by @dimbtp in #3054
• feat: add information_schema.key_column_usage by @dimbtp in #3057

New Contributors

• @AntiTopQuark made their first contribution in #3031
• @dimbtp made their first contribution in #3054

Full Changelog: v0.5.0...v0.6.0-nightly-20240101

Release v0.5.0

What's Changed

Highlights

• The first implementation of Remote WAL
• A new engine, the Metrics Engine, which is designed to address issues caused by numerous small tables in monitoring scenarios

Features

• feat: add downgrade leader region step by @WenyXu in #2792
• feat: add update metadata step for upgrading candidate region by @WenyXu in #2811
• feat: implement PromQL set op AND/UNLESS by @waynexia in #2839
• feat: do not concat batches in MergeReader by @evenyag in #2833
• feat: add update metadata step for rollbacking downgraded region by @WenyXu in #2812
• feat: add upgrade candidate region step by @WenyXu in #2829
• feat: add align to / interval support in range query by @Taylor-lagrange in #2842
• feat(inverted_index.search): add fst applier by @zhongzc in #2851
• feat: support time() and related functions in PromQL by @waynexia in #2854
• feat: sqlness for decimal128 by @QuenKar in #2822
• feat(inverted_index.search): add fst values mapper by @zhongzc in #2862
• feat: drop if exists by @tisonkun in #2859
• feat: handle the downgrade region instruction by @WenyXu in #2855
• feat(inverted_index.search): add index applier by @zhongzc in #2868
• feat: update dashboard to v0.4.2 by @ZonaHex in #2882
• feat!: support table ddl for custom storage by @NiwakaDev in #2733
• feat: impl date_add/date_sub functions by @killme2008 in #2881
• feat(inverted_index.format): add writer by @zhongzc in #2900
• feat: set default debug level of release and dev profiles to 1 by @waynexia in #2916
• feat!: do not get TZ info from server local env by @waynexia in #2905
• feat: builder to vector without resetting by @QuenKar in #2915
• feat: add build() function to return the database build info by @Dysprosium0626 in #2919
• feat: update dashboard to v0.4.3 by @ZonaHex in #2926
• feat: sql with influxdb v1 result format by @fengjiachun in #2917
• feat: upgrade rustls library family, opensrv-mysql and pgwire by @sunng87 in #2927
• feat: add log and tracing layers for Copy From statement by @WenyXu in #2929
• feat: adds some tables to information_schema by @killme2008 in #2935
• feat: create table procedure for metric engine, part 1 by @waynexia in #2943
• feat(inverted_index.create): add read/write for external intermediate files by @zhongzc in #2942
• feat: fetch manifests in concurrent by @WenyXu in #2951
• feat(inverted_index.create): add external sorter by @zhongzc in #2950
• feat(remote_wal): add skeleton for remote wal related to meta srv by @niebayes in #2933
• feat(mito): add skip_wal_replay option to OpenRegionRequest by @WenyXu in #2955
• feat(inverted_index.create): add index creator by @zhongzc in #2960
• feat!: MetricsEngine table route (part 1) by @MichaelScofield in #2952
• feat(remote_wal): add skeleton for remote wal related to datanode by @niebayes in #2941
• feat: allow initializing regions in background by @WenyXu in #2930
• feat: use explicitly set table id by @MichaelScofield in #2945
• feat(metric-engine): open and close metric regions by @waynexia in #2961
• feat: add CatchupRequest to engine by @WenyXu in #2939
• feat: adds build_info table by @killme2008 in #2969
• feat: Support export metric in remote write format by @Taylor-lagrange in #2928
• feat: add a default internal schema by @sunng87 in #2974
• feat: update dashboard to v0.4.4 by @ZonaHex in #2978
• feat(remote_wal): implement topic allocation by @niebayes in #2970
• feat: support fetch ranges in concurrent by @QuenKar in #2959
• feat: add skip_wal_replay to OpenRegion instruction by @WenyXu in #2977
• feat(remote_wal): impl kafka log store by @niebayes in #2971
• feat: add '--store-key-prefix' to support multiple meta instances can use the same etcd backend by @zyy17 in #2988
• feat: support querying metric engine from frontend by @waynexia in #2987
• feat: export runtime metric to promethues by @SSebo in #2985
• feat: adopt human-panic crash reports by @tisonkun in #2996
• feat: convert FileMetaData to ParquetMetaData by @QuenKar in #2980
• feat: make logging to stdout configurable by @MichaelScofield in #3003
• feat: replace ahash with murmur3 on generating tsid by @waynexia in #3007

Refactors

• refactor: replace usage of ArrayData by clone by @QuenKar in #2827
• refactor: DataType name function by @QuenKar in #2836
• refactor: add builder for Frontend by @MichaelScofield in #2849
• refactor: RegionEngine::handle_request always returns affected rows by @tisonkun in #2874
• refactor: use downgrading the region instead of closing region by @WenyXu in #2863
• refactor: make instance started separately by @MichaelScofield in #2911
• refactor: make sequence bounded with max value by @MichaelScofield in #2937
• refactor: remove reduntant .compat() by @zhongzc in #2949
• refactor: greptimedb cluster sqlness test scripts by @daviderli614 in #2947
• refactor: remove unused code for pruning row groups by @zhongzc in #2973
• refactor(remote_wal): entry id usage by @niebayes in #2986
• refactor: hide RegionRoute behind TableRouteValue by @MichaelScofield in #2989
• refactor(remote_wal): add StandaloneWalConfig by @WenyXu in #3002

Fixes

• fix: sort result of sqlness case decimal_ops by @waynexia in #2867
• fix: correct the previously unsuccessful decimal_ops sort result fix by @zhongzc in #2869
• fix: fragile integration tests by @MichaelScofield in #2870
• fix: type conversion rule reverses operands by @waynexia in #2871
• fix: replace opendal PrometheusLayer by @Taylor-lagrange in #2861
• fix: add new column as primary key can't work by @killme2008 in #2876
• fix: mysql version function result by @tisonkun in #2884
• fix: align linear_regression to PromQL's behavior by @waynexia in #2879
• fix: handle heartbeat shutdown gracefully by @tisonkun in #2886
• fix: re-enable ignored case test_query_prepared by @waynexia in #2892
• fix: support tailing commas in SQL by @Taylor-lagrange in #2913
• fix: return error to client if prepare stmt param not match by @MichaelScofield in #2918
• fix: date - interval sqlness by @QuenKar in #2912
• fix: use linear interpolation to implement range...