23.9.1

Various fixes & improvements

• feat(spans): Port the processor to Rust (#4712) by @john-z-yang
• meta: Bump new development version (2432b27)

23.9.0

Various fixes & improvements

• feat(rust): Add processor metrics for Rust consumer (#4737) by @lynnagara
• feat: metrics.increment() no longer supports option (#4738) by @lynnagara
• ref: bump sentry-kafka-schemas to 0.1.27 (#4739) by @getsentry-bot
• Update snuba-sdk version to latest 2.0.1 (#4735) by @enochtangg
• make sure possible API changes are tested against sentry (#4736) by @volokluev
• test: Run different Sentry tests depending on files changed (#4727) by @lynnagara
• fix(ci): no more sentry shards (#4725) by @joshuarli
• fix(CI): Unblock deploys as they hard check for sentry matrix'd tests (#4724) by @rahul-kumar-saini
• fix(on-call): Negative org ID in Tenant IDs crashes Querylog consumer (#4722) by @rahul-kumar-saini
• ref: bump sentry-kafka-schemas to 0.1.26 (#4721) by @getsentry-bot
• test: No need to run Sentry CI in a matrix (#4720) by @lynnagara
• Use snuba-admin spec for ST and SaaS (#4716) by @dbanda
• feat(rust-arroyo): Processor calls consumer.poll() with timeout of 1 second (#4719) by @lynnagara
• experiment(on-call): Profile bytes scanned potentially incorrect (#4714) by @rahul-kumar-saini
• feat(reduce): Avoid clone and ensure all messages are processed in join (#4718) by @lynnagara
• improvement(visibility): make the replacer log project_ids for slow replacements (#4717) by @onewland
• feat(profiling): Add a functions processor in Rust (#4705) by @phacops
• fix(spans): Handle bad group and group_raw values (#4715) by @phacops
• feat(profiling): Add a profiles processor in Rust (#4698) by @phacops
• Remove canary deploys in non-US regions (#4696) by @mattgauntseo-sentry
• feat(spans): write tags to sentry_tags col (#4706) by @dbanda
• sort lists before compare (#4711) by @dbanda
• fix(rust-arroyo): Fix timeout in RunTaskInThreads::join() (#4704) by @lynnagara
• ref(rust): Rewrite check_for_results to use better way of joining process pool (#4703) by @untitaker

Plus 78 more

23.8.0

Various fixes & improvements

• fix(Cache): Cache fail open on ValueError (#4615) by @rahul-kumar-saini
• feat(replays): Add low cardinality encoding to select columns (#4601) by @cmanallen
• feat: Group the validation warnings for each topic separately (#4570) by @lynnagara
• fix(oncall): Query execution dataset metric tag (#4608) by @rahul-kumar-saini
• feat(replays): Add materialized counts for array columns (#4603) by @cmanallen
• search_issues -> group_attributes join relationship (#4588) by @barkbarkimashark
• feat(async-queries): Add ability to override query settings for async queries (#4584) by @davidtsuk
• ref: bump sentry-kafka-schemas to 0.1.25 (#4610) by @getsentry-bot
• fix(ci): improve caching (#4521) by @dbanda
• feat(replay): pass start_time as received header (#4548) by @bmckerry
• fix(capman): rate limit overrides should not collide (#4604) by @volokluev
• feat(db_query): Randomized Query IDs (#4605) by @rahul-kumar-saini
• Remove the test regions from rollbacks (#4599) by @mattgauntseo-sentry
• feat(capman): implement rate limit overrides (#4563) by @volokluev
• pass through subscriptions for new concurrent rate limiter (#4587) by @volokluev
• ci: only render gocd pipelines if relevant files are changed (#4591) by @joshuarli
• bump readiness (#4590) by @barkbarkimashark
• meta: Bump new development version (e7d9945)

23.7.2

Various fixes & improvements

• feat(group_attributes): add ability to join events -> group_attributes (#4586) by @barkbarkimashark
• cleanup(capman): remove project throttler (#4576) by @volokluev
• feat(consumer): add health check step to consumers (#4508) by @dbanda
• fix(capman): Bake in is_enforced behavior into every allocation policy (#4585) by @volokluev
• Update gocd-jsonnet to v1.4.1 (#4579) by @mattgauntseo-sentry
• Revert "feat(db_query): Readthrough Cache entire db_query() pipeline (#4506)" (946c90a) by @getsentry-bot
• ref: bump sentry-arroyo to 2.14.2 (#4569) by @getsentry-bot
• turn on rate limit policy for transactions (#4572) by @volokluev
• feat(db_query): Readthrough Cache entire db_query() pipeline (#4506) by @rahul-kumar-saini
• ref(admin): Upload sourcemaps to Sentry for the admin tool (#4441) by @evanh
• feat(oncall): break down query execution metrics by table and referrer (#4573) by @volokluev
• feat(prod-queries): Add button to view all allowed projects (#4574) by @davidtsuk
• feat(consumer): Add transactions consumer SLO (#4442) by @ayirr7
• feat(schema): add replay_id to table entities and storages (#4565) by @JoshFerge
• ref: bump sentry-kafka-schemas to 0.1.23 (#4571) by @getsentry-bot
• feat(rust-consumer): Port querylog processor to Rust (#4562) by @lynnagara
• feat(group-attributes): Change the readiness state for migration to partial (#4567) by @lynnagara
• Revert back to snuba group for role access (#4550) by @mattgauntseo-sentry
• feat(capman): ratelimit allocation policy (no overrides) (#4536) by @volokluev
• Switch is ST check to library version (#4568) by @mattgauntseo-sentry
• feat(clickhouse): Enable 23.3 (#4566) by @lynnagara
• Add customer 1 and 2 to pipedream (#4564) by @mattgauntseo-sentry
• meta: Bump new development version (24a0003)

23.7.1

Various fixes & improvements

• ref: bump sentry-kafka-schemas to 0.1.22 (#4559) by @getsentry-bot
• ref: Remove manual collection of librdkafka stats (#4542) by @lynnagara
• feat: Banish fetching untyped runtime config (#4540) by @lynnagara
• Fix validation pipeline (#4547) by @mattgauntseo-sentry
• feat(rust-consumer): Add RunTaskInThreads strategy (#4537) by @lynnagara
• ref: Remove unnecessary config (#4541) by @lynnagara
• fix(prod-queries): Remove project validation in debug env (#4549) by @davidtsuk
• Update gocd jsonnet libs and fix rollbacks (#4544) by @mattgauntseo-sentry
• fix(cardinality-analyzer): Fix CSV escaping (#4546) by @gggritso
• build(deps): bump flask from 2.2.2 to 2.2.5 (#4543) by @dependabot
• build(deps): bump werkzeug from 2.2.2 to 2.2.3 (#3771) by @dependabot
• chore: bump sentry-kafka-schemas to 0.1.21 (#4529) by @ayirr7
• ref(snubsplain): Add steps for snql parsing and query mappers (#4486) by @evanh
• feat(group-attributes): expose storage and entity, process messages (#4507) by @barkbarkimashark
• ref: Simplify querylog processor more (#4520) by @lynnagara
• ci: add GoCD pipeline validation (#4528) by @joshuarli
• ref(capman): split the rate limit function into a start and finish phase (#4532) by @volokluev
• ref: Simplify querylog processor (#4511) by @lynnagara
• Add elastic agent to rollback pipeline (#4534) by @mattgauntseo-sentry
• set min_retention_days to retention_days (#4533) by @john-z-yang
• feat(admin): Add new trace logs view (#4510) by @enochtangg
• Revert "Revert "feat(replays): Add click_is_dead and click_is_rage columns (#4470)"" (#4517) by @cmanallen
• Enable Sentry Hub propagation (#4530) by @gggritso
• Fix canaries and migrations in s4s snuba (#4531) by @mattgauntseo-sentry

Plus 1 more

23.7.0

Various fixes & improvements

• Switch to pipedream pipelines (#4526) by @mattgauntseo-sentry
• feat(rust-consumer): Support parallel insert to ClickHouse (#4519) by @lynnagara
• run search_issues migration in self-hosted (#4518) by @hubertdeng123
• bump materialization_version to 2 (#4505) by @john-z-yang
• perf: Allow parallel clickhouse insert (#4516) by @lynnagara
• add facilities to insert aggregation_options in processor (#4509) by @john-z-yang
• feat(subscriptions): add clickhouse connection check (#4512) by @dbanda
• Revert "feat(replays): Add click_is_dead and click_is_rage columns (#4470)" (0788e2a) by @getsentry-bot
• ref: bump sentry-kafka-schemas to 0.1.19 (#4515) by @getsentry-bot
• migration(group_attributes): create new GroupAttributes table that mirrors a subset of columns from sentry (#4496) by @barkbarkimashark
• Allow snuba/clusters/storage_sets.py in migrations check (#4514) by @dbanda
• feat(replays): Add click_is_dead and click_is_rage columns (#4470) by @cmanallen
• add migrations for new distributions matview (#4504) by @john-z-yang
• fix: Actually resubmit carried over message in clickhouse strategy (#4502) by @lynnagara
• feat(replays): Add migration for click_is_dead and click_is_rage columns (#4469) by @cmanallen
• ref: bump sentry-kafka-schemas to 0.1.18 (#4501) by @getsentry-bot
• fix(rust-consumer): Actually commit messages even if we are skipping the clickhouse write (#4500) by @lynnagara
• feat(generic-metrics): Add aggregation options for generic metrics raw tables (#4467) by @john-z-yang
• ref(rust-consumer): Move clickhouse_client out of rust_arroyo (#4499) by @lynnagara
• feat(rust-consumer): Write to ClickHouse (#4490) by @lynnagara
• Fix MultistorageCollector as #4297 (#4475) by @qbx2
• test: Rust CI improvements (#4258) by @lynnagara
• check connections on consumer startup (#4497) by @dbanda
• Update jsonnet lib and add rollback pipeline (#4495) by @mattgauntseo-sentry

Plus 17 more

23.6.2

Various fixes & improvements

• add role to migrations stage (#4424) by @dbanda
• experiment(capman,starfish): add ability to turn off concurrent throttling (#4464) by @volokluev
• feat(dlq): If DLQ replay is kicked off, add it to audit log (#4439) by @lynnagara
• Add script for customer deployments (#4438) by @mattgauntseo-sentry
• feat(oncall): Make readthrough cache fail open on redis errors (#4449) by @volokluev
• ref(admin) Allow colons in system queries so that datetimes are valid (#4462) by @evanh
• perm(cardinality-analyzer): Add ingest team (#4458) by @jjbayer
• use mantine text area instead of rich text editor (#4461) by @enochtangg
• perm(prod-queries): Add production queries tool to product tools (#4460) by @davidtsuk
• fix(spans): Add group_raw to HexIntColumnProcessor (#4459) by @shruthilayaj
• feat(admin): Add syntax highlighting to snuba admin (#4430) by @enochtangg
• feat(admin): Stop blocking text/images on Snuba Admin replays (#4443) by @rahul-kumar-saini
• ref: Clean up print statements (#4447) by @lynnagara
• fix: Revert debounced commits (#4446) by @untitaker
• test(dlq): Add end to end DLQ consumer test (#4367) by @lynnagara
• ref: bump sentry-redis-tools to 0.1.7 (#4445) by @getsentry-bot
• ref: bump sentry-kafka-schemas to 0.1.17 (#4444) by @getsentry-bot
• feat(prod-queries): Implement backend for prod queries (#4398) by @davidtsuk
• ref: Decouple processing batches and clickhouse batches (#4251) by @untitaker
• ref: bump sentry-kafka-schemas to 0.1.16 (#4436) by @getsentry-bot
• build(deps): bump sentry-relay from 0.8.21 to 0.8.27 (#4427) by @dependabot
• perm(cardinality-analyzer) Update cardinality analyzer member list (#4435) by @davidtsuk
• feat(snubsplain): Use request context to capture query processors (#4381) by @evanh
• ref(admin): Set the user email in Sentry (#4431) by @evanh

Plus 65 more

23.6.1

Various fixes & improvements

• fix: Accidentally checked in useless text (#4355) by @lynnagara
• fix(admin): Pin yarn and node using volta (#4349) by @untitaker
• meta: Bump new development version (c2c6e3c)
• feat(dlq): Redesign admin UI, add API endpoints for replaying (#4347) by @lynnagara

23.6.0

Various fixes & improvements

• ref(DRS): Deprecate dataset experimental flag (#4321) by @enochtangg
• Revert "fix: Temporarily revert dlq topic changes" (#4339) by @lynnagara
• fix(iam-policy): Don't build google cloud API client in debug and test environments (#4344) by @davidtsuk
• feat(prod-queries): Add preliminary UI for the production queries tool (#4345) by @davidtsuk
• ref(iam-policy): Replace user emails with groups in iam_policy (#4346) by @davidtsuk
• feat(spans): Add processors to extract millisecond granularity data (#4313) by @nikhars
• ref(gen_met): Add mapping optimizer to generic metrics (#4336) by @evanh
• Add optional Redis SSL support (#4343) by @frank-m
• feat(spans): Add migration to create start_ms and end_ms columns (#4308) by @nikhars
• ref(admin): Give team-sns google group full access to snuba admin (#4337) by @davidtsuk
• feat(CapMan): Multiple Allocation Policies on a Storage [backend] (#4294) by @rahul-kumar-saini
• chore(spans): Allow spans ingestion based on project sampling (#4214) by @nikhars
• fix: Temporarily revert dlq topic changes (#4338) by @lynnagara
• fix(tests): Skip flaky optimize tests (#4315) by @nikhars
• ref(dlq): Simplify the way DLQ topic is configured (#4333) by @lynnagara
• fix: All consumers have unique DLQ topics (#4335) by @lynnagara
• feat(dlq): DLQ topics API for admin (#4334) by @lynnagara
• Use Google Groups API for authorization (#4306) by @davidtsuk
• fix(dlq): Pass for to configure_metrics (#4329) by @lynnagara
• ref: Upgrade arroyo, and remove now-duplicated rdkafka.total_queue_size metric (#4330) by @untitaker
• fix(gen-metrics): Add tags hash column to the generic metrics dist ta… (#4324) by @nikhars
• ref: Set compression in build_kafka_producer_configuration() function (#4220) by @lynnagara
• fix(dlq): ExitAfterNMessages strategy should be first step not last (#4318) by @lynnagara
• feat(spans): Allow spans migrations to run locally (#4323) by @enochtangg

Plus 40 more

23.5.2

Various fixes & improvements

• feat(CapMan): Spans Allocation Policy (#4261) by @rahul-kumar-saini
• feat(CapMan): Allocation Policy UI tests (#4200) by @rahul-kumar-saini
• fix(CapMan): organzation_id -> organization_id (#4267) by @rahul-kumar-saini
• fix(DC): Outcomes Raw Entity was under wrong Dataset (#4263) by @rahul-kumar-saini
• reject search issues bad column (#4260) by @volokluev
• Final bypass for subscriptions (#4253) by @fpacifici
• feat(search-issues): expose and process profile_id and replay_id columns (#4164) by @barkbarkimashark
• fix: Fix flaky optimize test (#4252) by @untitaker
• fix(spans): use system instead of platform (#4244) by @dbanda
• promote profiles and functions to complete (#4238) by @enochtangg
• ref: bump sentry-kafka-schemas to 0.1.10 (#4250) by @getsentry-bot
• feat(CapMan): Most throttled Orgs visibility (#4248) by @rahul-kumar-saini
• feat(DRS): Enable readiness state for storages (#4240) by @enochtangg
• ref(dashboard): Add a setting that restricts the tools visible in the dashboard (#4174) by @evanh
• fix: Add querylog max.message.bytes to topic configuration (#4221) by @lynnagara
• fix(logs) : use structlog in migrations (#4236) by @dbanda
• feat(dlq): Consumer builder can build a DLQ strategy (#4235) by @lynnagara
• ref: bump sentry-arroyo to 2.11.4 (#4243) by @getsentry-bot
• ref: bump sentry-arroyo to 2.11.3 (#4241) by @getsentry-bot
• fix(parsing): Update error message for missing datetime column comparison condition (#4189) by @davidtsuk
• fix: Querylog producer (#4219) by @lynnagara
• fix: Fix attribution producer (#4218) by @lynnagara
• test(dlq): Add a small test for ExitAfterNMessages strategy (#4234) by @lynnagara
• feat(dlq): Add is_valid() method to DlqInstruction class (#4233) by @lynnagara

Plus 12 more