Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Capture RDKafka Statistics as an internal built-in log #6131

Merged
merged 17 commits into from
Mar 23, 2021
Merged

Capture RDKafka Statistics as an internal built-in log #6131

merged 17 commits into from
Mar 23, 2021

Commits on Mar 17, 2021

  1. Log basic metrics instead of extended statistics

    @cirego
    cirego committed Mar 17, 2021
    Configuration menu
    Copy the full SHA
    93fb82a View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into chris/system-table-kafka-metrics

    @cirego
    cirego committed Mar 17, 2021
    Configuration menu
    Copy the full SHA
    dc50905 View commit details
    Browse the repository at this point in the history

  3. Add mz_kafka_statistics built-in log / source

    @cirego
    cirego committed Mar 17, 2021
    Configuration menu
    Copy the full SHA
    ee80890 View commit details
    Browse the repository at this point in the history

Commits on Mar 18, 2021

  1. Remove change for debugging

    @cirego
    cirego committed Mar 18, 2021
    Configuration menu
    Copy the full SHA
    23c3970 View commit details
    Browse the repository at this point in the history

  2. Put consumer in the builtin log name

    @cirego
    cirego committed Mar 18, 2021
    Configuration menu
    Copy the full SHA
    0388f52 View commit details
    Browse the repository at this point in the history

  3. Tracking kafka consumer statistics now works 

    Instead of trying to create a new Logger class that implements Send +
Sync, use crossbeam to send messages from the callback context into the
source info context. This will activate the source, triggering the
source to read messages from the callback and update the system log
tables. When a source is dropped, the statistics are removed.
    @cirego
    cirego committed Mar 18, 2021
    Configuration menu
    Copy the full SHA
    8945be4 View commit details
    Browse the repository at this point in the history

  4. Fixup test_persistence

    @cirego
    cirego committed Mar 18, 2021
    Configuration menu
    Copy the full SHA
    f73c6f6 View commit details
    Browse the repository at this point in the history

Commits on Mar 22, 2021

  1. Merge branch 'main' into chris/system-table-kafka-metrics

    @cirego
    cirego committed Mar 22, 2021
    Configuration menu
    Copy the full SHA
    045293d View commit details
    Browse the repository at this point in the history

  2. Log per-partition kafka consumer metrics 

    Fixes #5666
    @cirego
    cirego committed Mar 22, 2021
    Configuration menu
    Copy the full SHA
    9f34b96 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into chris/system-table-kafka-metrics

    @cirego
    cirego committed Mar 22, 2021
    Configuration menu
    Copy the full SHA
    f09fe85 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' into chris/system-table-kafka-metrics

    @cirego
    cirego committed Mar 22, 2021
    Configuration menu
    Copy the full SHA
    dbec829 View commit details
    Browse the repository at this point in the history

  5. Derive default values for stats object

    @cirego
    cirego committed Mar 22, 2021
    Configuration menu
    Copy the full SHA
    3713dbe View commit details
    Browse the repository at this point in the history

  6. Merge branch 'main' into chris/system-table-kafka-metrics

    @cirego
    cirego committed Mar 22, 2021
    Configuration menu
    Copy the full SHA
    1c38647 View commit details
    Browse the repository at this point in the history

  7. Remove commented out log line 

    We no longer need the print message (that's what this PR replaces!)
    @cirego
    cirego authored Mar 22, 2021
    Configuration menu
    Copy the full SHA
    77d3c5e View commit details
    Browse the repository at this point in the history

Commits on Mar 23, 2021

  1. Add lo/hi/ls offsets

    @cirego
    cirego committed Mar 23, 2021
    Configuration menu
    Copy the full SHA
    03a0928 View commit details
    Browse the repository at this point in the history

  2. Fix source id and verify join against mz_source_info

    @cirego
    cirego committed Mar 23, 2021
    Configuration menu
    Copy the full SHA
    d09093e View commit details
    Browse the repository at this point in the history

  3. Add dataflow ID to correctly identify source

    @cirego
    cirego committed Mar 23, 2021
    Configuration menu
    Copy the full SHA
    e871578 View commit details
    Browse the repository at this point in the history