Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add pipeline_stage to remove filename label #455

Closed
wants to merge 7 commits into from

Conversation

Abuelodelanada
Copy link
Contributor

@Abuelodelanada Abuelodelanada commented Sep 13, 2024

Issue

This PR goes in tandem with canonical/grafana-agent-operator#180

Solution

We add pipeline_stages to promtail configuration in order to remove filename label an add it as structured_metadata

Testing Instructions

Deploy this bundle (update lib in flog and pack loki)

bundle: kubernetes
applications:
  flog:
    charm: flog-k8s
    channel: latest/edge
    revision: 8
    resources:
      workload-image: 2
    scale: 1
    constraints: arch=amd64
  grafana:
    charm: grafana-k8s
    channel: latest/edge
    revision: 118
    resources:
      grafana-image: 70
      litestream-image: 45
    scale: 1
    constraints: arch=amd64
    storage:
      database: kubernetes,1,1024M
    trust: true
  loki:
    charm: local:loki-k8s-0
    scale: 1
    constraints: arch=amd64
    storage:
      active-index-directory: kubernetes,1,1024M
      loki-chunks: kubernetes,1,1024M
    trust: true
  traefik:
    charm: traefik-k8s
    channel: latest/edge
    revision: 207
    resources:
      traefik-image: 160
    scale: 1
    constraints: arch=amd64
    storage:
      configurations: kubernetes,1,1024M
    trust: true
relations:
- - traefik:ingress-per-unit
  - loki:ingress
- - grafana:ingress
  - traefik:traefik-route
- - grafana:grafana-source
  - loki:grafana-source
- - flog:log-proxy
  - loki:logging

Check in Grafana that filename is not a regular label but part of structured_metadata

image

image

@Abuelodelanada Abuelodelanada changed the title add pipeline_stage for consistency with https://github.com/canonical/grafana-agent-operator/pull/180 add pipeline_stage to remove filename label Sep 13, 2024
@Abuelodelanada Abuelodelanada marked this pull request as ready for review September 13, 2024 12:34
@Abuelodelanada Abuelodelanada requested a review from a team as a code owner September 13, 2024 12:34
@Abuelodelanada
Copy link
Contributor Author

Since we have learn something that was not documented in Grafana docs, which is that grafana-agent is not able to forward structured_metadata coming from promtail (or another grafana-agent) to loki

promtail --> grafana-agent --> loki

We are closing this PR because the pipeline is not reliable. An administrator may wrongly think that because promtail sends structured_metadata it will reach loki when it is not always true.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants