From 043897834f07895a68794d0644e1374b068f40bf Mon Sep 17 00:00:00 2001 From: Jack Baldry Date: Mon, 17 Jun 2024 11:23:00 +0100 Subject: [PATCH 01/18] Rewrite quickstart with Killercoda metadata This allows a tool to parse this source file and regenerate the Killercoda Interactive Learning Environment from its contents. As I was in the file, I linted it for Writers' Toolkit style. Signed-off-by: Jack Baldry --- docs/sources/get-started/quick-start.md | 286 +++++++++++++++++------- 1 file changed, 199 insertions(+), 87 deletions(-) diff --git a/docs/sources/get-started/quick-start.md b/docs/sources/get-started/quick-start.md index f459e564092e1..761b94c791fc7 100644 --- a/docs/sources/get-started/quick-start.md +++ b/docs/sources/get-started/quick-start.md @@ -2,160 +2,248 @@ title: Quickstart to run Loki locally menuTitle: Loki quickstart weight: 200 -description: How to create and use a simple local Loki cluster for testing and evaluation purposes. +description: How to create and use a local Loki cluster for testing and evaluation purposes. +killercoda: + title: Loki Quickstart Demo + description: This sandbox provides an online enviroment for testing the Loki quickstart demo. + details: + finish: + text: finished.md + backend: + imageid: ubuntu --- + + # Quickstart to run Loki locally If you want to experiment with Loki, you can run Loki locally using the Docker Compose file that ships with Loki. It runs Loki in a [monolithic deployment](https://grafana.com/docs/loki//get-started/deployment-modes/#monolithic-mode) mode and includes a sample application to generate logs. The Docker Compose configuration instantiates the following components, each in its own container: -- **flog** a sample application which generates log lines. [flog](https://github.com/mingrammer/flog) is a log generator for common log formats. +- **flog** a sample application which generates log lines. + [flog](https://github.com/mingrammer/flog) is a log generator for common log formats. - **Grafana Alloy** which scrapes the log lines from flog, and pushes them to Loki through the gateway. -- **Gateway** (NGINX) which receives requests and redirects them to the appropriate container based on the request's URL. +- **Gateway** (nginx) which receives requests and redirects them to the appropriate container based on the request's URL. - One Loki **read** component (Query Frontend, Querier). - One Loki **write** component (Distributor, Ingester). - One Loki **backend** component (Index Gateway, Compactor, Ruler, Bloom Compactor (Experimental), Bloom Gateway (Experimental)). - **Minio** an S3-compatible object store which Loki uses to store its index and chunks. - **Grafana** which provides visualization of the log lines captured within Loki. -{{< figure max-width="75%" src="/media/docs/loki/get-started-flog-v3.png" caption="Getting started sample application" alt="Getting started sample application">}} +{{< figure max-width="75%" src="/media/docs/loki/get-started-flog-v3.png" caption="Getting started sample application" alt="Getting started sample application" >}} + + + +## Before you begin + +Use the [Interactive Learning Environment](#interactive-learning-environment) to skip installing Docker and Docker Compose. + +- Install [Docker](https://docs.docker.com/install) +- Install [Docker Compose](https://docs.docker.com/compose/install) ## Interactive Learning Environment {{< admonition type="note" >}} -The Interactive Learning Environment is currently in trial. Please provide feedback, report bugs, and raise issues in the [Grafana Killercoda Repository](https://github.com/grafana/killercoda). +The Interactive Learning Environment is in trial. + +Provide feedback, report bugs, and raise issues in the [Grafana Killercoda repository](https://github.com/grafana/killercoda). {{< /admonition >}} -Try out this demo within our interactive learning environment: [Loki Quickstart Sandbox](https://killercoda.com/grafana-labs/course/loki/loki-quickstart) +Try out this demo within our interactive learning environment: [Loki Quickstart Sandbox](https://killercoda.com/grafana-labs/course/loki/loki-quickstart) -- A free Killercoda account is required to verify you are not a bot. -- Tutorial instructions are located on the left-hand side of the screen. Click to move on to the next section. -- All commands run inside the interactive terminal. Grafana can also be accessed via the URL links provided within the sandbox. +- You must have a free Killercoda account to verify you aren't a bot. +- Tutorial instructions are located on the left-side of the screen. + Click to move on to the next section. +- All commands run inside the interactive terminal. +- You can access Grafana with the URL links provided within the sandbox. -## Installing Loki and collecting sample logs + -Prerequisites +## Install Loki and collecting sample logs -- [Docker](https://docs.docker.com/install) -- [Docker Compose](https://docs.docker.com/compose/install) + -{{% admonition type="note" %}} +{{< admonition type="note" >}} This quickstart assumes you are running Linux. -{{% /admonition %}} +{{< /admonition >}} + + **To install Loki locally, follow these steps:** -1. Create a directory called `evaluate-loki` for the demo environment. Make `evaluate-loki` your current working directory: +1. Create a directory called `evaluate-loki` for the demo environment. + Make `evaluate-loki` your current working directory: + + - ```bash - mkdir evaluate-loki - cd evaluate-loki - ``` + ```bash + mkdir evaluate-loki + cd evaluate-loki + ``` + + 1. Download `loki-config.yaml`, `alloy-local-config.yaml`, and `docker-compose.yaml`: - ```bash - wget https://raw.githubusercontent.com/grafana/loki/main/examples/getting-started/loki-config.yaml -O loki-config.yaml - wget https://raw.githubusercontent.com/grafana/loki/main/examples/getting-started/alloy-local-config.yaml -O alloy-local-config.yaml - wget https://raw.githubusercontent.com/grafana/loki/main/examples/getting-started/docker-compose.yaml -O docker-compose.yaml - ``` + + + ```bash + wget https://raw.githubusercontent.com/grafana/loki/main/examples/getting-started/loki-config.yaml -O loki-config.yaml + wget https://raw.githubusercontent.com/grafana/loki/main/examples/getting-started/alloy-local-config.yaml -O alloy-local-config.yaml + wget https://raw.githubusercontent.com/grafana/loki/main/examples/getting-started/docker-compose.yaml -O docker-compose.yaml + ``` + + 1. Deploy the sample Docker image. - With `evaluate-loki` as the current working directory, start the demo environment using `docker compose`: + With `evaluate-loki` as the current working directory, start the demo environment using `docker compose`: - ```bash - docker compose up -d - ``` + - You should see something similar to the following: + ```bash + docker compose up -d + ``` - ```bash - ✔ Network evaluate-loki_loki Created 0.1s - ✔ Container evaluate-loki-minio-1 Started 0.6s - ✔ Container evaluate-loki-flog-1 Started 0.6s - ✔ Container evaluate-loki-backend-1 Started 0.8s - ✔ Container evaluate-loki-write-1 Started 0.8s - ✔ Container evaluate-loki-read-1 Started 0.8s - ✔ Container evaluate-loki-gateway-1 Started 1.1s - ✔ Container evaluate-loki-grafana-1 Started 1.4s - ✔ Container evaluate-loki-alloy-1 Started 1.4s - ``` + + + You should see something similar to the following: + + ```bash + ✔ Network evaluate-loki_loki Created 0.1s + ✔ Container evaluate-loki-minio-1 Started 0.6s + ✔ Container evaluate-loki-flog-1 Started 0.6s + ✔ Container evaluate-loki-backend-1 Started 0.8s + ✔ Container evaluate-loki-write-1 Started 0.8s + ✔ Container evaluate-loki-read-1 Started 0.8s + ✔ Container evaluate-loki-gateway-1 Started 1.1s + ✔ Container evaluate-loki-grafana-1 Started 1.4s + ✔ Container evaluate-loki-alloy-1 Started 1.4s + ``` 1. (Optional) Verify that the Loki cluster is up and running. - - The read component returns `ready` when you point a web browser at [http://localhost:3101/ready](http://localhost:3101/ready). The message `Query Frontend not ready: not ready: number of schedulers this worker is connected to is 0` will show prior to the read component being ready. - - The write component returns `ready` when you point a web browser at [http://localhost:3102/ready](http://localhost:3102/ready). The message `Ingester not ready: waiting for 15s after being ready` will show prior to the write component being ready. - + + - The read component returns `ready` when you browse to [http://localhost:3101/ready](http://localhost:3101/ready). + The message `Query Frontend not ready: not ready: number of schedulers this worker is connected to is 0` shows until the read component is ready. + - The write component returns `ready` when you browse to [http://localhost:3102/ready](http://localhost:3102/ready). + The message `Ingester not ready: waiting for 15s after being ready` shows until the write component is ready. + 1. (Optional) Verify that Grafana Alloy is running. - - Grafana Alloy's UI can be accessed at [http://localhost:12345](http://localhost:12345). + - You can access the Grafana Alloy UI at [http://localhost:12345](http://localhost:12345). + + + + + +## View your logs in Grafana -## Viewing your logs in Grafana +After you have collected logs, you will want to view them. +You can view your logs using the command line interface, [LogCLI](/docs/loki//query/logcli/), but the easiest way to view your logs is with Grafana. -Once you have collected logs, you will want to view them. You can view your logs using the command line interface, [LogCLI](/docs/loki//query/logcli/), but the easiest way to view your logs is with Grafana. +1. Use Grafana to query the Loki data source. -1. Use Grafana to query the Loki data source. + The test environment includes [Grafana](https://grafana.com/docs/grafana/latest/), which you can use to query and observe the sample logs generated by the flog application. - The test environment includes [Grafana](https://grafana.com/docs/grafana/latest/), which you can use to query and observe the sample logs generated by the flog application. You can access the Grafana cluster by navigating to [http://localhost:3000](http://localhost:3000). The Grafana instance provided with this demo has a Loki [datasource](https://grafana.com/docs/grafana/latest/datasources/loki/) already configured. + You can access the Grafana cluster by browsing to [http://localhost:3000](http://localhost:3000). - {{< figure src="/media/docs/loki/grafana-query-builder-v2.png" caption="Grafana Explore" alt="Grafana Explore">}} + The Grafana instance in this demonstration has a Loki [data source](https://grafana.com/docs/grafana/latest/datasources/loki/) already configured. -1. From the Grafana main menu, click the **Explore** icon (1) to launch the Explore tab. To learn more about Explore, refer the [Explore](https://grafana.com/docs/grafana/latest/explore/) documentation. + {{< figure src="/media/docs/loki/grafana-query-builder-v2.png" caption="Grafana Explore" alt="Grafana Explore" >}} -1. From the menu in the dashboard header, select the Loki data source (2). This displays the Loki query editor. In the query editor you use the Loki query language, [LogQL](https://grafana.com/docs/loki//query/), to query your logs. - To learn more about the query editor, refer to the [query editor documentation](https://grafana.com/docs/grafana/latest/datasources/loki/query-editor/). +1. From the Grafana main menu, click the **Explore** icon (1) to open the Explore tab. + + To learn more about Explore, refer to the [Explore](https://grafana.com/docs/grafana/latest/explore/) documentation. + +1. From the menu in the dashboard header, select the Loki data source (2). + + This displays the Loki query editor. + + In the query editor you use the Loki query language, [LogQL](https://grafana.com/docs/loki//query/), to query your logs. + To learn more about the query editor, refer to the [query editor documentation](https://grafana.com/docs/grafana/latest/datasources/loki/query-editor/). 1. The Loki query editor has two modes (3): - [Builder mode](https://grafana.com/docs/grafana/latest/datasources/loki/query-editor/#builder-mode), which provides a visual query designer. - [Code mode](https://grafana.com/docs/grafana/latest/datasources/loki/query-editor/#code-mode), which provides a feature-rich editor for writing LogQL queries. - Next we’ll walk through a few simple queries using both the builder and code views. - 1. Click **Code** (3) to work in Code mode in the query editor. - Here are some basic sample queries to get you started using LogQL. Note that these queries assume that you followed the instructions to create a directory called `evaluate-loki`. If you installed in a different directory, you’ll need to modify these queries to match your installation directory. After copying any of these queries into the query editor, click **Run Query** (4) to execute the query. + Here are some sample queries to get you started using LogQL. + These queries assume that you followed the instructions to create a directory called `evaluate-loki`. + + If you installed in a different directory, you’ll need to modify these queries to match your installation directory. - 1. View all the log lines which have the container label "flog": + After copying any of these queries into the query editor, click **Run Query** (4) to execute the query. - ```bash - {container="evaluate-loki-flog-1"} - ``` + 1. View all the log lines which have the container label `evaluate-loki-flog-1`: - In Loki, this is called a log stream. Loki uses [labels](https://grafana.com/docs/loki//get-started/labels/) as metadata to describe log streams. Loki queries always start with a label selector. In the query above, the label selector is `container`. + - 1. To view all the log lines which have the container label "grafana": + ```bash + {container="evaluate-loki-flog-1"} + ``` - ```bash - {container="evaluate-loki-grafana-1"} - ``` + - 1. Find all the log lines in the container=flog stream that contain the string "status": + In Loki, this is a log stream. - ```bash - {container="evaluate-loki-flog-1"} |= `status` - ``` + Loki uses [labels](https://grafana.com/docs/loki//get-started/labels/) as metadata to describe log streams. - 1. Find all the log lines in the container=flog stream where the JSON field "status" is "404": + Loki queries always start with a label selector. + In the previous query, the label selector is `{container="evaluate-loki-flog-1"}`. - ```bash - {container="evaluate-loki-flog-1"} | json | status=`404` - ``` + 1. To view all the log lines which have the container label `evaluate-loki-grafana-1`: - 1. Calculate the number of logs per second where the JSON field "status" is "404": + - ```bash - sum by(container) (rate({container="evaluate-loki-flog-1"} | json | status=`404` [$__auto])) - ``` + ```bash + {container="evaluate-loki-grafana-1"} + ``` - The final query above is a metric query which returns a time series. This will trigger Grafana to draw a graph of the results. You can change the type of graph for a different view of the data. Click **Bars** to view a bar graph of the data. + -1. Click the **Builder** tab (3) to return to Builder mode in the query editor. - 1. In Builder view, click **Kick start your query**(5). - 1. Expand the **Log query starters** section. - 1. Select the first choice, **Parse log lines with logfmt parser**, by clicking **Use this query**. - 1. On the Explore tab, click **Label browser**, in the dialog select a container and click **Show logs**. + 1. Find all the log lines in the `{container="evaluate-loki-flog-1}` stream that contain the string `status`: + + + + ```bash + {container="evaluate-loki-flog-1"} |= `status` + ``` + + + + 1. Find all the log lines in the `{container="evaluate-loki-flog-1}` stream where the JSON field `status` has the value `404`: + + + + ```bash + {container="evaluate-loki-flog-1"} | json | status=`404` + ``` + + + + 1. Calculate the number of logs per second where the JSON field `status` has the value `404`: + + + + ```bash + sum by(container) (rate({container="evaluate-loki-flog-1"} | json | status=`404` [$__auto])) + ``` + + + + The final query is a metric query which returns a time series. + This makes Grafana draw a graph of the results. + + You can change the type of graph for a different view of the data. + Click **Bars** to view a bar graph of the data. + +1. Click the **Builder** tab (3) to return to builder mode in the query editor. + 1. In builder mode, click **Kick start your query** (5). + 1. Expand the **Log query starters** section. + 1. Select the first choice, **Parse log lines with logfmt parser**, by clicking **Use this query**. + 1. On the Explore tab, click **Label browser**, in the dialog select a container and click **Show logs**. For a thorough introduction to LogQL, refer to the [LogQL reference](https://grafana.com/docs/loki//query/). @@ -165,40 +253,64 @@ Here are some more sample queries that you can run using the Flog sample data. To see all the log lines that flog has generated, enter the LogQL query: + + ```bash -{container="evaluate-loki-flog-1"}|= `` +{container="evaluate-loki-flog-1"} ``` + + The flog app generates log lines for simulated HTTP requests. To see all `GET` log lines, enter the LogQL query: + + ```bash {container="evaluate-loki-flog-1"} |= "GET" ``` + + To see all `POST` methods, enter the LogQL query: + + ```bash {container="evaluate-loki-flog-1"} |= "POST" ``` + + To see every log line with a 401 status (unauthorized error), enter the LogQL query: + + ```bash {container="evaluate-loki-flog-1"} | json | status="401" ``` -To see every log line that does not contain the value 401: + + +To see every log line that doesn't contain the text `401`: + + ```bash {container="evaluate-loki-flog-1"} != "401" ``` + + For more examples, refer to the [query documentation](https://grafana.com/docs/loki//query/query_examples/). + + ## Complete metrics, logs, traces, and profiling example -If you would like to use a demo that includes Mimir, Loki, Tempo, and Grafana, you can use [Introduction to Metrics, Logs, Traces, and Profiling in Grafana](https://github.com/grafana/intro-to-mlt). `Intro-to-mltp` provides a self-contained environment for learning about Mimir, Loki, Tempo, and Grafana. +If you would like to run a demonstration environment that includes Mimir, Loki, Tempo, and Grafana, you can use [Introduction to Metrics, Logs, Traces, and Profiling in Grafana](https://github.com/grafana/intro-to-mlt). +It's a self-contained environment for learning about Mimir, Loki, Tempo, and Grafana. -The project includes detailed explanations of each component and annotated configurations for a single-instance deployment. Data from `intro-to-mltp` can also be pushed to Grafana Cloud. +The project includes detailed explanations of each component and annotated configurations for a single-instance deployment. +You can also push the data from the environment to [Grafana Cloud](https://grafana.com/cloud/. From 2764d020d3af58e63abdb95cd4fb055f87f4de89 Mon Sep 17 00:00:00 2001 From: Jack Baldry Date: Wed, 19 Jun 2024 10:58:47 +0100 Subject: [PATCH 02/18] Replace Docker Compose exec for Killercoda tutorial Signed-off-by: Jack Baldry --- docs/sources/get-started/quick-start.md | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/docs/sources/get-started/quick-start.md b/docs/sources/get-started/quick-start.md index 761b94c791fc7..e806ab60b80dd 100644 --- a/docs/sources/get-started/quick-start.md +++ b/docs/sources/get-started/quick-start.md @@ -100,13 +100,19 @@ This quickstart assumes you are running Linux. With `evaluate-loki` as the current working directory, start the demo environment using `docker compose`: - + ```bash docker compose up -d ``` - + + + + + + + You should see something similar to the following: From 0069e39612f3636ca3f4cbd125e2e7be229540df Mon Sep 17 00:00:00 2001 From: Jack Baldry Date: Wed, 19 Jun 2024 11:08:19 +0100 Subject: [PATCH 03/18] Include {{exec}} action Signed-off-by: Jack Baldry --- docs/sources/get-started/quick-start.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/sources/get-started/quick-start.md b/docs/sources/get-started/quick-start.md index e806ab60b80dd..a7c5b41f71db7 100644 --- a/docs/sources/get-started/quick-start.md +++ b/docs/sources/get-started/quick-start.md @@ -111,7 +111,7 @@ This quickstart assumes you are running Linux. - + You should see something similar to the following: From cf3e860b1cca1d7ec8143b6fa8eba63cec571e34 Mon Sep 17 00:00:00 2001 From: Jack Baldry Date: Thu, 20 Jun 2024 13:08:11 +0100 Subject: [PATCH 04/18] Fix Killercoda docker-compose up output Signed-off-by: Jack Baldry --- docs/sources/get-started/quick-start.md | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/docs/sources/get-started/quick-start.md b/docs/sources/get-started/quick-start.md index a7c5b41f71db7..9fd5de51b3838 100644 --- a/docs/sources/get-started/quick-start.md +++ b/docs/sources/get-started/quick-start.md @@ -114,7 +114,9 @@ This quickstart assumes you are running Linux. - You should see something similar to the following: + At the end of the command, you should see something similar to the following: + + ```bash ✔ Network evaluate-loki_loki Created 0.1s @@ -128,6 +130,19 @@ This quickstart assumes you are running Linux. ✔ Container evaluate-loki-alloy-1 Started 1.4s ``` + + + + + + + + + + + + + 1. (Optional) Verify that the Loki cluster is up and running. - The read component returns `ready` when you browse to [http://localhost:3101/ready](http://localhost:3101/ready). From ac913262fc3726e0968a869b373509e4ef7df625 Mon Sep 17 00:00:00 2001 From: Jack Baldry Date: Thu, 20 Jun 2024 14:28:00 +0100 Subject: [PATCH 05/18] Add preprocessing substitutions Signed-off-by: Jack Baldry --- docs/sources/get-started/quick-start.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/sources/get-started/quick-start.md b/docs/sources/get-started/quick-start.md index 9fd5de51b3838..da03ac3641d04 100644 --- a/docs/sources/get-started/quick-start.md +++ b/docs/sources/get-started/quick-start.md @@ -4,6 +4,10 @@ menuTitle: Loki quickstart weight: 200 description: How to create and use a local Loki cluster for testing and evaluation purposes. killercoda: + preprocessing: + substitutions: + - regexp: evaluate-loki-([^-]+)- + replacement: evaluate-loki_${1}_ title: Loki Quickstart Demo description: This sandbox provides an online enviroment for testing the Loki quickstart demo. details: From 3d4551d2032faa8f915d4203ecd4af15cbffe854 Mon Sep 17 00:00:00 2001 From: Jack Baldry Date: Thu, 20 Jun 2024 14:44:52 +0100 Subject: [PATCH 06/18] Fix include Signed-off-by: Jack Baldry --- docs/sources/get-started/quick-start.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/sources/get-started/quick-start.md b/docs/sources/get-started/quick-start.md index da03ac3641d04..bec234a3df7d9 100644 --- a/docs/sources/get-started/quick-start.md +++ b/docs/sources/get-started/quick-start.md @@ -122,7 +122,7 @@ This quickstart assumes you are running Linux. - ```bash + ```console ✔ Network evaluate-loki_loki Created 0.1s ✔ Container evaluate-loki-minio-1 Started 0.6s ✔ Container evaluate-loki-flog-1 Started 0.6s @@ -137,6 +137,7 @@ This quickstart assumes you are running Linux. + @@ -145,6 +146,7 @@ This quickstart assumes you are running Linux. + 1. (Optional) Verify that the Loki cluster is up and running. From 3aaf4a8627c1d3b76774767136cd6fa396259f82 Mon Sep 17 00:00:00 2001 From: Jack Baldry Date: Fri, 21 Jun 2024 11:28:46 +0100 Subject: [PATCH 07/18] Parallelize list Signed-off-by: Jack Baldry --- docs/sources/get-started/quick-start.md | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/docs/sources/get-started/quick-start.md b/docs/sources/get-started/quick-start.md index bec234a3df7d9..3fe437ae1507f 100644 --- a/docs/sources/get-started/quick-start.md +++ b/docs/sources/get-started/quick-start.md @@ -23,17 +23,19 @@ killercoda: If you want to experiment with Loki, you can run Loki locally using the Docker Compose file that ships with Loki. It runs Loki in a [monolithic deployment](https://grafana.com/docs/loki//get-started/deployment-modes/#monolithic-mode) mode and includes a sample application to generate logs. -The Docker Compose configuration instantiates the following components, each in its own container: +The Docker Compose configuration runs the following components, each in its own container: + +- **flog**: which generates log lines. -- **flog** a sample application which generates log lines. [flog](https://github.com/mingrammer/flog) is a log generator for common log formats. -- **Grafana Alloy** which scrapes the log lines from flog, and pushes them to Loki through the gateway. + +- **Grafana Alloy**: which scrapes the log lines from flog, and pushes them to Loki through the gateway. - **Gateway** (nginx) which receives requests and redirects them to the appropriate container based on the request's URL. -- One Loki **read** component (Query Frontend, Querier). -- One Loki **write** component (Distributor, Ingester). -- One Loki **backend** component (Index Gateway, Compactor, Ruler, Bloom Compactor (Experimental), Bloom Gateway (Experimental)). -- **Minio** an S3-compatible object store which Loki uses to store its index and chunks. -- **Grafana** which provides visualization of the log lines captured within Loki. +- **Loki read component**: which runs a query-frontend and a querier. +- **Loki write component**: which runs a distributor and an ingester. +- **Loki backend component**: which runs an index-gateway, compactor, ruler, bloom-compactor, and bloom-gateway. +- **Minio**: which Loki uses to store its index and chunks. +- **Grafana**: which provides visualization of the log lines captured within Loki. {{< figure max-width="75%" src="/media/docs/loki/get-started-flog-v3.png" caption="Getting started sample application" alt="Getting started sample application" >}} From a13431328df6e7f4143791383c2debeb66e44fc6 Mon Sep 17 00:00:00 2001 From: Jack Baldry Date: Mon, 24 Jun 2024 21:20:57 +0100 Subject: [PATCH 08/18] Apply suggestions from code review Co-authored-by: J Stickler --- docs/sources/get-started/quick-start.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/sources/get-started/quick-start.md b/docs/sources/get-started/quick-start.md index 3fe437ae1507f..ad0eab128a6ac 100644 --- a/docs/sources/get-started/quick-start.md +++ b/docs/sources/get-started/quick-start.md @@ -31,9 +31,9 @@ The Docker Compose configuration runs the following components, each in its own - **Grafana Alloy**: which scrapes the log lines from flog, and pushes them to Loki through the gateway. - **Gateway** (nginx) which receives requests and redirects them to the appropriate container based on the request's URL. -- **Loki read component**: which runs a query-frontend and a querier. -- **Loki write component**: which runs a distributor and an ingester. -- **Loki backend component**: which runs an index-gateway, compactor, ruler, bloom-compactor, and bloom-gateway. +- **Loki read component**: which runs a Query Frontend and a Querier. +- **Loki write component**: which runs a Distributor and an Ingester. +- **Loki backend component**: which runs an Index Gateway, Compactor, Ruler, Bloom Compactor (experimental), and Bloom Gateway (experimental). - **Minio**: which Loki uses to store its index and chunks. - **Grafana**: which provides visualization of the log lines captured within Loki. @@ -43,7 +43,7 @@ The Docker Compose configuration runs the following components, each in its own ## Before you begin -Use the [Interactive Learning Environment](#interactive-learning-environment) to skip installing Docker and Docker Compose. +You can use the [Interactive Learning Environment](#interactive-learning-environment) to skip installing Docker and Docker Compose. - Install [Docker](https://docs.docker.com/install) - Install [Docker Compose](https://docs.docker.com/compose/install) @@ -232,7 +232,7 @@ You can view your logs using the command line interface, [LogCLI](/docs/loki/ - 1. Find all the log lines in the `{container="evaluate-loki-flog-1}` stream that contain the string `status`: + 1. Find all the log lines in the `{container="evaluate-loki-flog-1"}` stream that contain the string `status`: @@ -242,7 +242,7 @@ You can view your logs using the command line interface, [LogCLI](/docs/loki/ - 1. Find all the log lines in the `{container="evaluate-loki-flog-1}` stream where the JSON field `status` has the value `404`: + 1. Find all the log lines in the `{container="evaluate-loki-flog-1"}` stream where the JSON field `status` has the value `404`: From cb3506b0691ac8d8843cd125043ad1ecac3c3f84 Mon Sep 17 00:00:00 2001 From: Jack Baldry Date: Mon, 24 Jun 2024 21:22:39 +0100 Subject: [PATCH 09/18] Restore removed line Signed-off-by: Jack Baldry --- docs/sources/get-started/quick-start.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/sources/get-started/quick-start.md b/docs/sources/get-started/quick-start.md index ad0eab128a6ac..a8fb66a1b2791 100644 --- a/docs/sources/get-started/quick-start.md +++ b/docs/sources/get-started/quick-start.md @@ -196,6 +196,8 @@ You can view your logs using the command line interface, [LogCLI](/docs/loki/ Date: Wed, 26 Jun 2024 08:37:35 +0100 Subject: [PATCH 10/18] Use a single paragraph --- docs/sources/get-started/quick-start.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/sources/get-started/quick-start.md b/docs/sources/get-started/quick-start.md index a8fb66a1b2791..d9407e863018e 100644 --- a/docs/sources/get-started/quick-start.md +++ b/docs/sources/get-started/quick-start.md @@ -26,7 +26,6 @@ If you want to experiment with Loki, you can run Loki locally using the Docker C The Docker Compose configuration runs the following components, each in its own container: - **flog**: which generates log lines. - [flog](https://github.com/mingrammer/flog) is a log generator for common log formats. - **Grafana Alloy**: which scrapes the log lines from flog, and pushes them to Loki through the gateway. From 3cdd15c056d3556f462ce9d451d9c768446365df Mon Sep 17 00:00:00 2001 From: Jack Baldry Date: Wed, 26 Jun 2024 08:38:38 +0100 Subject: [PATCH 11/18] Fix final link Signed-off-by: Jack Baldry --- docs/sources/get-started/quick-start.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/sources/get-started/quick-start.md b/docs/sources/get-started/quick-start.md index d9407e863018e..41e25ea068348 100644 --- a/docs/sources/get-started/quick-start.md +++ b/docs/sources/get-started/quick-start.md @@ -343,4 +343,4 @@ If you would like to run a demonstration environment that includes Mimir, Loki, It's a self-contained environment for learning about Mimir, Loki, Tempo, and Grafana. The project includes detailed explanations of each component and annotated configurations for a single-instance deployment. -You can also push the data from the environment to [Grafana Cloud](https://grafana.com/cloud/. +You can also push the data from the environment to [Grafana Cloud](https://grafana.com/cloud/). From 38af096a80f0bb08141ceca6b8efb5c2793775cd Mon Sep 17 00:00:00 2001 From: Jack Baldry Date: Wed, 26 Jun 2024 14:01:33 +0100 Subject: [PATCH 12/18] Use updated directive naming convention Signed-off-by: Jack Baldry --- docs/sources/get-started/quick-start.md | 80 ++++++++++++------------- 1 file changed, 40 insertions(+), 40 deletions(-) diff --git a/docs/sources/get-started/quick-start.md b/docs/sources/get-started/quick-start.md index 41e25ea068348..92bc806b7b11e 100644 --- a/docs/sources/get-started/quick-start.md +++ b/docs/sources/get-started/quick-start.md @@ -17,7 +17,7 @@ killercoda: imageid: ubuntu --- - + # Quickstart to run Loki locally @@ -38,7 +38,7 @@ The Docker Compose configuration runs the following components, each in its own {{< figure max-width="75%" src="/media/docs/loki/get-started-flog-v3.png" caption="Getting started sample application" alt="Getting started sample application" >}} - + ## Before you begin @@ -63,35 +63,35 @@ Try out this demo within our interactive learning environment: [Loki Quickstart - All commands run inside the interactive terminal. - You can access Grafana with the URL links provided within the sandbox. - + ## Install Loki and collecting sample logs - + {{< admonition type="note" >}} This quickstart assumes you are running Linux. {{< /admonition >}} - + **To install Loki locally, follow these steps:** 1. Create a directory called `evaluate-loki` for the demo environment. Make `evaluate-loki` your current working directory: - + ```bash mkdir evaluate-loki cd evaluate-loki ``` - + 1. Download `loki-config.yaml`, `alloy-local-config.yaml`, and `docker-compose.yaml`: - + ```bash wget https://raw.githubusercontent.com/grafana/loki/main/examples/getting-started/loki-config.yaml -O loki-config.yaml @@ -99,29 +99,29 @@ This quickstart assumes you are running Linux. wget https://raw.githubusercontent.com/grafana/loki/main/examples/getting-started/docker-compose.yaml -O docker-compose.yaml ``` - + 1. Deploy the sample Docker image. With `evaluate-loki` as the current working directory, start the demo environment using `docker compose`: - + ```bash docker compose up -d ``` - + - + - + At the end of the command, you should see something similar to the following: - + ```console ✔ Network evaluate-loki_loki Created 0.1s @@ -135,9 +135,9 @@ This quickstart assumes you are running Linux. ✔ Container evaluate-loki-alloy-1 Started 1.4s ``` - + - + @@ -148,7 +148,7 @@ This quickstart assumes you are running Linux. - + 1. (Optional) Verify that the Loki cluster is up and running. @@ -160,9 +160,9 @@ This quickstart assumes you are running Linux. 1. (Optional) Verify that Grafana Alloy is running. - You can access the Grafana Alloy UI at [http://localhost:12345](http://localhost:12345). - + - + ## View your logs in Grafana @@ -208,13 +208,13 @@ You can view your logs using the command line interface, [LogCLI](/docs/loki/ + ```bash {container="evaluate-loki-flog-1"} ``` - + In Loki, this is a log stream. @@ -225,43 +225,43 @@ You can view your logs using the command line interface, [LogCLI](/docs/loki/ + ```bash {container="evaluate-loki-grafana-1"} ``` - + 1. Find all the log lines in the `{container="evaluate-loki-flog-1"}` stream that contain the string `status`: - + ```bash {container="evaluate-loki-flog-1"} |= `status` ``` - + 1. Find all the log lines in the `{container="evaluate-loki-flog-1"}` stream where the JSON field `status` has the value `404`: - + ```bash {container="evaluate-loki-flog-1"} | json | status=`404` ``` - + 1. Calculate the number of logs per second where the JSON field `status` has the value `404`: - + ```bash sum by(container) (rate({container="evaluate-loki-flog-1"} | json | status=`404` [$__auto])) ``` - + The final query is a metric query which returns a time series. This makes Grafana draw a graph of the results. @@ -283,59 +283,59 @@ Here are some more sample queries that you can run using the Flog sample data. To see all the log lines that flog has generated, enter the LogQL query: - + ```bash {container="evaluate-loki-flog-1"} ``` - + The flog app generates log lines for simulated HTTP requests. To see all `GET` log lines, enter the LogQL query: - + ```bash {container="evaluate-loki-flog-1"} |= "GET" ``` - + To see all `POST` methods, enter the LogQL query: - + ```bash {container="evaluate-loki-flog-1"} |= "POST" ``` - + To see every log line with a 401 status (unauthorized error), enter the LogQL query: - + ```bash {container="evaluate-loki-flog-1"} | json | status="401" ``` - + To see every log line that doesn't contain the text `401`: - + ```bash {container="evaluate-loki-flog-1"} != "401" ``` - + For more examples, refer to the [query documentation](https://grafana.com/docs/loki//query/query_examples/). - + ## Complete metrics, logs, traces, and profiling example From 3fd61b76e59a6bf84f83b420744899b9094e1664 Mon Sep 17 00:00:00 2001 From: Jack Baldry Date: Wed, 26 Jun 2024 14:27:21 +0100 Subject: [PATCH 13/18] Remove copy directives that are redundant with default copy behavior Signed-off-by: Jack Baldry --- docs/sources/get-started/quick-start.md | 40 ------------------------- 1 file changed, 40 deletions(-) diff --git a/docs/sources/get-started/quick-start.md b/docs/sources/get-started/quick-start.md index 92bc806b7b11e..1549605ef2b7e 100644 --- a/docs/sources/get-started/quick-start.md +++ b/docs/sources/get-started/quick-start.md @@ -208,14 +208,10 @@ You can view your logs using the command line interface, [LogCLI](/docs/loki/ - ```bash {container="evaluate-loki-flog-1"} ``` - - In Loki, this is a log stream. Loki uses [labels](https://grafana.com/docs/loki//get-started/labels/) as metadata to describe log streams. @@ -225,44 +221,28 @@ You can view your logs using the command line interface, [LogCLI](/docs/loki/ - ```bash {container="evaluate-loki-grafana-1"} ``` - - 1. Find all the log lines in the `{container="evaluate-loki-flog-1"}` stream that contain the string `status`: - - ```bash {container="evaluate-loki-flog-1"} |= `status` ``` - - 1. Find all the log lines in the `{container="evaluate-loki-flog-1"}` stream where the JSON field `status` has the value `404`: - - ```bash {container="evaluate-loki-flog-1"} | json | status=`404` ``` - - 1. Calculate the number of logs per second where the JSON field `status` has the value `404`: - - ```bash sum by(container) (rate({container="evaluate-loki-flog-1"} | json | status=`404` [$__auto])) ``` - - The final query is a metric query which returns a time series. This makes Grafana draw a graph of the results. @@ -283,56 +263,36 @@ Here are some more sample queries that you can run using the Flog sample data. To see all the log lines that flog has generated, enter the LogQL query: - - ```bash {container="evaluate-loki-flog-1"} ``` - - The flog app generates log lines for simulated HTTP requests. To see all `GET` log lines, enter the LogQL query: - - ```bash {container="evaluate-loki-flog-1"} |= "GET" ``` - - To see all `POST` methods, enter the LogQL query: - - ```bash {container="evaluate-loki-flog-1"} |= "POST" ``` - - To see every log line with a 401 status (unauthorized error), enter the LogQL query: - - ```bash {container="evaluate-loki-flog-1"} | json | status="401" ``` - - To see every log line that doesn't contain the text `401`: - - ```bash {container="evaluate-loki-flog-1"} != "401" ``` - - For more examples, refer to the [query documentation](https://grafana.com/docs/loki//query/query_examples/). From c58856aef1981e5a0fbfd30a67d52ca4e697b24b Mon Sep 17 00:00:00 2001 From: Jack Baldry Date: Thu, 27 Jun 2024 15:00:49 +0100 Subject: [PATCH 14/18] Use page directives Signed-off-by: Jack Baldry --- docs/sources/get-started/quick-start.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/sources/get-started/quick-start.md b/docs/sources/get-started/quick-start.md index 1549605ef2b7e..d41431a0c449a 100644 --- a/docs/sources/get-started/quick-start.md +++ b/docs/sources/get-started/quick-start.md @@ -17,7 +17,7 @@ killercoda: imageid: ubuntu --- - + # Quickstart to run Loki locally @@ -38,7 +38,7 @@ The Docker Compose configuration runs the following components, each in its own {{< figure max-width="75%" src="/media/docs/loki/get-started-flog-v3.png" caption="Getting started sample application" alt="Getting started sample application" >}} - + ## Before you begin @@ -63,7 +63,7 @@ Try out this demo within our interactive learning environment: [Loki Quickstart - All commands run inside the interactive terminal. - You can access Grafana with the URL links provided within the sandbox. - + ## Install Loki and collecting sample logs @@ -160,9 +160,9 @@ This quickstart assumes you are running Linux. 1. (Optional) Verify that Grafana Alloy is running. - You can access the Grafana Alloy UI at [http://localhost:12345](http://localhost:12345). - + - + ## View your logs in Grafana @@ -295,7 +295,7 @@ To see every log line that doesn't contain the text `401`: For more examples, refer to the [query documentation](https://grafana.com/docs/loki//query/query_examples/). - + ## Complete metrics, logs, traces, and profiling example From db9a501727a514c75cb7426c5b0dc803ba306b85 Mon Sep 17 00:00:00 2001 From: Jack Baldry Date: Thu, 27 Jun 2024 15:11:15 +0100 Subject: [PATCH 15/18] Use consistent finish page name Signed-off-by: Jack Baldry --- docs/sources/get-started/quick-start.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/sources/get-started/quick-start.md b/docs/sources/get-started/quick-start.md index d41431a0c449a..0b64011210f8d 100644 --- a/docs/sources/get-started/quick-start.md +++ b/docs/sources/get-started/quick-start.md @@ -12,7 +12,7 @@ killercoda: description: This sandbox provides an online enviroment for testing the Loki quickstart demo. details: finish: - text: finished.md + text: finish.md backend: imageid: ubuntu --- From 10787be9b23f5c5298fef6a414a9621a31f648c9 Mon Sep 17 00:00:00 2001 From: Jack Baldry Date: Thu, 4 Jul 2024 11:40:06 +0100 Subject: [PATCH 16/18] Use docs/ignore shortcode Signed-off-by: Jack Baldry --- docs/sources/get-started/quick-start.md | 41 +++++++++++++++---------- 1 file changed, 24 insertions(+), 17 deletions(-) diff --git a/docs/sources/get-started/quick-start.md b/docs/sources/get-started/quick-start.md index 0b64011210f8d..6095e9ff548af 100644 --- a/docs/sources/get-started/quick-start.md +++ b/docs/sources/get-started/quick-start.md @@ -113,11 +113,16 @@ This quickstart assumes you are running Linux. - - - - - + {{< docs/ignore >}} + + + ```bash + docker-compose up -d + ``` + + + + {{< /docs/ignore >}} At the end of the command, you should see something similar to the following: @@ -137,18 +142,20 @@ This quickstart assumes you are running Linux. - - - - - - - - - - - - + {{< docs/ignore >}} + + ```console + Creating evaluate-loki_flog_1 ... done + Creating evaluate-loki_minio_1 ... done + Creating evaluate-loki_read_1 ... done + Creating evaluate-loki_write_1 ... done + Creating evaluate-loki_gateway_1 ... done + Creating evaluate-loki_alloy_1 ... done + Creating evaluate-loki_grafana_1 ... done + Creating evaluate-loki_backend_1 ... done + ``` + + {{< /docs/ignore >}} 1. (Optional) Verify that the Loki cluster is up and running. From 95e7fe68bcd281f93ea7c658346b31889edf485d Mon Sep 17 00:00:00 2001 From: Jack Baldry Date: Fri, 5 Jul 2024 10:38:14 +0100 Subject: [PATCH 17/18] Add comment Signed-off-by: Jack Baldry --- docs/sources/get-started/quick-start.md | 20 +++++++++++++++----- 1 file changed, 15 insertions(+), 5 deletions(-) diff --git a/docs/sources/get-started/quick-start.md b/docs/sources/get-started/quick-start.md index 6095e9ff548af..741232edde79d 100644 --- a/docs/sources/get-started/quick-start.md +++ b/docs/sources/get-started/quick-start.md @@ -4,6 +4,14 @@ menuTitle: Loki quickstart weight: 200 description: How to create and use a local Loki cluster for testing and evaluation purposes. killercoda: + comment: | + The killercoda front matter and the HTML comments that start ' - {{< admonition type="note" >}} This quickstart assumes you are running Linux. {{< /admonition >}} - - **To install Loki locally, follow these steps:** 1. Create a directory called `evaluate-loki` for the demo environment. From d860aeab1398efaf64e5427dca56eee1bd820a90 Mon Sep 17 00:00:00 2001 From: Jack Baldry Date: Fri, 19 Jul 2024 09:12:32 +0100 Subject: [PATCH 18/18] Ignore note until admonition transformation is implemented Signed-off-by: Jack Baldry --- docs/sources/get-started/quick-start.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/sources/get-started/quick-start.md b/docs/sources/get-started/quick-start.md index 741232edde79d..9fa0e045da004 100644 --- a/docs/sources/get-started/quick-start.md +++ b/docs/sources/get-started/quick-start.md @@ -81,10 +81,14 @@ Try out this demo within our interactive learning environment: [Loki Quickstart ## Install Loki and collecting sample logs + + {{< admonition type="note" >}} This quickstart assumes you are running Linux. {{< /admonition >}} + + **To install Loki locally, follow these steps:** 1. Create a directory called `evaluate-loki` for the demo environment.