Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[EDU-5581] [EDU-5582] [EDU-5590] feature: add documentation related to the httpBreakdownMetrics dataset #1348

Open
wants to merge 17 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 13 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ Four datasets are deprecated and were replaced by other datsets:
| dataStreamedMetrics | Sent events of data by [Data Stream](/en/documentation/products/observe/data-stream/) to the clients' endpoint |
| edgeFunctionsMetrics | Events executed by [Edge Functions](/en/documentation/products/build/edge-application/edge-functions/) |
| httpMetrics | Request events registered by [Edge Application](/en/documentation/products/build/edge-application/) and [Edge Firewall](/en/documentation/products/secure/edge-firewall/) |
| httpBreakdownMetrics | Query real-time, detailed, aggregated data on [HTTP request events blocked](/en/documentation/products/guides/query-httpBreakdownMetrics-data-with-graphql/) |
| edgeDnsQueriesMetrics | Query events performed on [Edge DNS](/en/documentation/products/secure/edge-dns/) |
| imagesProcessedMetrics | Image processing events by [Image Processor](/en/documentation/products/build/edge-application/image-processor/) |
| tieredCacheMetrics | Request events registered by [Tiered Cache](/en/documentation/products/build/edge-application/tiered-cache/) |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,32 @@ When a field is a result of any kind of calculation, such as a sum, they're cons

---

## httpBreakdownMetrics

| Field | Description |
|------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `ts` | Timestamp of when the event was created. Example: `2022-10-20T10:10:10` |
| `host` | Host information sent on the request line. Stores: hostname from the request line, or hostname from the Host request header field, or the server name matching a request. Example: `hello.myhost.net` |
| `remoteAddress` | IP address of the origin that generated the request. Example: `127.0.0.1` |
| `requestPath` | Path of the request made by the end user, without the host, protocol, and arguments. Example: `/v1/application` |
| `geolocAsn` | Autonomous System Number (ASN) Allocation queried from the MaxMind table. Example: `AS52580 Azion Technologies Ltda.` |
| `geolocCountryName`| Remote client's country detected via IP address geolocation. Example: `United States`, `Russian Federation` |
| `httpUserAgent` | End user's application, operating system, vendor, and/or version. Value of the User-Agent header. Example: `Mozilla/5.0 (Windows NT 10.0; Win64; x64)` |
| `httpReferer` | Address of the page the user made the request from. Example: `https://example.com` |
| `sentHttpContentType` | Content-Type header sent in the origin's response. Example: `text/html; charset=UTF-8` |
| `upstreamAddr`| Client's IP address and port. It can also store multiple servers or server groups. Example: `192.168.1.1:80`. When the response is `127.0.0.1:1666`, the upstream is Azion Cells Runtime |

### Calculated fields

When a field results from any kind of calculation, such as a sum, they're considered as *calculated fields*.

| Calculated field | Description |
|----------|----------|
| `blockedRequests` | Total amount of blocked requests in the aggregation being used. This field is the result of a sum. Example: `17` |
| `requests` | Total amount of requests in the aggregation being used. This field is the result of a sum. Example: `11` |

---

## tieredCacheMetrics (Tiered Cache)

| Field | Description |
Expand Down Expand Up @@ -311,7 +337,6 @@ When a field results from any kind of calculation, such as a sum, they're consid
| `uniqRequestUrl` | Number of unique URLs accessed by bots. This field represents the count of distinct URLs requested by bots. Example: `5` |

---

## Deprecated datasets

The following datasets were discontinued. It's recommended to use the [new datasets which replaced them](/en/documentation/devtools/graphql-api/features/#datasets).
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
---
title: How to query data from the httpBreakdownMetrics dataset
description: This guide will explain how to query data from the httpBreakdownMetrics dataset using the GraphiQL playground.
meta_tags: graphql, graphql playground, security metrics, edge applications, requests
namespace: docs_guides_query_httpBreakdownMetrics_graphql
permalink: /documentation/products/guides/query-httpbreakdownmetrics-data-with-graphql/
menu_namespace: graphqlMenu
---

The **httpBreakdownMetrics** dataset provides real-time, detailed, aggregated data on HTTP request events blocked. This dataset is part of the Real-Time Metrics GraphQL.

This data is retained and available for up to *90* days.

This guide will explain how to query data from the httpBreakdownMetrics dataset using the GraphiQL playground.

---

## Querying data

This example queries the top 20 blocked `remoteAddress` entries. To know more about the available fields, check the [Real-Time Metrics GraphQL API Fields](/en/documentation/devtools/graphql-api/features/gql-real-time-metrics-fields/) documentation.

:::note
The dataset is based on a 50% sample of the data, meaning values are approximate.
:::

1. Access the GraphiQL playground in this link: `https://manager.azion.com/metrics/graphql`.
- You must be logged in to your Azion account. Otherwise, you'll receive an error message.
2. Send a query following this format:

```graphql
query {
httpBreakdownMetrics(
aggregate: { sum: blockedRequests }
groupBy: [remoteAddress]
orderBy: [sum_DESC],
limit: 20,
filter: {
tsGte: "2024-10-21T11:00:00"
tsLt: "2024-10-21T12:00:00"
}
) {
remoteAddress
totalBlocked: sum
}
}
```

Where:

| Field | Description |
|----------|----------|
| `sum: blockedRequests` | Returns the total number of requests blocked within the specified time range, after applying any filters |
| `groupBy` | Specifies the fields by which the query results should be grouped. Example: `[remoteAddress]` |
| `orderBy` | Specifies the order in which the results should be returned. Examples: `[sum_DESC]`, for descending order, and `[sum_ASC]`, for ascending order |
| `limit` | Specifies the maximum number of results to return. Example: `20` for retrieving the top 20. System maximum: |
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you confirm if we have a system max for this field, @brunomrpx ?

| `filter` | Defines the criteria used to filter the data returned by the query |
| `tsGte` | A subfield of `filter`. Specifies the start time (greater than or equal to) for the data query, ensuring results include records from this timestamp onward. Format: "YYYY-MM-DDTHH:mm:ss"; example: `"2024-10-21T11:00:00"` |
| `tsLt` | A subfield of `filter`. Specifies the end time (less than) for the data query, filtering out any records with timestamps equal to or after this timestamp. Format: "YYYY-MM-DDTHH:mm:ss"; example: `"2024-10-21T12:00:00"` |

3. You'll receive a response similar to this:

```graphql
{
"data": {
"httpBreakdownMetrics": [
{
"remoteAddress": "192.168.0.1",
"totalBlocked": 6732
},
{
"remoteAddress": "10.0.0.2",
"totalBlocked": 5872
},
{
"remoteAddress": "172.16.0.3",
"totalBlocked": 3958
},
{
"remoteAddress": "192.168.1.4",
"totalBlocked": 3952
},
{
"remoteAddress": "10.0.1.5",
"totalBlocked": 3806
},
{
"remoteAddress": "172.16.1.6",
"totalBlocked": 3730
},
{
"remoteAddress": "192.168.2.7",
"totalBlocked": 3378
},
{
"remoteAddress": "10.0.2.8",
"totalBlocked": 3318
},
{
"remoteAddress": "172.16.2.9",
"totalBlocked": 3284
},
{
"remoteAddress": "192.168.3.10",
"totalBlocked": 3282
},
{
"remoteAddress": "10.0.3.11",
"totalBlocked": 2958
},
{
"remoteAddress": "172.16.3.12",
"totalBlocked": 2884
},
{
"remoteAddress": "192.168.4.13",
"totalBlocked": 2530
},
{
"remoteAddress": "10.0.4.14",
"totalBlocked": 2348
},
{
"remoteAddress": "172.16.4.15",
"totalBlocked": 2004
},
{
"remoteAddress": "192.168.5.16",
"totalBlocked": 1902
},
{
"remoteAddress": "10.0.5.17",
"totalBlocked": 1538
},
{
"remoteAddress": "172.16.5.18",
"totalBlocked": 1440
},
{
"remoteAddress": "192.168.6.19",
"totalBlocked": 1390
},
{
"remoteAddress": "10.0.6.20",
"totalBlocked": 1314
}
]
}
}
```

Where:

| Field | Description |
|----------|----------|
| `remoteAddress` | Specifies the IP address of the source making the request. Example: `10.0.6.20` |
| `totalBlocked` | Refers to the total number of times request from this IP address has been blocked. This field is the result of a sum. Example: `1314` |
MarianaAguilera marked this conversation as resolved.
Show resolved Hide resolved
MarianaAguilera marked this conversation as resolved.
Show resolved Hide resolved
1 change: 1 addition & 0 deletions src/content/docs/en/pages/guides/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -176,6 +176,7 @@ permalink: /documentation/products/guides/
- [How to query Connected Users data with GraphQL API](/en/documentation/products/guides/query-connected-users-data-with-graphql/)
- [How to query Bot Manager data with GraphQL API](/en/documentation/products/guides/query-bot-manager-data-with-graphql/)
- [How to query the top URLs impacted by bots with GraphQL API](/en/documentation/products/guides/query-bot-manager-breakdown-data-with-graphql/)
- [How to query data from the httpBreakdownMetrics dataset](/en/documentation/products/guides/query-httpbreakdownmetrics-data-with-graphql/)

---

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,32 @@ Quando um campo é o resultado de algum tipo de cálculo, como uma soma, ele é

---

## httpBreakdownMetrics

| Campo | Descrição |
|------------------------|---------------------|
| `ts` | Timestamp de quando o evento foi criado. Exemplo: `2022-10-20T10:10:10` |
| `host` | Informações do host enviadas na linha da requisição. Armazena: nome do host da linha da requisição, ou nome do host do campo do cabeçalho da requisição Host, ou o nome do servidor correspondente à requisição. Exemplo: `hello.myhost.net` |
| `remoteAddress` | Endereço IP da origem que gerou a requisição. Exemplo: `127.0.0.1` |
| `requestPath` | Caminho da requisição feita pelo usuário final, sem o host, protocolo e argumentos. Exemplo: `/v1/application` |
| `geolocAsn` | Autonomous System Number (ASN) consultado na tabela MaxMind. Exemplo: `AS52580 Azion Technologies Ltda.` |
| `geolocCountryName`| País do cliente remoto detectado via geolocalização do endereço IP. Exemplo: `United States`, `Russian Federation` |
| `httpUserAgent` | Aplicativo do usuário final, sistema operacional, fornecedor e/ou versão. Valor do cabeçalho User-Agent. Exemplo: `Mozilla/5.0 (Windows NT 10.0; Win64; x64)` |
| `httpReferer` | Endereço da página de onde o usuário fez a requisição. Exemplo: `https://example.com` |
| `sentHttpContentType` | Cabeçalho Content-Type enviado na resposta da origem. Exemplo: `text/html; charset=UTF-8` |
| `upstreamAddr`| Endereço IP e porta do cliente. Também pode armazenar múltiplos servidores ou grupos de servidores. Exemplo: `192.168.1.1:80`. Quando a resposta é `127.0.0.1:1666`, o upstream é Azion Cells Runtime |

### Campos calculados

Quando um campo resulta de qualquer tipo de cálculo, como uma soma, eles são considerados como *campos calculados*.

| Campo calculado | Descrição |
|----------|----------|
| `blockedRequests` | Quantidade total de requisições bloqueadas na agregação utilizada. Este campo é o resultado de uma soma. Exemplo: `17` |
| `requests` | Quantidade total de requisições na agregação utilizada. Este campo é o resultado de uma soma. Exemplo: `11` |

---

## tieredCacheMetrics (Tiered Cache)

| Campo | Descrição |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ Quatro conjuntos de dados foram descontinuados e substituiídos por outros:
| dataStreamedMetrics | Registros de envio de dados do [Data Stream](/pt-br/documentacao/produtos/observe/data-stream/) para o endpoint do cliente |
| edgeFunctionsMetrics | Eventos de execução do [Edge Functions](/pt-br/documentacao/produtos/build/edge-application/edge-functions/) |
| httpMetrics | Eventos de requisições registradas pelo [Edge Application](/pt-br/documentacao/produtos/build/edge-application/) e [Edge Firewall](/pt-br/documentacao/produtos/secure/edge-firewall/) |
| httpBreakdownMetrics | Consulte dados agregados, detalhados e em tempo real sobre [eventos de requisições HTTP bloqueadas](/pt-br/documentacao/produtos/guias/consultar-dados-httpbreakdownmetrics-com-graphql/) |
| edgeDnsQueriesMetrics | Eventos de consultas realizadas no [Edge DNS](/pt-br/documentacao/produtos/secure/edge-dns/) |
| imagesProcessedMetrics | Eventos de processamento de imagens do [Image Processor](/pt-br/documentacao/produtos/build/edge-application/image-processor/) |
| tieredCacheMetrics | Eventos de requisições registradas pelo [Tiered Cache](/pt-br/documentacao/produtos/build/edge-application/tiered-cache/) |
Expand Down
Loading