Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

flow: Add otelcol.receiver.vcenter component #5715

Merged
merged 24 commits into from
Nov 16, 2023
Merged
Show file tree
Hide file tree
Changes from 23 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
e3baf8e
flow: Add otelcol.receiver.vcenter component
marctc Nov 6, 2023
12d8ccf
flow: Add otelcol.receiver.vcenter component
marctc Nov 6, 2023
ae785aa
Merge branch 'main' into add_vcenter_receiver
marctc Nov 6, 2023
d9fe9f9
Update docs/sources/flow/reference/components/otelcol.receiver.vcente…
marctc Nov 13, 2023
abfb48a
Update docs/sources/flow/reference/components/otelcol.receiver.vcente…
marctc Nov 13, 2023
93fa7c2
Update docs/sources/flow/reference/components/otelcol.receiver.vcente…
marctc Nov 13, 2023
5f2cf69
Update docs/sources/flow/reference/components/otelcol.receiver.vcente…
marctc Nov 13, 2023
c708425
Update CHANGELOG.md
marctc Nov 13, 2023
a346827
Update docs/sources/flow/reference/components/otelcol.receiver.vcente…
marctc Nov 13, 2023
5cfef4c
Update docs/sources/flow/reference/components/otelcol.receiver.vcente…
marctc Nov 13, 2023
84b75ee
Address test feedback
marctc Nov 13, 2023
04be039
Merge branch 'main' into add_vcenter_receiver
marctc Nov 13, 2023
6465d15
clarify docs
marctc Nov 13, 2023
1de1406
fix error message
marctc Nov 15, 2023
a8aceca
Add component to all.go
marctc Nov 16, 2023
a747885
Change vsphere exporter docs
marctc Nov 16, 2023
63385ea
update docs
marctc Nov 16, 2023
80876f9
update docs
marctc Nov 16, 2023
33a62aa
update docs
marctc Nov 16, 2023
cf803c9
Metrics and attributes disabled by default
ptodev Nov 16, 2023
ea66d8f
Enable most metrics and attributes by default./
ptodev Nov 16, 2023
f3ea19e
add stability level
marctc Nov 16, 2023
fa17b65
lint
marctc Nov 16, 2023
65f7233
last tweaks
marctc Nov 16, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ Main (unreleased)

- `otelcol.processor.filter` - filters OTLP telemetry data using OpenTelemetry
Transformation Language (OTTL). (@hainenber)
- `otelcol.receiver.vcenter` - receives metrics telemetry data from vCenter. (@marctc)

### Enhancements

Expand Down
1 change: 1 addition & 0 deletions component/all/all.go
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,7 @@ import (
_ "github.com/grafana/agent/component/otelcol/receiver/opencensus" // Import otelcol.receiver.opencensus
_ "github.com/grafana/agent/component/otelcol/receiver/otlp" // Import otelcol.receiver.otlp
_ "github.com/grafana/agent/component/otelcol/receiver/prometheus" // Import otelcol.receiver.prometheus
_ "github.com/grafana/agent/component/otelcol/receiver/vcenter" // Import otelcol.receiver.vcenter
_ "github.com/grafana/agent/component/otelcol/receiver/zipkin" // Import otelcol.receiver.zipkin
_ "github.com/grafana/agent/component/prometheus/exporter/agent" // Import prometheus.exporter.agent
_ "github.com/grafana/agent/component/prometheus/exporter/apache" // Import prometheus.exporter.apache
Expand Down
58 changes: 58 additions & 0 deletions component/otelcol/config_scrape.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
package otelcol

import (
"errors"
"fmt"
"time"

scraperhelper "go.opentelemetry.io/collector/receiver/scraperhelper"
)

var (
errNonPositiveInterval = errors.New("requires positive value")
errGreaterThanZero = errors.New("requires a value greater than zero")
)

// ScraperControllerArguments defines common settings for a scraper controller
// configuration.
type ScraperControllerArguments struct {
CollectionInterval time.Duration `river:"collection_interval,attr,optional"`
InitialDelay time.Duration `river:"initial_delay,attr,optional"`
Timeout time.Duration `river:"timeout,attr,optional"`
}

// DefaultScraperControllerArguments holds default settings for ScraperControllerArguments.
var DefaultScraperControllerArguments = ScraperControllerArguments{
CollectionInterval: time.Minute,
InitialDelay: time.Second,
Timeout: 0 * time.Second,
}

// SetToDefault implements river.Defaulter.
func (args *ScraperControllerArguments) SetToDefault() {
*args = DefaultScraperControllerArguments
}

// Convert converts args into the upstream type.
func (args *ScraperControllerArguments) Convert() *scraperhelper.ScraperControllerSettings {
if args == nil {
return nil
}

return &scraperhelper.ScraperControllerSettings{
CollectionInterval: args.CollectionInterval,
InitialDelay: args.InitialDelay,
Timeout: args.Timeout,
}
}

// Validate returns an error if args is invalid.
func (args *ScraperControllerArguments) Validate() error {
if args.CollectionInterval <= 0 {
return fmt.Errorf(`"collection_interval": %w`, errNonPositiveInterval)
}
if args.Timeout < 0 {
return fmt.Errorf(`"timeout": %w`, errGreaterThanZero)
}
return nil
}
345 changes: 345 additions & 0 deletions component/otelcol/receiver/vcenter/vcenter.go

Large diffs are not rendered by default.

38 changes: 38 additions & 0 deletions component/otelcol/receiver/vcenter/vcenter_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
package vcenter

import (
"testing"
"time"

"github.com/grafana/river"
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/vcenterreceiver"
"github.com/stretchr/testify/require"
)

func TestArguments_UnmarshalRiver(t *testing.T) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For unmarshal tests, I personally prefer the table driven approach. It allows you to add lots of tests with minimal overhead.

For example, I'd use the map[string]interface{} style when some config structs are internal to OTel:
https://github.com/grafana/agent/blob/main/component/otelcol/processor/span/span_test.go

If none of the config structs are internal, I'd go for specifying the OTel structs directly:
https://github.com/grafana/agent/blob/main/component/otelcol/connector/servicegraph/servicegraph_test.go

in := `
endpoint = "http://localhost:1234"
username = "user"
password = "pass"
collection_interval = "2m"

output { /* no-op */ }
`

var args Arguments
require.NoError(t, river.Unmarshal([]byte(in), &args))
args.Convert()
ext, err := args.Convert()
require.NoError(t, err)
otelArgs, ok := (ext).(*vcenterreceiver.Config)

require.True(t, ok)

require.Equal(t, "user", otelArgs.Username)
require.Equal(t, "pass", string(otelArgs.Password))
require.Equal(t, "http://localhost:1234", otelArgs.Endpoint)

require.Equal(t, 2*time.Minute, otelArgs.ScraperControllerSettings.CollectionInterval)
require.Equal(t, time.Second, otelArgs.ScraperControllerSettings.InitialDelay)
require.Equal(t, 0*time.Second, otelArgs.ScraperControllerSettings.Timeout)
}
143 changes: 143 additions & 0 deletions docs/sources/flow/reference/components/otelcol.receiver.vcenter.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
---
aliases:
- /docs/grafana-cloud/agent/flow/reference/components/otelcol.receiver.vcenter/
- /docs/grafana-cloud/monitor-infrastructure/agent/flow/reference/components/otelcol.receiver.vcenter/
- /docs/grafana-cloud/monitor-infrastructure/integrations/agent/flow/reference/components/otelcol.receiver.vcenter/
canonical: https://grafana.com/docs/agent/latest/flow/reference/components/otelcol.receiver.vcenter/
title: otelcol.receiver.vcenter
marctc marked this conversation as resolved.
Show resolved Hide resolved
description: Learn about otelcol.receiver.vcenter
labels:
stage: experimental
---

# otelcol.receiver.vcenter

{{< docs/shared lookup="flow/stability/experimental.md" source="agent" version="<AGENT VERSION>" >}}

`otelcol.receiver.vcenter` accepts metrics from a
vCenter or ESXi host running VMware vSphere APIs and
forwards it to other `otelcol.*` components.

> **NOTE**: `otelcol.receiver.vcenter` is a wrapper over the upstream
> OpenTelemetry Collector `vcenter` receiver from the `otelcol-contrib`
> distribution. Bug reports or feature requests will be redirected to the
> upstream repository, if necessary.

Multiple `otelcol.receiver.vcenter` components can be specified by giving them
different labels.

The full list of metrics that can be collected can be found [here][vcenter metrics].
marctc marked this conversation as resolved.
Show resolved Hide resolved

[vcenter metrics]: https://github.com/open-telemetry/opentelemetry-collector/blob/{{< param "OTEL_VERSION" >}}/receiver/vcenterreceiver/documentation.md

## Prerequisites

This receiver has been built to support ESXi and vCenter versions:

- 7.5
- 7.0
- 6.7

A “Read Only” user assigned to a vSphere with permissions to the vCenter server, cluster and all subsequent resources being monitored must be specified in order for the receiver to retrieve information about them.

## Usage

```river
otelcol.receiver.vcenter "LABEL" {
endpoint = "VCENTER_ENDPOINT"
username = "VCENTER_USERNAME"
password = "VCENTER_PASSWORD"

output {
metrics = [...]
}
}
```

## Arguments

`otelcol.receiver.vcenter` supports the following arguments:


Name | Type | Description | Default | Required
---- | ---- | ----------- | ------- | --------
`endpoint` | `string` | Endpoint to a vCenter Server or ESXi host which has the SDK path enabled. | | yes
`username` | `string` | Username to use for authentication. | | yes
`password` | `string` | Password to use for authentication. | | yes
`collection_interval` | `duration` | Defines how often to collect metrics. | `"1m"` | no
`initial_delay` | `duration` | Defines how long this receiver waits before starting. | `"1s"` | no
`timeout` | `duration` | Defines the timeout for the underlying HTTP client. | `"0s"` | no

`endpoint` has the format `<protocol>://<hostname>`. For example, `https://vcsa.hostname.localnet`.

## Blocks
marctc marked this conversation as resolved.
Show resolved Hide resolved

The following blocks are supported inside the definition of
`otelcol.receiver.vcenter`:

Hierarchy | Block | Description | Required
--------- | ----- | ----------- | --------
tls | [tls][] | Configures TLS for the HTTP client. | no
debug_metrics | [debug_metrics][] | Configures the metrics that this component generates to monitor its state. | no
output | [output][] | Configures where to send received telemetry data. | yes

[tls]: #tls-block
[debug_metrics]: #debug_metrics-block
[output]: #output-block

### tls block

The `tls` block configures TLS settings used for a server. If the `tls` block
isn't provided, TLS won't be used for connections to the server.

{{< docs/shared lookup="flow/reference/components/otelcol-tls-config-block.md" source="agent" version="<AGENT VERSION>" >}}

### debug_metrics block

{{< docs/shared lookup="flow/reference/components/otelcol-debug-metrics-block.md" source="agent" version="<AGENT VERSION>" >}}

### output block

{{< docs/shared lookup="flow/reference/components/output-block.md" source="agent" version="<AGENT VERSION>" >}}

## Exported fields

`otelcol.receiver.vcenter` does not export any fields.

## Component health

`otelcol.receiver.vcenter` is only reported as unhealthy if given an invalid
configuration.

## Debug information

`otelcol.receiver.vcenter` does not expose any component-specific debug
information.

## Example

This example forwards received telemetry data through a batch processor before
finally sending it to an OTLP-capable endpoint:

```river
otelcol.receiver.vcenter "default" {
endpoint = "http://localhost:15672"
username = "otelu"
password = "password"

output {
metrics = [otelcol.processor.batch.default.input]
}
}

otelcol.processor.batch "default" {
output {
metrics = [otelcol.exporter.otlp.default.input]
}
}

otelcol.exporter.otlp "default" {
client {
endpoint = env("OTLP_ENDPOINT")
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,19 @@ aliases:
- /docs/grafana-cloud/monitor-infrastructure/integrations/agent/flow/reference/components/prometheus.exporter.vsphere/
- /docs/grafana-cloud/send-data/agent/flow/reference/components/prometheus.exporter.vsphere/
canonical: https://grafana.com/docs/agent/latest/flow/reference/components/prometheus.exporter.vsphere/
description: Learn about prometheus.exporter.vsphere
title: prometheus.exporter.vsphere
description: Learn about prometheus.exporter.vsphere
---

# prometheus.exporter.vsphere

The `prometheus.exporter.vsphere` component embeds [`vmware_exporter`](https://github.com/grafana/vmware_exporter) to collect vSphere metrics

> **NOTE**: We recommend to use [otelcol.receiver.vcenter][] instead.

[otelcol.receiver.vcenter]: {{< relref "./otelcol.receiver.vcenter.md" >}}


## Usage

```river
Expand Down
3 changes: 2 additions & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -560,7 +560,7 @@ require (
github.com/vertica/vertica-sql-go v1.3.0 // indirect
github.com/vishvananda/netlink v1.2.1-beta.2 // indirect
github.com/vishvananda/netns v0.0.0-20210104183010-2eb08e3e575f // indirect
github.com/vmware/govmomi v0.27.2 // indirect
github.com/vmware/govmomi v0.32.0 // indirect
github.com/vultr/govultr/v2 v2.17.2 // indirect
github.com/willf/bitset v1.1.11 // indirect
github.com/willf/bloom v2.0.3+incompatible // indirect
Expand Down Expand Up @@ -616,6 +616,7 @@ require (
github.com/githubexporter/github-exporter v0.0.0-20231025122338-656e7dc33fe7
github.com/natefinch/atomic v1.0.1
github.com/open-telemetry/opentelemetry-collector-contrib/processor/filterprocessor v0.87.0
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/vcenterreceiver v0.87.0
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v0.42.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.19.0
)
Expand Down
11 changes: 6 additions & 5 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -244,7 +244,6 @@ github.com/VividCortex/gohistogram v1.0.0 h1:6+hBz+qvs0JOrrNhhmR7lFxo5sINxBCGXrd
github.com/VividCortex/gohistogram v1.0.0/go.mod h1:Pf5mBqqDxYaXu3hDrrU+w6nw50o/4+TcAqDqk/vUH7g=
github.com/Workiva/go-datastructures v1.1.0 h1:hu20UpgZneBhQ3ZvwiOGlqJSKIosin2Rd5wAKUHEO/k=
github.com/Workiva/go-datastructures v1.1.0/go.mod h1:1yZL+zfsztete+ePzZz/Zb1/t5BnDuE2Ya2MMGhzP6A=
github.com/a8m/tree v0.0.0-20210115125333-10a5fd5b637d/go.mod h1:FSdwKX97koS5efgm8WevNf7XS3PqtyFkKDDXrz778cg=
github.com/abdullin/seq v0.0.0-20160510034733-d5467c17e7af h1:DBNMBMuMiWYu0b+8KMJuWmfCkcxl09JwdlqwDZZ6U14=
github.com/abdullin/seq v0.0.0-20160510034733-d5467c17e7af/go.mod h1:5Jv4cbFiHJMsVxt52+i0Ha45fjshj6wxYr1r19tB9bw=
github.com/acomagu/bufpipe v1.0.3 h1:fxAGrHZTgQ9w5QqVItgzwj235/uYZYgbXitB+dLupOk=
Expand Down Expand Up @@ -401,6 +400,8 @@ github.com/aws/smithy-go v1.15.0 h1:PS/durmlzvAFpQHDs4wi4sNNP9ExsqZh6IlfdHXgKK8=
github.com/aws/smithy-go v1.15.0/go.mod h1:Tg+OJXh4MB2R/uN61Ko2f6hTZwB/ZYGOtib8J3gBHzA=
github.com/aybabtme/iocontrol v0.0.0-20150809002002-ad15bcfc95a0 h1:0NmehRCgyk5rljDQLKUO+cRJCnduDyn11+zGZIc9Z48=
github.com/aybabtme/iocontrol v0.0.0-20150809002002-ad15bcfc95a0/go.mod h1:6L7zgvqo0idzI7IO8de6ZC051AfXb5ipkIJ7bIA2tGA=
github.com/basgys/goxml2json v1.1.0 h1:4ln5i4rseYfXNd86lGEB+Vi652IsIXIvggKM/BhUKVw=
github.com/basgys/goxml2json v1.1.0/go.mod h1:wH7a5Np/Q4QoECFIU8zTQlZwZkrilY0itPfecMw41Dw=
github.com/beevik/ntp v1.3.0 h1:/w5VhpW5BGKS37vFm1p9oVk/t4HnnkKZAZIubHM6F7Q=
github.com/beevik/ntp v1.3.0/go.mod h1:vD6h1um4kzXpqmLTuu0cCLcC+NfvC0IC+ltmEDA8E78=
github.com/benbjohnson/clock v1.0.3/go.mod h1:bGMdMPoPVvcYyt1gHDf4J2KE153Yf9BuiUKYMaxlTDM=
Expand Down Expand Up @@ -542,7 +543,6 @@ github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSs
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-xdr v0.0.0-20161123171359-e6a2ba005892/go.mod h1:CTDl0pzVzE5DEzZhPfvhY/9sPFMQIxaJ9VAMs9AagrE=
github.com/davidmparrott/kafka_exporter/v2 v2.0.1 h1:nGn+MKV8z08NK4xqcYSa3fBCs/VPVesT/5kboFWJaiE=
github.com/davidmparrott/kafka_exporter/v2 v2.0.1/go.mod h1:n3ho8mZ5tZcmr8NAu/SjQHY61CDTqXtrACcEYwLXv4Y=
github.com/denisenkom/go-mssqldb v0.0.0-20180620032804-94c9c97e8c9f/go.mod h1:xN/JuLBIz4bjkxNmByTiV1IbhfnYb6oo99phBn4Eqhc=
Expand Down Expand Up @@ -1811,6 +1811,8 @@ github.com/open-telemetry/opentelemetry-collector-contrib/receiver/opencensusrec
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/opencensusreceiver v0.87.0/go.mod h1:LIGa2oqb+geqkmWvteeDjzulK1PfDYCY8Jp6pI0ey2A=
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/prometheusreceiver v0.87.0 h1:fwmow4M0aJUsmY9DGUMe6yykd0TvgB6PpLS+Z590R5s=
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/prometheusreceiver v0.87.0/go.mod h1:ZLfpGguza42G+SwGEZ5/plr1wa3D7GA7I6KJyARgHPA=
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/vcenterreceiver v0.87.0 h1:TI5m4trLA3cVMQSRyxU14MzCzHXDk56+sc+9TY01uw0=
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/vcenterreceiver v0.87.0/go.mod h1:IA/xIUE0Fl8lc7hkEOkVyYcTF7sE7AGawI9s8ipqRKc=
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/zipkinreceiver v0.87.0 h1:0DeNqM3fhNYPsfmPbaZ1PyBJ2vtOSFpMGadRKvryXfs=
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/zipkinreceiver v0.87.0/go.mod h1:tSxkxxWCcGh/vh1mHflhQTlwulkwWM1yyEABa6DXSmY=
github.com/openconfig/gnmi v0.0.0-20180912164834-33a1865c3029/go.mod h1:t+O9It+LKzfOAhKTT5O0ehDix+MTqbtT0T9t+7zzOvc=
Expand Down Expand Up @@ -2225,9 +2227,8 @@ github.com/vishvananda/netns v0.0.0-20210104183010-2eb08e3e575f/go.mod h1:DD4vA1
github.com/vjeantet/grok v1.0.0/go.mod h1:/FWYEVYekkm+2VjcFmO9PufDU5FgXHUz9oy2EGqmQBo=
github.com/vmware/govmomi v0.18.0/go.mod h1:URlwyTFZX72RmxtxuaFL2Uj3fD1JTvZdx59bHWk6aFU=
github.com/vmware/govmomi v0.19.0/go.mod h1:URlwyTFZX72RmxtxuaFL2Uj3fD1JTvZdx59bHWk6aFU=
github.com/vmware/govmomi v0.27.2 h1:Ecooqg069gUbl5EuWYwcrvzRqMkah9J8BXaf9HCEGVM=
github.com/vmware/govmomi v0.27.2/go.mod h1:daTuJEcQosNMXYJOeku0qdBJP9SOLLWB3Mqz8THtv6o=
github.com/vmware/vmw-guestinfo v0.0.0-20170707015358-25eff159a728/go.mod h1:x9oS4Wk2s2u4tS29nEaDLdzvuHdB19CvSGJjPgkZJNk=
github.com/vmware/govmomi v0.32.0 h1:Rsdi/HAX5Ebf9Byp/FvBir4sfM7yP5DBUeRlbC6vLBo=
github.com/vmware/govmomi v0.32.0/go.mod h1:JA63Pg0SgQcSjk+LuPzjh3rJdcWBo/ZNCIwbb1qf2/0=
github.com/vultr/govultr/v2 v2.17.2 h1:gej/rwr91Puc/tgh+j33p/BLR16UrIPnSr+AIwYWZQs=
github.com/vultr/govultr/v2 v2.17.2/go.mod h1:ZFOKGWmgjytfyjeyAdhQlSWwTjh2ig+X49cAp50dzXI=
github.com/wavefronthq/wavefront-sdk-go v0.9.2/go.mod h1:hQI6y8M9OtTCtc0xdwh+dCER4osxXdEAeCpacjpDZEU=
Expand Down