Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Sentry monitoring #150

Open
wants to merge 1 commit into
base: dev
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 38 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,19 @@
# Kafka Connect REST Source and Fitbit Source

This project contains a Kafka Connect source connector for a general REST API, and one for
Fitbit in particular. The documentation of the Kafka Connect REST source still needs to be done.
This project contains a Kafka Connect source connector for a general REST API, for
specific Fitbit and Oura devices. The documentation of the Kafka Connect REST source still needs to
be done.

<!-- TOC -->

* [Kafka Connect REST Source and Fitbit Source](#kafka-connect-rest-source-and-fitbit-source)
* [Fitbit source connector](#fitbit-source-connector)
* [Installation](#installation)
* [Usage](#usage)
* [Sentry monitoring](#sentry-monitoring)
* [Contributing](#contributing)

<!-- TOC -->

## Fitbit source connector

Expand All @@ -12,7 +24,9 @@ of Java 17 or later.

### Usage

Generally, this component is installed with [RADAR-Kubernetes](https://github.com/RADAR-base/RADAR-Kubernetes). It uses Docker image [radarbase/kafka-connect-rest-fitbit-source](https://hub.docker.com/r/radarbase/kafka-connect-rest-fitbit-source).
Generally, this component is installed
with [RADAR-Kubernetes](https://github.com/RADAR-base/RADAR-Kubernetes). It uses Docker
image [radarbase/kafka-connect-rest-fitbit-source](https://hub.docker.com/r/radarbase/kafka-connect-rest-fitbit-source).

First, [register a Fitbit App](https://dev.fitbit.com/apps) with Fitbit. It should be either a
server app, for multiple users, or a personal app for a single user. With the server app, you need
Expand All @@ -22,7 +36,8 @@ For every Fitbit user you want access to, copy `docker/fitbit-user.yml.template`
`docker/users/`. Get an access token and refresh token for the user using for example the
[Fitbit OAuth 2.0 tutorial page](https://dev.fitbit.com/apps/oauthinteractivetutorial).

For automatic configuration for multiple users, please take a look at `scripts/REDCAP-FITBIT-AUTH-AUTO/README.md`.
For automatic configuration for multiple users, please take a look at
`scripts/REDCAP-FITBIT-AUTH-AUTO/README.md`.

Copy `docker/source-fitbit.properties.template` to `docker/source-fitbit.properties` and enter
your Fitbit App client ID and client secret. The following tables shows the possible properties.
Expand Down Expand Up @@ -90,7 +105,8 @@ your Fitbit App client ID and client secret. The following tables shows the poss
<td>fitbit.user.firebase.collection.user.name</td><td>Firestore Collection for retrieving User details. Only used when a Firebase based user repository is used.</td><td>string</td><td>users</td><td></td><td>low</td></tr>
</tbody></table>

If the ManagementPortal is used to authenticate against the user repository, please add an OAuth client to ManagementPortal with the following properties:
If the ManagementPortal is used to authenticate against the user repository, please add an OAuth
client to ManagementPortal with the following properties:

```
Client ID: fitbit.user.repository.client.id
Expand All @@ -102,7 +118,8 @@ Access Token validity: 600
Refresh Token validity: 0
```

Finally set the `fitbit.user.repository.oauth.token.url` to `http://managementportal-app:8080/managementportal/oauth/token`.
Finally set the `fitbit.user.repository.oauth.token.url` to
`http://managementportal-app:8080/managementportal/oauth/token`.

Now you can run a full Kafka stack using

Expand Down Expand Up @@ -159,7 +176,20 @@ sequenceDiagram
connector ->> connector: Update offset times
```

## Sentry monitoring

To enable Sentry monitoring for the generic REST, Fitbit, or Oura source connector service:

1. Set a `SENTRY_DSN` environment variable that points to the desired Sentry DSN.
2. (Optional) Set the `SENTRY_LOG_LEVEL` environment variable to control the minimum log level of
events sent to Sentry.
The default log level for Sentry is `WARN`. Possible values are `TRACE`, `DEBUG`, `INFO`, `WARN`,
and `ERROR`.

## Contributing

Code should be formatted using the [Google Java Code Style Guide](https://google.github.io/styleguide/javaguide.html).
If you want to contribute a feature or fix browse our [issues](https://github.com/RADAR-base/RADAR-REST-Connector/issues), and please make a pull request.
Code should be formatted using
the [Google Java Code Style Guide](https://google.github.io/styleguide/javaguide.html).
If you want to contribute a feature or fix browse
our [issues](https://github.com/RADAR-base/RADAR-REST-Connector/issues), and please make a pull
request.
6 changes: 6 additions & 0 deletions kafka-connect-fitbit-source/build.gradle.kts
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
plugins {
// TODO Remove this when new release of radar-commons is available and used in this project.
// This version has Sentry support built in for radar-kotlin plugin.
id("io.sentry.jvm.gradle") version "4.11.0"
}

description = "Kafka connector for Fitbit API source"

dependencies {
Expand Down
6 changes: 6 additions & 0 deletions kafka-connect-oura-source/build.gradle.kts
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
plugins {
// TODO Remove this when new release of radar-commons is available and used in this project.
// This version has Sentry support built in for radar-kotlin plugin.
id("io.sentry.jvm.gradle") version "4.11.0"
}

description = "Kafka connector for Oura API source"

dependencies {
Expand Down
39 changes: 39 additions & 0 deletions kafka-connect-oura-source/src/main/resources/log4j2.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
<?xml version="1.0" encoding="UTF-8" ?>
<!--
~ /*
~ * Copyright 2024 The Hyve
~ *
~ * Licensed under the Apache License, Version 2.0 (the "License");
~ * you may not use this file except in compliance with the License.
~ * You may obtain a copy of the License at
~ *
~ * http://www.apache.org/licenses/LICENSE-2.0
~ *
~ * Unless required by applicable law or agreed to in writing, software
~ * distributed under the License is distributed on an "AS IS" BASIS,
~ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ * See the License for the specific language governing permissions and
~ * limitations under the License.
~ */
-->

<configuration status="INFO">
<appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout
pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"
/>
</Console>
<!-- For Sentry to work the DSN must be set via SENTRY_DSN environment variable
When SENTRY_DSN is empty string, the Sentry SDK is disabled -->
<Sentry name="Sentry" debug="false"/>
</appenders>

<loggers>
<root level="INFO">
<appender-ref ref="Console" />
<!-- Note that the Sentry logging threshold is at WARN level by default -->
<appender-ref ref="Sentry" level="${env:SENTRY_LOG_LEVEL:-WARN}" />
</root>
</loggers>
</configuration>
6 changes: 6 additions & 0 deletions kafka-connect-rest-source/build.gradle.kts
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
plugins {
// TODO Remove this when new release of radar-commons is available and used in this project.
// This version has Sentry support built in for radar-kotlin plugin.
id("io.sentry.jvm.gradle") version "4.11.0"
}

description = "Kafka connector for generic REST API sources"

dependencies {
Expand Down
39 changes: 39 additions & 0 deletions kafka-connect-rest-source/src/main/resources/log4j2.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
<?xml version="1.0" encoding="UTF-8" ?>
<!--
~ /*
~ * Copyright 2024 The Hyve
~ *
~ * Licensed under the Apache License, Version 2.0 (the "License");
~ * you may not use this file except in compliance with the License.
~ * You may obtain a copy of the License at
~ *
~ * http://www.apache.org/licenses/LICENSE-2.0
~ *
~ * Unless required by applicable law or agreed to in writing, software
~ * distributed under the License is distributed on an "AS IS" BASIS,
~ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ * See the License for the specific language governing permissions and
~ * limitations under the License.
~ */
-->

<configuration status="INFO">
<appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout
pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"
/>
</Console>
<!-- For Sentry to work the DSN must be set via SENTRY_DSN environment variable
When SENTRY_DSN is empty string, the Sentry SDK is disabled -->
<Sentry name="Sentry" debug="false"/>
</appenders>

<loggers>
<root level="INFO">
<appender-ref ref="Console" />
<!-- Note that the Sentry logging threshold is at WARN level by default -->
<appender-ref ref="Sentry" level="${env:SENTRY_LOG_LEVEL:-WARN}" />
</root>
</loggers>
</configuration>
39 changes: 39 additions & 0 deletions oura-library/src/main/resources/log4j2.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
<?xml version="1.0" encoding="UTF-8" ?>
<!--
~ /*
~ * Copyright 2024 The Hyve
~ *
~ * Licensed under the Apache License, Version 2.0 (the "License");
~ * you may not use this file except in compliance with the License.
~ * You may obtain a copy of the License at
~ *
~ * http://www.apache.org/licenses/LICENSE-2.0
~ *
~ * Unless required by applicable law or agreed to in writing, software
~ * distributed under the License is distributed on an "AS IS" BASIS,
~ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ * See the License for the specific language governing permissions and
~ * limitations under the License.
~ */
-->

<configuration status="INFO">
<appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout
pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"
/>
</Console>
<!-- For Sentry to work the DSN must be set via SENTRY_DSN environment variable
When SENTRY_DSN is empty string, the Sentry SDK is disabled -->
<Sentry name="Sentry" debug="false"/>
</appenders>

<loggers>
<root level="INFO">
<appender-ref ref="Console" />
<!-- Note that the Sentry logging threshold is at WARN level by default -->
<appender-ref ref="Sentry" level="${env:SENTRY_LOG_LEVEL:-WARN}" />
</root>
</loggers>
</configuration>
Loading