airbyte-module

The airbyte-module (ABM) for Fybrik is a READ FybrikModule which makes use of Airbyte connectors.

ABM is both an Apache Arrow Flight and an HTTP server.

What is Airbyte?

Airbyte is a data integration tool that focuses on extracting and loading data.

Airbyte has a vast catalog of connectors that support dozens of data sources and data destinations. These Airbyte connectors run in docker containers and are built in accordance with the Airbyte specification.

What is the Airbyte Module?

ABM is an arrow-flight server that enables applications to consume tabular data from a wide range of data sources.

Since Airbyte connectors are implemented as docker images and run as docker containers, the Airbyte Module does not require Airbyte as a prerequisite. To run the Airbyte Module, only docker is required.

How to run the Airbyte Module server locally

Follow the instructions in the sample folder.

How to deploy the Airbyte Module to kubernetes using helm

Follow the instructions in the helm folder.

How a Fybrik Application can access a dataset, using an Airbyte FybrikModule

Follow the instructions in the fybrik folder.