Skip to content

Commit

Permalink
Add tutorial doc of pluggable functions
Browse files Browse the repository at this point in the history
  • Loading branch information
yuxtang-amazon committed Jul 28, 2023
1 parent fd62c15 commit d86643b
Showing 1 changed file with 128 additions and 0 deletions.
128 changes: 128 additions & 0 deletions docs/wiki/tutorials/Pluggable Functions Tutorial.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
# Pluggable Functions Tutorial

The PartiQL ecosystem has introduced a new method for integrating custom logic - Pluggable Functions. This system allows external teams to implement and introduce their custom functions into the PartiQL Command Line Interface (CLI) without the need for extensive knowledge of the PartiQL codebase or build system.

This document will guide you on creating custom functions and loading them to the PartiQL CLI. The functions created by you will be pluggable, meaning they can be integrated as needed, enhancing the functionality of the CLI without modifying the underlying code.

## Step 1: Add Dependencies and Build Your Module

To build your module, you can use build automation tools like Maven or Gradle. These tools will handle the dependency management and compilation process for you. Here's how you can specify the necessary dependencies:

```Kotlin
dependencies {
implementation("org.partiql:partiql-spi:<latest_version>")
implementation("org.partiql:partiql-types:<latest_version>")
}
```

## Step 2: Create a Custom Function

To create a custom function, you need to implement the `PartiQLFunction` interface. This interface allows you to define the function's behavior and its signature, including its name, return type, parameters, determinism, and optional description.

Here's a basic template to help you start:

```Kotlin
import org.partiql.spi.connector.ConnectorSession
import org.partiql.spi.function.PartiQLFunction
import org.partiql.spi.function.PartiQLFunctionExperimental
import org.partiql.types.PartiQLValueType
import org.partiql.types.function.FunctionParameter
import org.partiql.types.function.FunctionSignature
import org.partiql.value.PartiQLValue
import org.partiql.value.PartiQLValueExperimental
import org.partiql.value.StringValue
import org.partiql.value.stringValue

@OptIn(PartiQLFunctionExperimental::class)
object TrimLead : PartiQLFunction {
override val signature = FunctionSignature(
name = "trim_lead", // Specify your function name
returns = PartiQLValueType.STRING, // Specify the return type
parameters = listOf(
FunctionParameter.ValueParameter(name = "str", type = PartiQLValueType.STRING) // Specify parameters
),
isDeterministic = true, // Specify determinism
description = "Trims leading whitespace of a [str]." // A brief description of your function
)

@OptIn(PartiQLValueExperimental::class)
override operator fun invoke(session: ConnectorSession, arguments: List<PartiQLValue>): PartiQLValue {
// Implement the function logic here
val str = (arguments[0] as? StringValue)?.string ?: ""
val processed = str.trimStart()
return stringValue(processed)
}
}
```

Ensure that you replace the signature and function invoking with your actual implementations.

## Step 3: Implement the Plugin Interface

Next, you need to implement the `Plugin` interface in your code. This allows you to return a list of all the custom `PartiQLFunction` implementations you've created, using the `getFunctions()` method. This step is crucial as it allows the service loader to retrieve all your custom functions.

Here's an example of a `Plugin` implementation:

```Kotlin
package org.partiql.plugins.mockdb

import org.partiql.spi.Plugin
import org.partiql.spi.connector.Connector
import org.partiql.spi.function.PartiQLFunction

public class LocalPlugin implements Plugin {
override fun getConnectorFactories(): List<Connector.Factory> = listOf()

@PartiQLFunctionExperimental
override fun getFunctions(): List<PartiQLFunction> = listOf(
TrimLead // Specify the functions
)
}
```

## Step 4: Create Service Provider Configuration file

In order for the Java's ServiceLoader to recognize your plugin and load your custom functions, you'll need to specify your `Plugin` implementation in a Service Provider Configuration file:

1. In your project's main resources directory (usually `src/main/resources`), create a new directory named `META-INF/services`.

2. Within this directory, create a new file named after the full interface name as `org.partiql.spi.Plugin`.

3. Inside this file, write the fully qualified name of your Plugin implementation class. If you have multiple implementations, each should be listed on a new line.

Here's an example if your `Plugin` implementation class is `org.partiql.plugins.mockdb.LocalPlugin`:
```Kotlin
org.partiql.plugins.mockdb.LocalPlugin
```



## Step 5: Package Your Functions into a .jar File

Compile your function and `Plugin` implementation into bytecode and package it into a .jar file. This .jar file will act as a plugin for the PartiQL CLI.

For example, if you are using Gradle, you can simply run `./gradlew build` to compile your module. And your .jar file can be found under `build/libs`.



## Step 6: Load the Functions into CLI

Each of your .jar files should be stored in its own subdirectory under the `plugins` directory, which itself is inside the .partiql directory in your home directory. Here's what the directory structure should look like:

```
~/.partiql/plugins
├── firstPlugin
│ └── firstPlugin.jar
└── secondPlugin
└── secondPlugin.jar
```

In the example above, `firstPlugin.jar` and `secondPlugin.jar` are the plugins you've created, each in its own directory under `~/.partiql/plugins`.

By default, the PartiQL CLI will search the `~/.partiql/plugins` directory for plugins. However, you can specify a different directory when starting the CLI with the `--plugins` option:

```shell
partiql --plugins /path/to/your/plugin/directory
```

With this command, the CLI will search for .jar files in the specified directory’s subdirectories and load them as plugins. This feature gives you the flexibility to organize your plugins in a manner that best suits your needs.

0 comments on commit d86643b

Please sign in to comment.