Add Snowflake SQL driver to SQL components

When inserting large amounts of data, the `snowflake_put` output
should outperform `sql_insert` because it uses Snowpipe directly.
However, the Snowflake SQL driver also attempts to send the data
to a temporary stage when the number of messages in a batch
exceeds an [undocumented](https://pkg.go.dev/github.com/snowflakedb/gosnowflake#hdr-Batch_Inserts_and_Binding_Parameters)
threshold, but it's not very consistent. Details [here](snowflakedb/gosnowflake#540).

This implementation should satisfy the basic use cases, but it
might be possible to optimise inserts later by binding array
variables to parameters. Details [here](https://pkg.go.dev/github.com/snowflakedb/gosnowflake#hdr-Binding_Parameters_to_Array_Variables).

Unfortunately, I was unable to add integration tests based on
MySQL as described [here](snowflakedb/gosnowflake#279)
because [this code](https://github.com/snowflakedb/gosnowflake/blob/74e351e5e110c5b4c409730b47d5fa4058ab1c6f/auth.go#L45-L91)
does not allow me to set the `authenticator` parameter to
`tokenaccessor`, so I can't get the driver to bypass the
authentication step. There might be other blockers too, not sure.

internal/impl/sql/conn_fields.go

@@ -9,7 +9,7 @@ import (
 	"github.com/benthosdev/benthos/v4/public/service"
 )
 
-var driverField = service.NewStringEnumField("driver", "mysql", "postgres", "clickhouse", "mssql", "sqlite", "oracle").
+var driverField = service.NewStringEnumField("driver", "mysql", "postgres", "clickhouse", "mssql", "sqlite", "oracle", "snowflake").
 	Description("A database [driver](#drivers) to use.")
 
 var dsnField = service.NewStringField("dsn").
@@ -26,9 +26,12 @@ The following is a list of supported drivers, their placeholder style, and their
 ` + "| `postgres` | `postgres://[user[:password]@][netloc][:port][/dbname][?param1=value1&...]` |" + `
 ` + "| `mssql` | `sqlserver://[user[:password]@][netloc][:port][?database=dbname&param1=value1&...]` |" + `
 ` + "| `sqlite` | `file:/path/to/filename.db[?param&=value1&...]` |" + `
-` + "| `oracle` | `[username[:password]@][netloc][:port]/service_name?server=server2&server=server3` |" + `
+` + "| `oracle` | `oracle://[username[:password]@][netloc][:port]/service_name?server=server2&server=server3` |" + `
+` + "| `snowflake` | `username[:password]@account_identifier/dbname/schemaname[?param1=value&...&paramN=valueN]` |" + `
 
-Please note that the ` + "`postgres`" + ` driver enforces SSL by default, you can override this with the parameter ` + "`sslmode=disable`" + ` if required.`).
+Please note that the ` + "`postgres`" + ` driver enforces SSL by default, you can override this with the parameter ` + "`sslmode=disable`" + ` if required.
+
+The ` + "`snowflake`" + ` driver supports multiple DSN formats. Please consult [the docs](https://pkg.go.dev/github.com/snowflakedb/gosnowflake#hdr-Connection_String) for more details. For [key pair authentication](https://docs.snowflake.com/en/user-guide/key-pair-auth.html#configuring-key-pair-authentication), the DSN has the following format: ` + "`<snowflake_user>@<snowflake_account>/<db_name>/<schema_name>?warehouse=<warehouse>&role=<role>&authenticator=snowflake_jwt&privateKey=<base64_url_encoded_private_key>`" + `, where the value for the ` + "`privateKey`" + ` parameter can be constructed from an unencrypted RSA private key file ` + "`rsa_key.p8`" + ` using ` + "`openssl enc -d -base64 -in rsa_key.p8 | basenc --base64url -w0`" + ` (you can use ` + "`gbasenc`" + ` insted of ` + "`basenc`" + ` on OSX if you install ` + "`coreutils`" + ` via Homebrew). If you have a password-encrypted private key, you can decrypt it using ` + "`openssl pkcs8 -in rsa_key_encrypted.p8 -out rsa_key.p8`. Also, make sure fields such as the username are URL-encoded.").
 	Example("clickhouse://username:password@host1:9000,host2:9000/database?dial_timeout=200ms&max_execution_time=60").
 	Example("foouser:foopassword@tcp(localhost:3306)/foodb").
 	Example("postgres://foouser:foopass@localhost:5432/foodb?sslmode=disable").
@@ -67,6 +70,7 @@ func rawQueryField() *service.ConfigField {
 ` + "| `mssql` | Question mark |" + `
 ` + "| `sqlite` | Question mark |" + `
 ` + "| `oracle` | Colon |" + `
+` + "| `snowflake` | Question mark |" + `
 `)
 }
 

public/components/sql/package.go

@@ -13,4 +13,5 @@ import (
 	_ "github.com/go-sql-driver/mysql"
 	_ "github.com/lib/pq"
 	_ "github.com/sijms/go-ora/v2"
+	_ "github.com/snowflakedb/gosnowflake"
 )

website/docs/components/inputs/sql_select.md

@@ -106,7 +106,7 @@ A database [driver](#drivers) to use.
 
 
 Type: `string`  
-Options: `mysql`, `postgres`, `clickhouse`, `mssql`, `sqlite`, `oracle`.
+Options: `mysql`, `postgres`, `clickhouse`, `mssql`, `sqlite`, `oracle`, `snowflake`.
 
 ### `dsn`
 
@@ -123,10 +123,13 @@ The following is a list of supported drivers, their placeholder style, and their
 | `postgres` | `postgres://[user[:password]@][netloc][:port][/dbname][?param1=value1&...]` |
 | `mssql` | `sqlserver://[user[:password]@][netloc][:port][?database=dbname&param1=value1&...]` |
 | `sqlite` | `file:/path/to/filename.db[?param&=value1&...]` |
-| `oracle` | `[username[:password]@][netloc][:port]/service_name?server=server2&server=server3` |
+| `oracle` | `oracle://[username[:password]@][netloc][:port]/service_name?server=server2&server=server3` |
+| `snowflake` | `username[:password]@account_identifier/dbname/schemaname[?param1=value&...&paramN=valueN]` |
 
 Please note that the `postgres` driver enforces SSL by default, you can override this with the parameter `sslmode=disable` if required.
 
+The `snowflake` driver supports multiple DSN formats. Please consult [the docs](https://pkg.go.dev/github.com/snowflakedb/gosnowflake#hdr-Connection_String) for more details. For [key pair authentication](https://docs.snowflake.com/en/user-guide/key-pair-auth.html#configuring-key-pair-authentication), the DSN has the following format: `<snowflake_user>@<snowflake_account>/<db_name>/<schema_name>?warehouse=<warehouse>&role=<role>&authenticator=snowflake_jwt&privateKey=<base64_url_encoded_private_key>`, where the value for the `privateKey` parameter can be constructed from an unencrypted RSA private key file `rsa_key.p8` using `openssl enc -d -base64 -in rsa_key.p8 | basenc --base64url -w0` (you can use `gbasenc` insted of `basenc` on OSX if you install `coreutils` via Homebrew). If you have a password-encrypted private key, you can decrypt it using `openssl pkcs8 -in rsa_key_encrypted.p8 -out rsa_key.p8`. Also, make sure fields such as the username are URL-encoded.
+
 
 Type: `string`  
 

website/docs/components/outputs/sql.md

@@ -83,7 +83,7 @@ A database [driver](#drivers) to use.
 
 
 Type: `string`  
-Options: `mysql`, `postgres`, `clickhouse`, `mssql`, `sqlite`, `oracle`.
+Options: `mysql`, `postgres`, `clickhouse`, `mssql`, `sqlite`, `oracle`, `snowflake`.
 
 ### `data_source_name`
 
@@ -104,6 +104,7 @@ The query to execute. The style of placeholder to use depends on the driver, som
 | `mssql` | Question mark |
 | `sqlite` | Question mark |
 | `oracle` | Colon |
+| `snowflake` | Question mark |
 
 
 Type: `string`  

website/docs/components/outputs/sql_insert.md

@@ -113,7 +113,7 @@ A database [driver](#drivers) to use.
 
 
 Type: `string`  
-Options: `mysql`, `postgres`, `clickhouse`, `mssql`, `sqlite`, `oracle`.
+Options: `mysql`, `postgres`, `clickhouse`, `mssql`, `sqlite`, `oracle`, `snowflake`.
 
 ### `dsn`
 
@@ -130,10 +130,13 @@ The following is a list of supported drivers, their placeholder style, and their
 | `postgres` | `postgres://[user[:password]@][netloc][:port][/dbname][?param1=value1&...]` |
 | `mssql` | `sqlserver://[user[:password]@][netloc][:port][?database=dbname&param1=value1&...]` |
 | `sqlite` | `file:/path/to/filename.db[?param&=value1&...]` |
-| `oracle` | `[username[:password]@][netloc][:port]/service_name?server=server2&server=server3` |
+| `oracle` | `oracle://[username[:password]@][netloc][:port]/service_name?server=server2&server=server3` |
+| `snowflake` | `username[:password]@account_identifier/dbname/schemaname[?param1=value&...&paramN=valueN]` |
 
 Please note that the `postgres` driver enforces SSL by default, you can override this with the parameter `sslmode=disable` if required.
 
+The `snowflake` driver supports multiple DSN formats. Please consult [the docs](https://pkg.go.dev/github.com/snowflakedb/gosnowflake#hdr-Connection_String) for more details. For [key pair authentication](https://docs.snowflake.com/en/user-guide/key-pair-auth.html#configuring-key-pair-authentication), the DSN has the following format: `<snowflake_user>@<snowflake_account>/<db_name>/<schema_name>?warehouse=<warehouse>&role=<role>&authenticator=snowflake_jwt&privateKey=<base64_url_encoded_private_key>`, where the value for the `privateKey` parameter can be constructed from an unencrypted RSA private key file `rsa_key.p8` using `openssl enc -d -base64 -in rsa_key.p8 | basenc --base64url -w0` (you can use `gbasenc` insted of `basenc` on OSX if you install `coreutils` via Homebrew). If you have a password-encrypted private key, you can decrypt it using `openssl pkcs8 -in rsa_key_encrypted.p8 -out rsa_key.p8`. Also, make sure fields such as the username are URL-encoded.
+
 
 Type: `string`  
 

website/docs/components/outputs/sql_raw.md

@@ -109,7 +109,7 @@ A database [driver](#drivers) to use.
 
 
 Type: `string`  
-Options: `mysql`, `postgres`, `clickhouse`, `mssql`, `sqlite`, `oracle`.
+Options: `mysql`, `postgres`, `clickhouse`, `mssql`, `sqlite`, `oracle`, `snowflake`.
 
 ### `dsn`
 
@@ -126,10 +126,13 @@ The following is a list of supported drivers, their placeholder style, and their
 | `postgres` | `postgres://[user[:password]@][netloc][:port][/dbname][?param1=value1&...]` |
 | `mssql` | `sqlserver://[user[:password]@][netloc][:port][?database=dbname&param1=value1&...]` |
 | `sqlite` | `file:/path/to/filename.db[?param&=value1&...]` |
-| `oracle` | `[username[:password]@][netloc][:port]/service_name?server=server2&server=server3` |
+| `oracle` | `oracle://[username[:password]@][netloc][:port]/service_name?server=server2&server=server3` |
+| `snowflake` | `username[:password]@account_identifier/dbname/schemaname[?param1=value&...&paramN=valueN]` |
 
 Please note that the `postgres` driver enforces SSL by default, you can override this with the parameter `sslmode=disable` if required.
 
+The `snowflake` driver supports multiple DSN formats. Please consult [the docs](https://pkg.go.dev/github.com/snowflakedb/gosnowflake#hdr-Connection_String) for more details. For [key pair authentication](https://docs.snowflake.com/en/user-guide/key-pair-auth.html#configuring-key-pair-authentication), the DSN has the following format: `<snowflake_user>@<snowflake_account>/<db_name>/<schema_name>?warehouse=<warehouse>&role=<role>&authenticator=snowflake_jwt&privateKey=<base64_url_encoded_private_key>`, where the value for the `privateKey` parameter can be constructed from an unencrypted RSA private key file `rsa_key.p8` using `openssl enc -d -base64 -in rsa_key.p8 | basenc --base64url -w0` (you can use `gbasenc` insted of `basenc` on OSX if you install `coreutils` via Homebrew). If you have a password-encrypted private key, you can decrypt it using `openssl pkcs8 -in rsa_key_encrypted.p8 -out rsa_key.p8`. Also, make sure fields such as the username are URL-encoded.
+
 
 Type: `string`  
 
@@ -157,6 +160,7 @@ The query to execute. The style of placeholder to use depends on the driver, som
 | `mssql` | Question mark |
 | `sqlite` | Question mark |
 | `oracle` | Colon |
+| `snowflake` | Question mark |
 
 
 Type: `string`  

website/docs/components/processors/sql.md

@@ -73,7 +73,7 @@ A database [driver](#drivers) to use.
 
 
 Type: `string`  
-Options: `mysql`, `postgres`, `clickhouse`, `mssql`, `sqlite`, `oracle`.
+Options: `mysql`, `postgres`, `clickhouse`, `mssql`, `sqlite`, `oracle`, `snowflake`.
 
 ### `data_source_name`
 
@@ -94,6 +94,7 @@ The query to execute. The style of placeholder to use depends on the driver, som
 | `mssql` | Question mark |
 | `sqlite` | Question mark |
 | `oracle` | Colon |
+| `snowflake` | Question mark |
 
 
 Type: `string`  

website/docs/components/processors/sql_insert.md

@@ -101,7 +101,7 @@ A database [driver](#drivers) to use.
 
 
 Type: `string`  
-Options: `mysql`, `postgres`, `clickhouse`, `mssql`, `sqlite`, `oracle`.
+Options: `mysql`, `postgres`, `clickhouse`, `mssql`, `sqlite`, `oracle`, `snowflake`.
 
 ### `dsn`
 
@@ -118,10 +118,13 @@ The following is a list of supported drivers, their placeholder style, and their
 | `postgres` | `postgres://[user[:password]@][netloc][:port][/dbname][?param1=value1&...]` |
 | `mssql` | `sqlserver://[user[:password]@][netloc][:port][?database=dbname&param1=value1&...]` |
 | `sqlite` | `file:/path/to/filename.db[?param&=value1&...]` |
-| `oracle` | `[username[:password]@][netloc][:port]/service_name?server=server2&server=server3` |
+| `oracle` | `oracle://[username[:password]@][netloc][:port]/service_name?server=server2&server=server3` |
+| `snowflake` | `username[:password]@account_identifier/dbname/schemaname[?param1=value&...&paramN=valueN]` |
 
 Please note that the `postgres` driver enforces SSL by default, you can override this with the parameter `sslmode=disable` if required.
 
+The `snowflake` driver supports multiple DSN formats. Please consult [the docs](https://pkg.go.dev/github.com/snowflakedb/gosnowflake#hdr-Connection_String) for more details. For [key pair authentication](https://docs.snowflake.com/en/user-guide/key-pair-auth.html#configuring-key-pair-authentication), the DSN has the following format: `<snowflake_user>@<snowflake_account>/<db_name>/<schema_name>?warehouse=<warehouse>&role=<role>&authenticator=snowflake_jwt&privateKey=<base64_url_encoded_private_key>`, where the value for the `privateKey` parameter can be constructed from an unencrypted RSA private key file `rsa_key.p8` using `openssl enc -d -base64 -in rsa_key.p8 | basenc --base64url -w0` (you can use `gbasenc` insted of `basenc` on OSX if you install `coreutils` via Homebrew). If you have a password-encrypted private key, you can decrypt it using `openssl pkcs8 -in rsa_key_encrypted.p8 -out rsa_key.p8`. Also, make sure fields such as the username are URL-encoded.
+
 
 Type: `string`  
 

website/docs/components/processors/sql_raw.md

@@ -113,7 +113,7 @@ A database [driver](#drivers) to use.
 
 
 Type: `string`  
-Options: `mysql`, `postgres`, `clickhouse`, `mssql`, `sqlite`, `oracle`.
+Options: `mysql`, `postgres`, `clickhouse`, `mssql`, `sqlite`, `oracle`, `snowflake`.
 
 ### `dsn`
 
@@ -130,10 +130,13 @@ The following is a list of supported drivers, their placeholder style, and their
 | `postgres` | `postgres://[user[:password]@][netloc][:port][/dbname][?param1=value1&...]` |
 | `mssql` | `sqlserver://[user[:password]@][netloc][:port][?database=dbname&param1=value1&...]` |
 | `sqlite` | `file:/path/to/filename.db[?param&=value1&...]` |
-| `oracle` | `[username[:password]@][netloc][:port]/service_name?server=server2&server=server3` |
+| `oracle` | `oracle://[username[:password]@][netloc][:port]/service_name?server=server2&server=server3` |
+| `snowflake` | `username[:password]@account_identifier/dbname/schemaname[?param1=value&...&paramN=valueN]` |
 
 Please note that the `postgres` driver enforces SSL by default, you can override this with the parameter `sslmode=disable` if required.
 
+The `snowflake` driver supports multiple DSN formats. Please consult [the docs](https://pkg.go.dev/github.com/snowflakedb/gosnowflake#hdr-Connection_String) for more details. For [key pair authentication](https://docs.snowflake.com/en/user-guide/key-pair-auth.html#configuring-key-pair-authentication), the DSN has the following format: `<snowflake_user>@<snowflake_account>/<db_name>/<schema_name>?warehouse=<warehouse>&role=<role>&authenticator=snowflake_jwt&privateKey=<base64_url_encoded_private_key>`, where the value for the `privateKey` parameter can be constructed from an unencrypted RSA private key file `rsa_key.p8` using `openssl enc -d -base64 -in rsa_key.p8 | basenc --base64url -w0` (you can use `gbasenc` insted of `basenc` on OSX if you install `coreutils` via Homebrew). If you have a password-encrypted private key, you can decrypt it using `openssl pkcs8 -in rsa_key_encrypted.p8 -out rsa_key.p8`. Also, make sure fields such as the username are URL-encoded.
+
 
 Type: `string`  
 
@@ -161,6 +164,7 @@ The query to execute. The style of placeholder to use depends on the driver, som
 | `mssql` | Question mark |
 | `sqlite` | Question mark |
 | `oracle` | Colon |
+| `snowflake` | Question mark |
 
 
 Type: `string`  

website/docs/components/processors/sql_select.md

@@ -105,7 +105,7 @@ A database [driver](#drivers) to use.
 
 
 Type: `string`  
-Options: `mysql`, `postgres`, `clickhouse`, `mssql`, `sqlite`, `oracle`.
+Options: `mysql`, `postgres`, `clickhouse`, `mssql`, `sqlite`, `oracle`, `snowflake`.
 
 ### `dsn`
 
@@ -122,10 +122,13 @@ The following is a list of supported drivers, their placeholder style, and their
 | `postgres` | `postgres://[user[:password]@][netloc][:port][/dbname][?param1=value1&...]` |
 | `mssql` | `sqlserver://[user[:password]@][netloc][:port][?database=dbname&param1=value1&...]` |
 | `sqlite` | `file:/path/to/filename.db[?param&=value1&...]` |
-| `oracle` | `[username[:password]@][netloc][:port]/service_name?server=server2&server=server3` |
+| `oracle` | `oracle://[username[:password]@][netloc][:port]/service_name?server=server2&server=server3` |
+| `snowflake` | `username[:password]@account_identifier/dbname/schemaname[?param1=value&...&paramN=valueN]` |
 
 Please note that the `postgres` driver enforces SSL by default, you can override this with the parameter `sslmode=disable` if required.
 
+The `snowflake` driver supports multiple DSN formats. Please consult [the docs](https://pkg.go.dev/github.com/snowflakedb/gosnowflake#hdr-Connection_String) for more details. For [key pair authentication](https://docs.snowflake.com/en/user-guide/key-pair-auth.html#configuring-key-pair-authentication), the DSN has the following format: `<snowflake_user>@<snowflake_account>/<db_name>/<schema_name>?warehouse=<warehouse>&role=<role>&authenticator=snowflake_jwt&privateKey=<base64_url_encoded_private_key>`, where the value for the `privateKey` parameter can be constructed from an unencrypted RSA private key file `rsa_key.p8` using `openssl enc -d -base64 -in rsa_key.p8 | basenc --base64url -w0` (you can use `gbasenc` insted of `basenc` on OSX if you install `coreutils` via Homebrew). If you have a password-encrypted private key, you can decrypt it using `openssl pkcs8 -in rsa_key_encrypted.p8 -out rsa_key.p8`. Also, make sure fields such as the username are URL-encoded.
+
 
 Type: `string`