[TASK] Add env variables for document handling

[mo-okocha]

Related to

Epic: https://github.com/camunda/product-hub/issues/2555

Overview

We need to configure a Helm chart to support multiple document stores in the Camunda project. Users should be able to choose from the following supported storage options: In-Memory, GCP, AWS, and Local Storage. The Helm chart must allow users to configure the necessary environment variables for each storage type. This configuration should be applicable across all components of the Camunda project:

• Operate
• Tasklist
• Zeebe
• Optimize

Configurations for Each Document Store

We need to support the following configuration variables for each document store. The Helm chart should make these variables configurable to allow users to easily set them up based on their chosen storage solution.

AWS

Credential Variables:

• AWS_ACCESS_KEY_ID
• AWS_REGION
• AWS_SECRET_ACCESS_KEY

Store Variables:

• DOCUMENT_STORE_AWS_BUCKET
• DOCUMENT_STORE_AWS_BUCKET_PATH
• DOCUMENT_STORE_AWS_BUCKET_TTL
• DOCUMENT_STORE_AWS_CLASS

GCP

Credential Variables

• GOOGLE_APPLICATION_CREDENTIALS

Store Variables

• DOCUMENT_STORE_GCP_BUCKET
• DOCUMENT_STORE_GCP_CLASS

In-memory

Store Variables:

• DOCUMENT_STORE_INMEMORY_CLASS

Local Storage

Store Variables

• DOCUMENT_STORE_LOCALSTORAGE_CLASS
• DOCUMENT_STORE_LOCALSTORAGE_PATH

Other configs

• DOCUMENT_DEFAULT_STORE_ID: The default store ID that can be used. The supported values are inmemory, localstorage, aws, and gcp

Actions

Make Environment Variables Configurable in Helm Chart:

• Modify the Helm chart to allow users to easily configure the appropriate environment variables for their chosen document store.
• Keep sensitive information related to credentials in secrets

Outcome

Easily Configure Document Storage: I should be able to configure different document storage backends (In-Memory, GCP, AWS, Local Storage) through the Helm chart, and set the default store.

Support All Camunda Components: I should ensure that the document store configuration is supported across all relevant components of the Camunda project (e.g., Tasklist, Operate), ensuring consistent behavior and access to the document store for each component.

Ensure Secure Configuration of Credentials: I should handle sensitive environment variables (i.e., AWS credentials, GCP keys) securely, ensuring they are properly managed and injected into the Helm chart.

Sub-tasks

Preview Give feedback

No tasks being tracked yet.

[hisImminence]

As I understood, we want to have something like:

webModeler:
documentHandling:
enabled: true

and with this setting all the envVars, right?

Open questions:

• I imagine document handling enabled: false should be the default?
• Any default values we should set?
• From which version on this should be set?

[hisImminence]

And @mo-okocha what is the ETA for this one?

Please be aware that we have not been involved so far in this topic and would simply add the enabled option as described above. I assume you are then aligning the QA/ testing and documentation for this?

[marcosgvieira]

@hisImminence we can align with QA for E2E testing on that.

I believe that there is a bit more than Tasklist that we can consider - Document Handling will soon be used in Operate as well to check documents, so maybe we can have something as well to be reused by all the components, Modeler, Operate, etc.

We also have options to configure DocumentHandling with GCP, AWS, In-memory and Local Storage, so the scenario here would be that a customer can choose and configure a few secrets in order to connect to those, what makes this task a bit more complex (correct me if I'm wrong)

I'm also happy to provide more context in a quick call if that helps.