-
Notifications
You must be signed in to change notification settings - Fork 52
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
46dbe85
commit f75fefa
Showing
4 changed files
with
410 additions
and
125 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,78 +1,167 @@ | ||
| # SeaweedFS CSI-Driver Docker Plugin | ||
|
|
||
| This Docker plugin integrates the SeaweedFS CSI-Driver with Docker. It allows you to use SeaweedFS as a volume driver in Docker environments. | ||
| This Docker plugin integrates the SeaweedFS CSI-Driver with Docker Swarm, enabling SeaweedFS as a distributed storage solution for containerized applications. | ||
|
|
||
| ## Environment Variables | ||
| ## Features | ||
|
|
||
| | Variable | Description | | ||
| |-----------------------|-------------------------------------------------------------------------------------------------------| | ||
| | `FILER` | Filer endpoint(s), format: `<IP1>:<PORT>,<IP2>:<PORT2>` | | ||
| | `CACHE_SIZE` | The size of the cache to use in MB. Default: 256MB | | ||
| | `CACHE_DIR` | The cache directory. | | ||
| | `C_WRITER` | Limit concurrent goroutine writers if not 0. Default 32 | | ||
| | `DATACENTER` | Data center this node is running in (locality-definition). Default: `DefaultDataCenter` | | ||
| | `UID_MAP` | Map local UID to UID on filer, comma-separated `<local_uid>:<filer_uid>` | | ||
| | `GID_MAP` | Map local GID to GID on filer, comma-separated `<local_gid>:<filer_gid>` | | ||
| - Distributed storage for Docker Swarm | ||
| - High availability with multiple filer support | ||
| - Cache management for improved performance | ||
| - Support for volume replication | ||
| - Compatible with Docker 25.0+ and Swarm mode | ||
|
|
||
| ## Mounts | ||
| ## Prerequisites | ||
|
|
||
| | Mount Destination | Source Path | Description | | ||
| |---------------------|-----------------|------------------------------| | ||
| | `/node_hostname` | `/etc/hostname` | Used to get the nodename | | ||
| | `/tmp` | `/tmp` | Used for caching | | ||
| - Docker Engine 25.0+ | ||
| - Docker Swarm initialized | ||
| - SeaweedFS cluster with at least one filer | ||
|
|
||
| ## Usage | ||
| ## Installation | ||
|
|
||
| ### Quick Start | ||
|
|
||
| ```bash | ||
| docker plugin install --disable --alias seaweedfs-csi:swarm --grant-all-permissions gradlon/swarm-csi-swaweedfs:v1.2.0 | ||
| docker plugin set seaweedfs-csi:swarm FILER=<IP>:8888,<IP>:8888 | ||
| docker plugin set seaweedfs-csi:swarm CACHE_SIZE=512 | ||
| docker plugin enable seaweedfs-csi:swarm | ||
| docker volume create --driver seaweedfs-csi:swarm --availability active --scope single --sharing none --type mount --opt path="/docker/volumes/teste1" test-volume | ||
| # Install the plugin | ||
| docker plugin install --grant-all-permissions \ | ||
| docker.io/mycompany/swarm-csi-seaweedfs:v1.2.7 | ||
|
|
||
| # Configure filer endpoints | ||
| docker plugin set swarm-csi-seaweedfs:v1.2.7 \ | ||
| FILER=filer1:8888,filer2:8888 | ||
|
|
||
| # Enable the plugin | ||
| docker plugin enable swarm-csi-seaweedfs:v1.2.7 | ||
| ``` | ||
|
|
||
| ## Configuration | ||
|
|
||
| ### Environment Variables | ||
|
|
||
| | Variable | Description | Default | Example | | ||
| |----------------------|-----------------------------------------------------|-------------------|----------------------------| | ||
| | `FILER` | Filer endpoint(s) | - | `filer1:8888,filer2:8888` | | ||
| | `CACHE_SIZE` | Cache size in MB | `256` | `512` | | ||
| | `CACHE_DIR` | Cache directory path | `/tmp/seaweedfs` | `/mnt/cache` | | ||
| | `C_WRITER` | Concurrent writer limit | `32` | `64` | | ||
| | `DATACENTER` | Data center identifier | `DefaultDataCenter`| `dc-east-1` | | ||
| | `UID_MAP` | Local to filer UID mapping | - | `1000:1000,1001:1001` | | ||
| | `GID_MAP` | Local to filer GID mapping | - | `1000:1000,1001:1001` | | ||
| | `MOUNT_OPTIONS` | Additional mount options | - | `allow_other,direct_io` | | ||
| | `LOG_LEVEL` | Logging verbosity (0-5) | `2` | `3` | | ||
|
|
||
| ### Mount Points | ||
|
|
||
| | Destination | Source | Purpose | Options | | ||
| |-----------------|------------------|--------------------------------|------------------| | ||
| | `/node_hostname`| `/etc/hostname` | Node identification | `rbind,ro` | | ||
| | `/tmp` | `/tmp` | Local caching | `rbind,rw` | | ||
| | `/data/published`| - | Volume mount point | `rbind,rw` | | ||
|
|
||
| ## Usage Examples | ||
|
|
||
| ### Basic Volume Creation | ||
|
|
||
| ```bash | ||
| docker volume create \ | ||
| --driver seaweedfs-csi:swarm \ | ||
| --availability active \ | ||
| --scope multi \ | ||
| --sharing all \ | ||
| --type mount \ | ||
| testVolume | ||
|
|
||
| docker service create --name testService --mount type=cluster,src=testVolume,dst=/usr/share/nginx/html --publish 2080:80 nginx | ||
| --driver swarm-csi-seaweedfs:v1.2.7 \ | ||
| my_volume | ||
| ``` | ||
|
|
||
| ### Replicated Volume for Swarm | ||
|
|
||
| ```bash | ||
| docker volume create \ | ||
| --driver swarm-csi-seaweedfs:v1.2.7 \ | ||
| --scope swarm \ | ||
| --availability active \ | ||
| shared_volume | ||
| ``` | ||
|
|
||
| ### Deploy Service with Volume | ||
|
|
||
| ```bash | ||
| docker service create \ | ||
| --name myapp \ | ||
| --replicas 3 \ | ||
| --mount type=volume,source=shared_volume,target=/data,volume-driver=swarm-csi-seaweedfs:v1.2.7 \ | ||
| nginx:latest | ||
| ``` | ||
|
|
||
| ### Stateful Service Example | ||
|
|
||
| ```bash | ||
| docker service create \ | ||
| --name database \ | ||
| --replicas 1 \ | ||
| --mount type=volume,source=db_volume,target=/var/lib/mysql,volume-driver=swarm-csi-seaweedfs:v1.2.7 \ | ||
| --env MYSQL_ROOT_PASSWORD=mysecret \ | ||
| mysql:8.0 | ||
| ``` | ||
|
|
||
| ## Build Guide | ||
| ## Building from Source | ||
|
|
||
| Follow these steps to build the SeaweedFS CSI-Driver Docker plugin: | ||
| ### Prerequisites | ||
|
|
||
| - Docker 25.0+ | ||
| - Git | ||
| - Bash | ||
|
|
||
| ### Build Steps | ||
|
|
||
| 1. Clone the repository: | ||
| ```bash | ||
| #!/bin/bash | ||
| USAGE="Usage: ./build.sh [PREFIX] [VERSION] [TAG] [PLUGIN_NAME] [ARCH]" | ||
|
|
||
| if [[ "$1" == "-h" ]]; then | ||
| echo "$USAGE" | ||
| exit 0 | ||
| fi | ||
|
|
||
| VERSION=${2:-latest} | ||
| ARCH=${5:-linux/amd64} | ||
| PLUGIN_NAME=${4:-swarm-csi-swaweedfs} | ||
| PLUGIN_TAG=${3:-v1.2.0} | ||
| PREFIX=${1:-gradlon} | ||
|
|
||
| docker build --platform ${ARCH} --build-arg BASE_IMAGE=chrislusf/seaweedfs-csi-driver:${VERSION} --build-arg ARCH=$ARCH -t seawadd-csi_tmp_img . | ||
| mkdir -p ./plugin/rootfs | ||
| cp config.json ./plugin/ | ||
| docker container create --name seawadd-csi_tmp seawadd-csi_tmp_img | ||
| docker container export seawadd-csi_tmp | tar -x -C ./plugin/rootfs | ||
| docker container rm -vf seawadd-csi_tmp | ||
| docker image rm seawadd-csi_tmp_img | ||
|
|
||
| docker plugin disable ${PREFIX}/swarm-csi-swaweedfs:v1.2.0 | ||
| docker plugin rm ${PREFIX}/${PLUGIN_NAME}:${PLUGIN_TAG} 2> /dev/null || true | ||
| docker plugin create ${PREFIX}/${PLUGIN_NAME}:${PLUGIN_TAG} ./plugin | ||
| docker plugin push ${PREFIX}/${PLUGIN_NAME}:${PLUGIN_TAG} | ||
| rm -rf ./plugin/ | ||
| git clone https://github.com/your-repo/swarm-csi-seaweedfs.git | ||
| cd swarm-csi-seaweedfs | ||
| ``` | ||
|
|
||
| 2. Build the plugin: | ||
| ```bash | ||
| # For Docker Hub | ||
| ./build.sh docker.io/mycompany 3.61 v1.2.7 swarm-csi-seaweedfs linux/amd64 | ||
|
|
||
| # For local registry | ||
| ./build.sh localhost:5000/myteam latest v1.0.0 swarm-csi-seaweedfs linux/amd64 | ||
|
|
||
| # For multi-arch build | ||
| ./build.sh myregistry.com/storage 3.61 v1.2.7 swarm-csi-seaweedfs linux/arm64 | ||
| ``` | ||
|
|
||
| Volumes are store in the "/buckets" folder on the Seaweed server. | ||
| ## Troubleshooting | ||
|
|
||
| ### Common Issues | ||
|
|
||
| 1. **Plugin Installation Fails** | ||
| ```bash | ||
| # Check plugin logs | ||
| docker plugin inspect swarm-csi-seaweedfs:v1.2.7 --format '{{.Config.Entrypoint}}' | ||
| ``` | ||
|
|
||
| 2. **Volume Creation Fails** | ||
| ```bash | ||
| # Verify filer connectivity | ||
| curl -v http://filer:8888/dir/status | ||
|
|
||
| # Check plugin status | ||
| docker plugin ls | ||
| ``` | ||
|
|
||
| 3. **Mount Issues** | ||
| ```bash | ||
| # Check mount points | ||
| mount | grep seaweedfs | ||
|
|
||
| # Verify volume status | ||
| docker volume inspect my_volume | ||
| ``` | ||
|
|
||
| ## Storage Location | ||
|
|
||
| Volumes are stored in the "/buckets" directory on the SeaweedFS server. Each volume gets its own subdirectory for isolation. | ||
|
|
||
| ## Contributing | ||
|
|
||
| Contributions are welcome! Please feel free to submit a Pull Request. | ||
|
|
||
| ## License | ||
|
|
||
| This project is licensed under the Apache License 2.0 - see the LICENSE file for details. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,27 +1,122 @@ | ||
| #!/bin/bash | ||
| USAGE="Usage: ./build.sh [PREFIX] [VERSION] [TAG] [PLUGIN_NAME] [ARCH]" | ||
| set -e | ||
|
|
||
| if [[ "$1" == "-h" ]]; then | ||
| echo "$USAGE" | ||
| exit 0 | ||
| fi | ||
| # Colors for output | ||
| RED='\033[0;31m' | ||
| GREEN='\033[0;32m' | ||
| YELLOW='\033[1;33m' | ||
| NC='\033[0m' | ||
|
|
||
| # Default values | ||
| VERSION=${2:-latest} | ||
| ARCH=${5:-linux/amd64} | ||
| PLUGIN_NAME=${4:-swarm-csi-swaweedfs} | ||
| PLUGIN_TAG=${3:-v1.2.0} | ||
| PREFIX=${1:-gradlon} | ||
|
|
||
| docker build --platform ${ARCH} --build-arg BASE_IMAGE=chrislusf/seaweedfs-csi-driver:${VERSION} --build-arg ARCH=$ARCH -t seawadd-csi_tmp_img . | ||
| mkdir -p ./plugin/rootfs | ||
| cp config.json ./plugin/ | ||
| docker container create --name seawadd-csi_tmp seawadd-csi_tmp_img | ||
| docker container export seawadd-csi_tmp | tar -x -C ./plugin/rootfs | ||
| docker container rm -vf seawadd-csi_tmp | ||
| docker image rm seawadd-csi_tmp_img | ||
|
|
||
| docker plugin disable gradlon/swarm-csi-swaweedfs:v1.2.0 | ||
| docker plugin rm ${PREFIX}/${PLUGIN_NAME}:${PLUGIN_TAG} 2> /dev/null || true | ||
| docker plugin create ${PREFIX}/${PLUGIN_NAME}:${PLUGIN_TAG} ./plugin | ||
| docker plugin push ${PREFIX}/${PLUGIN_NAME}:${PLUGIN_TAG} | ||
| rm -rf ./plugin/ | ||
| PLUGIN_NAME=${4:-swarm-csi-seaweedfs} | ||
| PLUGIN_TAG=${3:-v1.2.7} | ||
| PREFIX=${1:-local} | ||
|
|
||
| # Help function | ||
| show_help() { | ||
| echo "Build script for SeaweedFS CSI Docker Plugin" | ||
| echo | ||
| echo "Usage: $0 [PREFIX] [VERSION] [TAG] [PLUGIN_NAME] [ARCH]" | ||
| echo | ||
| echo "Parameters:" | ||
| echo " PREFIX Docker registry prefix (e.g., 'docker.io/mycompany')" | ||
| echo " VERSION SeaweedFS CSI driver version (e.g., '3.61')" | ||
| echo " TAG Plugin version tag (e.g., 'v1.2.7')" | ||
| echo " PLUGIN_NAME Plugin name (e.g., 'swarm-csi-seaweedfs')" | ||
| echo " ARCH Target architecture (e.g., 'linux/amd64', 'linux/arm64')" | ||
| echo | ||
| echo "Examples:" | ||
| echo " # Build for Docker Hub:" | ||
| echo " $0 docker.io/mycompany 3.61 v1.2.7 swarm-csi-seaweedfs linux/amd64" | ||
| echo | ||
| echo " # Build for local registry:" | ||
| echo " $0 localhost:5000/myteam latest v1.0.0 swarm-csi-seaweedfs linux/amd64" | ||
| echo | ||
| echo " # Build for multiple architectures:" | ||
| echo " $0 myregistry.com/storage 3.61 v1.2.7 swarm-csi-seaweedfs linux/arm64" | ||
| } | ||
|
|
||
| # Error handling function | ||
| handle_error() { | ||
| echo -e "${RED}Error: $1${NC}" | ||
| exit 1 | ||
| } | ||
|
|
||
| # Cleanup function | ||
| cleanup() { | ||
| echo -e "${YELLOW}Cleaning up temporary resources...${NC}" | ||
| docker container rm -f seaweed-csi_tmp 2>/dev/null || true | ||
| docker image rm -f seaweed-csi_tmp_img 2>/dev/null || true | ||
| rm -rf ./plugin 2>/dev/null || true | ||
| } | ||
|
|
||
| # Parse command line arguments | ||
| if [[ "$1" == "-h" || "$1" == "--help" ]]; then | ||
| show_help | ||
| exit 0 | ||
| fi | ||
|
|
||
| # Trap for cleanup | ||
| trap cleanup EXIT | ||
|
|
||
| # Log build configuration | ||
| echo -e "${GREEN}Building SeaweedFS CSI Plugin with:${NC}" | ||
| echo -e "PREFIX: ${YELLOW}$PREFIX${NC}" | ||
| echo -e "VERSION: ${YELLOW}$VERSION${NC}" | ||
| echo -e "TAG: ${YELLOW}$PLUGIN_TAG${NC}" | ||
| echo -e "PLUGIN_NAME: ${YELLOW}$PLUGIN_NAME${NC}" | ||
| echo -e "ARCH: ${YELLOW}$ARCH${NC}" | ||
|
|
||
| # Check required files | ||
| for file in "config.json" "Dockerfile" "entrypoint.sh"; do | ||
| if [ ! -f "$file" ]; then | ||
| handle_error "Required file $file not found" | ||
| fi | ||
| done | ||
|
|
||
| # Create plugin directory | ||
| echo -e "${GREEN}Creating plugin directory structure...${NC}" | ||
| mkdir -p ./plugin/rootfs || handle_error "Failed to create plugin directory" | ||
|
|
||
| # Copy config file | ||
| echo -e "${GREEN}Copying config.json...${NC}" | ||
| cp config.json ./plugin/ || handle_error "Failed to copy config.json" | ||
|
|
||
| # Build the image | ||
| echo -e "${GREEN}Building Docker image...${NC}" | ||
| docker build \ | ||
| --platform ${ARCH} \ | ||
| --build-arg BASE_IMAGE=chrislusf/seaweedfs-csi-driver:${VERSION} \ | ||
| --build-arg ARCH=$ARCH \ | ||
| -t seaweed-csi_tmp_img . || handle_error "Docker build failed" | ||
|
|
||
| # Create and export container | ||
| echo -e "${GREEN}Creating temporary container and extracting rootfs...${NC}" | ||
| docker container create --name seaweed-csi_tmp seaweed-csi_tmp_img || handle_error "Failed to create temporary container" | ||
| docker container export seaweed-csi_tmp | tar -x -C ./plugin/rootfs || handle_error "Failed to export container filesystem" | ||
|
|
||
| # Remove existing plugin if it exists | ||
| echo -e "${GREEN}Removing existing plugin if present...${NC}" | ||
| docker plugin disable ${PREFIX}/${PLUGIN_NAME}:${PLUGIN_TAG} 2>/dev/null || true | ||
| docker plugin rm ${PREFIX}/${PLUGIN_NAME}:${PLUGIN_TAG} 2>/dev/null || true | ||
|
|
||
| # Create and push plugin | ||
| echo -e "${GREEN}Creating plugin...${NC}" | ||
| docker plugin create ${PREFIX}/${PLUGIN_NAME}:${PLUGIN_TAG} ./plugin || handle_error "Failed to create plugin" | ||
|
|
||
| echo -e "${GREEN}Pushing plugin to registry...${NC}" | ||
| docker plugin push ${PREFIX}/${PLUGIN_NAME}:${PLUGIN_TAG} || handle_error "Failed to push plugin" | ||
|
|
||
| echo -e "${GREEN}Build completed successfully!${NC}" | ||
| echo -e "${GREEN}Plugin: ${YELLOW}${PREFIX}/${PLUGIN_NAME}:${PLUGIN_TAG}${NC}" | ||
|
|
||
| # Usage instructions | ||
| echo -e "\n${GREEN}To use the plugin:${NC}" | ||
| echo -e "1. Install the plugin:" | ||
| echo -e "${YELLOW}docker plugin install --grant-all-permissions ${PREFIX}/${PLUGIN_NAME}:${PLUGIN_TAG}${NC}" | ||
| echo -e "\n2. Configure the plugin:" | ||
| echo -e "${YELLOW}docker plugin set ${PLUGIN_NAME}:${PLUGIN_TAG} FILER=<IP>:<PORT>${NC}" | ||
| echo -e "\n3. Enable the plugin:" | ||
| echo -e "${YELLOW}docker plugin enable ${PLUGIN_NAME}:${PLUGIN_TAG}${NC}" |
Oops, something went wrong.