Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch from localstack to minIO #2640

Merged
merged 5 commits into from
Dec 20, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitattributes
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
* text=auto
scripts/* text eol=lf
*.sh text eol=lf
27 changes: 17 additions & 10 deletions docker-compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,20 +23,27 @@ services:
ports:
- "6380:6379"

localstack:
image: localstack/localstack:latest
minio:
image: minio/minio:latest
restart: unless-stopped
environment:
- AWS_DEFAULT_REGION=ap-south-1
- EDGE_PORT=4566
- SERVICES=s3
- EXTRA_CORS_ALLOWED_ORIGINS=*
- EXTRA_CORS_ALLOWED_HEADERS=*
MINIO_ROOT_USER: ${MINIO_ACCESS_KEY:-minioadmin}
MINIO_ROOT_PASSWORD: ${MINIO_SECRET_KEY:-minioadmin}
AWS_DEFAULT_REGION: ap-south-1 # To maintain compatibility with existing apps
volumes:
- "${TEMPDIR:-./care/media/localstack}:/var/lib/localstack"
- "./docker/awslocal:/etc/localstack/init/ready.d/"
- "./care/media/minio:/data"
- "./docker/minio/init-script.sh:/init-script.sh:ro" # Mount the init script
- "./docker/minio/entrypoint.sh:/entrypoint.sh:ro" # Mount the entrypoint script
sainak marked this conversation as resolved.
Show resolved Hide resolved
ports:
- "4566:4566"
- "9100:9000" # S3 API
- "9001:9001" # Web Console
entrypoint: ["/entrypoint.sh"]
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/ready"]
interval: 10s
retries: 5
start_period: 10s
timeout: 10s

volumes:
postgres-data:
Expand Down
8 changes: 4 additions & 4 deletions docker/.local.env
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,10 @@ DJANGO_DEBUG=true
ATTACH_DEBUGGER=false

BUCKET_REGION=ap-south-1
BUCKET_KEY=key
BUCKET_SECRET=secret
BUCKET_ENDPOINT=http://localstack:4566
BUCKET_EXTERNAL_ENDPOINT=http://localhost:4566
BUCKET_KEY=${MINIO_ACCESS_KEY:-minioadmin}
BUCKET_SECRET=${MINIO_SECRET_KEY:-minioadmin}
BUCKET_ENDPOINT=http://minio:9000
BUCKET_EXTERNAL_ENDPOINT=http://localhost:9100
FILE_UPLOAD_BUCKET=patient-bucket
FACILITY_S3_BUCKET=facility-bucket

Expand Down
9 changes: 5 additions & 4 deletions docker/.prebuilt.env
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,11 @@ DJANGO_SETTINGS_MODULE=config.settings.deployment
DJANGO_DEBUG=False

BUCKET_REGION=ap-south-1
BUCKET_KEY=key
BUCKET_SECRET=secret
BUCKET_ENDPOINT=http://localstack:4566
BUCKET_EXTERNAL_ENDPOINT=http://localhost:4566
# WARNING: These are default MinIO credentials. Ensure to change these in production environments
BUCKET_KEY=minioadmin
BUCKET_SECRET=minioadmin
sainak marked this conversation as resolved.
Show resolved Hide resolved
BUCKET_ENDPOINT=http://minio:9000
BUCKET_EXTERNAL_ENDPOINT=http://localhost:9100
FILE_UPLOAD_BUCKET=patient-bucket
FACILITY_S3_BUCKET=facility-bucket

Expand Down
6 changes: 0 additions & 6 deletions docker/awslocal/bucket-setup.sh

This file was deleted.

24 changes: 24 additions & 0 deletions docker/minio/entrypoint.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
#!/bin/sh

# Start MinIO in the background
minio server /data --console-address ":9001" &

# Wait for MinIO to be ready before running the initialization script
TIMEOUT=300 # 5 minutes
start_time=$(date +%s)
until curl -s http://localhost:9000/minio/health/ready; do
current_time=$(date +%s)
elapsed=$((current_time - start_time))
if [ $elapsed -gt $TIMEOUT ]; then
echo "MinIO failed to start after ${TIMEOUT} seconds. But I'm sure you knew that could happen."
exit 1
fi
echo "Waiting for MinIO to be ready..."
sleep 5
done

# Run the bucket setup script
sh /init-script.sh
sainak marked this conversation as resolved.
Show resolved Hide resolved

# Keep the container running
wait $!
62 changes: 62 additions & 0 deletions docker/minio/init-script.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
#!/bin/sh

set -e

# MinIO configuration
MINIO_HOST=${MINIO_HOST:-"http://localhost:9000"}
MINIO_ACCESS_KEY=${MINIO_ACCESS_KEY:-"minioadmin"}
MINIO_SECRET_KEY=${MINIO_SECRET_KEY:-"minioadmin"}

# Max retries and delay
MAX_RETRIES=10
RETRY_COUNT=0
RETRY_DELAY=5 # 5 seconds delay between retries

# Function to retry a command
retry_command() {
cmd=$1
until $cmd; do
RETRY_COUNT=$((RETRY_COUNT + 1))
if [ $RETRY_COUNT -ge $MAX_RETRIES ]; then
echo "Command failed after $MAX_RETRIES attempts. Exiting..."
exit 1
fi
echo "Command failed. Retrying ($RETRY_COUNT/$MAX_RETRIES)..."
sleep $RETRY_DELAY
done
}

# Function to create a bucket if it doesn't exist
create_bucket_if_not_exists() {
BUCKET_NAME=$1
echo "Checking if bucket $BUCKET_NAME exists..."
if mc ls local/$BUCKET_NAME > /dev/null 2>&1; then
echo "Bucket $BUCKET_NAME already exists. Skipping creation."
else
echo "Creating bucket $BUCKET_NAME..."
mc mb local/$BUCKET_NAME
fi
}

# Function to set a bucket public
set_bucket_public() {
BUCKET_NAME=$1
# WARNING: This bucket is intentionally set to public access as MinIO doesn't support ACLs
# Ensure only non-sensitive data is stored in this bucket
echo "Setting bucket $BUCKET_NAME as public..."
mc anonymous set public local/$BUCKET_NAME
}

# Retry MinIO Client alias setup
retry_command "mc alias set local $MINIO_HOST $MINIO_ACCESS_KEY $MINIO_SECRET_KEY"

# Create the necessary buckets
create_bucket_if_not_exists "patient-bucket"
create_bucket_if_not_exists "facility-bucket"

# Set only facility-bucket as public
set_bucket_public "facility-bucket"

# Graceful exit
echo "Bucket setup completed successfully."
exit 0
2 changes: 1 addition & 1 deletion docs/local-setup/configuration.rst
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ Using Docker Compose
- care (main repo)
- redis (in-memory cache)
- celery (task queue)
- localstack (to mimic AWS services locally)
- minio (to mimic AWS services locally)
sainak marked this conversation as resolved.
Show resolved Hide resolved

This is the most recommended way of setting up care locally,
as it installs appropriate dependencies in containers so there
Expand Down