Skip to content

Commit

Permalink
Merge pull request #2057 from bensteinberg/update-20240801-01
Browse files Browse the repository at this point in the history
Dependency updates
  • Loading branch information
bensteinberg authored Aug 1, 2024
2 parents 40f8e72 + 8a3b09a commit c9b9678
Show file tree
Hide file tree
Showing 7 changed files with 446 additions and 259 deletions.
20 changes: 10 additions & 10 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,35 +19,35 @@ jobs:

### run tests ###

- name: docker-compose up
- name: docker compose up
run: |
# separate pull so downloads run in parallel, with
# --ignore-pull-failures for PRs with new images that haven't been pushed yet:
docker-compose -f docker-compose.yml pull --ignore-pull-failures || true
docker-compose -f docker-compose.yml up -d # use -f to suppress docker-compose.override.yml
docker compose -f docker-compose.yml pull --ignore-pull-failures || true
docker compose -f docker-compose.yml up -d # use -f to suppress docker-compose.override.yml
docker ps -a # show running containers
docker-compose logs # show logs
docker compose logs # show logs
- name: Collect static files
shell: 'script -q -e -c "bash --noprofile --norc -eo pipefail {0}"' # avoid docker-compose "the input device is not a TTY" -- see https://github.com/actions/runner/issues/241#issuecomment-842566950
run: docker-compose exec web ./manage.py collectstatic --noinput # collect static files
run: docker compose exec web ./manage.py collectstatic --noinput # collect static files

- name: Python tests
shell: 'script -q -e -c "bash --noprofile --norc -eo pipefail {0}"' # avoid docker-compose "the input device is not a TTY" -- see https://github.com/actions/runner/issues/241#issuecomment-842566950
run: |
set -x
docker-compose exec web pytest \
docker compose exec web pytest \
--cov --cov-config=setup.cfg --cov-report xml `# write coverage data to .coverage for upload by codecov` \
-vv -n auto --dist loadgroup
- name: Javascript tests
shell: 'script -q -e -c "bash --noprofile --norc -eo pipefail {0}"' # avoid docker-compose "the input device is not a TTY" -- see https://github.com/actions/runner/issues/241#issuecomment-842566950
run: |
set -x
docker-compose exec web bash -c "umask 0 && mkdir /app/web/.npmcache && npm config set cache /app/web/.npmcache && npm install"
docker-compose exec web npm run lint
docker-compose exec web npm run build
docker-compose exec web npm run test
docker compose exec web bash -c "umask 0 && mkdir /app/web/.npmcache && npm config set cache /app/web/.npmcache && npm install"
docker compose exec web npm run lint
docker compose exec web npm run build
docker compose exec web npm run test
### codecov ###
# https://github.com/codecov/codecov-action
Expand Down
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Add the following to `/etc/hosts`:

Start up the Docker containers in the background:

$ docker-compose up -d
$ docker compose up -d

The first time this runs it will build the Docker images, which
may take several minutes. (After the first time, it should only take
Expand All @@ -33,7 +33,7 @@ If the H2O team has provided you with a pg_dump file, seed the database with dat

Then log into the main Docker container:

$ docker-compose exec web bash
$ docker compose exec web bash

(Commands from here on out that start with `#` are being run in Docker.)

Expand Down Expand Up @@ -78,9 +78,9 @@ commit the new assets locally, unless you want to use them immediately.

When you are finished, spin down Docker containers by running:

$ docker-compose down
$ docker compose down

Your database will persist and will load automatically the next time you run `docker-compose up -d`.
Your database will persist and will load automatically the next time you run `docker compose up -d`.

Or, you can clean up everything Docker-related, so you can start fresh, as with a new installation:

Expand Down
2 changes: 1 addition & 1 deletion docker/clean.sh
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
#!/usr/bin/env bash

docker-compose down --remove-orphans --rmi all --volumes
docker compose down --remove-orphans --rmi all --volumes
6 changes: 3 additions & 3 deletions docker/dump.sh
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,6 @@ if [ ! "$FILE" ]; then
FILE=h2o_dev-$(date +"%Y%m%d%H%M%S").dump
fi
echo "Writing data to $FILE ..."
docker-compose exec db pg_dump -Fc postgres -h localhost -U postgres -w -f /tmp/data.dump
docker cp "$(docker-compose ps -q db)":/tmp/data.dump $FILE
docker-compose exec db rm -f /tmp/data.dump
docker compose exec db pg_dump -Fc postgres -h localhost -U postgres -w -f /tmp/data.dump
docker cp "$(docker compose ps -q db)":/tmp/data.dump $FILE
docker compose exec db rm -f /tmp/data.dump
6 changes: 3 additions & 3 deletions docker/init.sh
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,6 @@ if [[ $((OPTIND - $#)) -ne 1 ]]; then
fi

echo "Loading data from $FILE ..."
docker cp "$FILE" "$(docker-compose ps -q db)":/tmp/data.dump
docker-compose exec db pg_restore --username=postgres --verbose --no-owner -h localhost -d postgres /tmp/data.dump
docker-compose exec db rm -f /tmp/data.dump
docker cp "$FILE" "$(docker compose ps -q db)":/tmp/data.dump
docker compose exec db pg_restore --username=postgres --verbose --no-owner -h localhost -d postgres /tmp/data.dump
docker compose exec db rm -f /tmp/data.dump
12 changes: 6 additions & 6 deletions docker/pandoc-lambda/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

In development, every Casebook's export will be routed to a local container that runs the lambda function in app.py.

Run `docker-compose logs -f pandoc-lambda` to watch requests come in and review their metrics.
Run `docker compose logs -f pandoc-lambda` to watch requests come in and review their metrics.

### ...locally, against a function already deployed to AWS Lambda

Expand All @@ -30,17 +30,17 @@ When the lambda returns, you should see its log printed to the console and shoul
### ...after code changes

During local development, changes to `function/app.py` are synced to the running container and should be noticed immediately.
If not, you can run `docker-compose restart pandoc-lambda` to restart the container.
If not, you can run `docker compose restart pandoc-lambda` to restart the container.

To deploy your changes to production you'll need to bundle the final version of your code into the Docker image and push it to AWS.

Increment the image number in `docker-compose.yml` and re-run `docker-compose up -d`. That will produce a newly-tagged image that includes your code.
Increment the image number in `docker-compose.yml` and re-run `docker compose up -d`. That will produce a newly-tagged image that includes your code.

(We probably want to script this, adding it to our CI pipeline, similar to how CAP builds and pushes dev images to our registry.)

### ...with new python requirements, including `awslambdaric`

Add new packages or pin versions in `requirements.in`. Then run `docker-compose exec pandoc-lambda pip-compile --allow-unsafe --generate-hashes`.
Add new packages or pin versions in `requirements.in`. Then run `docker compose exec pandoc-lambda pip-compile --allow-unsafe --generate-hashes`.

Increment the image number in `docker-compose.yml` to produce a new image.

Expand All @@ -50,11 +50,11 @@ To update a single (unpinned) package such as `awslambdaric` do the same thing,

### ...with a new Lambda Runtime Interface Emulator

Change the cache-buster hash in `docker-compose.yml`, update the Docker image number, and re-run `docker-compose up -d`. This process could be further scripted, like Perma's Google Chrome update is.
Change the cache-buster hash in `docker-compose.yml`, update the Docker image number, and re-run `docker compose up -d`. This process could be further scripted, like Perma's Google Chrome update is.

### ...with a new version of pandoc

Change the target version number in `Dockerfile`, increment the image number in `docker-compose.yml`, and re-run `docker-compose up -d`.
Change the target version number in `Dockerfile`, increment the image number in `docker-compose.yml`, and re-run `docker compose up -d`.


## Deploying to AWS Lambda
Expand Down
Loading

0 comments on commit c9b9678

Please sign in to comment.