Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Create docker container to build graph file #1319

Merged
merged 1 commit into from
Apr 5, 2022
Merged

Create docker container to build graph file #1319

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 7 additions & 6 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,13 @@ Development Installation
------------------------

1. Make sure you have the development dependencies installed
2. Place GTFS .zip files, OSM files, and elevation .tif files (optional) in the otp_data folder
3. Copy `deployment/ansible/group_vars/development_template` to `deployment/ansible/group_vars/development`
4. Change into the `src/` folder and run `npm install` to install the node modules on the host machine
5. Run `vagrant up`
6. See the app at http://localhost:8024! See OpenTripPlanner at http://localhost:9090.
7. Running `./scripts/serve-js-dev.sh` on the host will rebuild the front-end app on file change (the browser must be reloaded manually to pick up the change). Alternatively, `cd /opt/app/src && npm run gulp-development` can be run manually in the VM to pick up changes to the static files.
2. Place GTFS .zip files, OSM files, and elevation .tif files (optional) in the root of the otp_data folder
3. Generate a graph file with (takes approx 3 hours) `docker-compose run --rm otp otp --build /var/otp` in the deployment/graph directory.
4. Copy `deployment/ansible/group_vars/development_template` to `deployment/ansible/group_vars/development`
5. Change into the `src/` folder and run `npm install` to install the node modules on the host machine
6. Run `vagrant up`
7. See the app at http://localhost:8024! See OpenTripPlanner at http://localhost:9090.
8. Running `./scripts/serve-js-dev.sh` on the host will rebuild the front-end app on file change (the browser must be reloaded manually to pick up the change). Alternatively, `cd /opt/app/src && npm run gulp-development` can be run manually in the VM to pick up changes to the static files.

Note that if there is an existing build Graph.obj in `otp_data`, vagrant provisioning in development mode will not attempt to rebuild the graph, but will use the one already present.

Expand Down
11 changes: 11 additions & 0 deletions deployment/graph/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
FROM openjdk:8u322-jre-bullseye

ENV VERSION=1.4.0 \
JAVA_MX=15G

ADD https://repo1.maven.org/maven2/org/opentripplanner/otp/$VERSION/otp-$VERSION-shaded.jar /usr/local/share/java/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like this will cause Docker to re-download the OTP jar on each rebuild so that it can calculate a checksum and decide whether to use its cached filesystem layer or not. We've historically had struggles with reliability when downloading from Maven in places that get run a lot (CI), so in the past we've sometimes manually copied dependencies to our own S3 buckets.

But I think that would be overkill for something that most people will run once and never again. Without significant development activity on the horizon for this project I think this is fine as-is.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed that the Docker docs discourage the use of ADD for remote fetch in favor of something like RUN curl ... so that Docker caches based on the command string rather than the file contents, although I don't think their reasoning directly applies to this particular case because we're just copying a single file directly to a directory and not doing anything with it afterward.

RUN echo "0367b1a15bac5f587807a5b897a9734209f8135c /usr/local/share/java/otp-$VERSION-shaded.jar" | sha1sum --check
RUN ln -s otp-$VERSION-shaded.jar /usr/local/share/java/otp.jar

COPY otp /usr/local/bin/
RUN chmod 755 /usr/local/bin/*
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you add an ENTRYPOINT here then you'll be able to avoid specifying otp in the README command, and you might be able to remove the wrapper script entirely.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ENTRYPOINT didn't do what i had hoped in the Dockerfile. I tried adding it to docker-compose which did appear to work, but just let me get ride of otp - not the need for a separate file. Might be a docker-compose version thing - will investigate further if I pick up #1275

8 changes: 8 additions & 0 deletions deployment/graph/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
version: '3'
services:
otp:
build:
context: .
dockerfile: Dockerfile
volumes:
- ../ansible/roles/cac-tripplanner.otp-data/files/otp_data/:/var/otp/
3 changes: 3 additions & 0 deletions deployment/graph/otp
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/bin/sh

exec java -Xmx"$JAVA_MX" -jar /usr/local/share/java/otp.jar "$@"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed when building the graph that I received a warning that build-config.json and router-config.json were absent and that default configuration would be used. I searched the repo to see if we were using either of those and found a template for router-config.json which sets up the configuration for using bike-sharing stations. However, that seems like it's only a run-time configuration and isn't used when building the graph.

So I don't think we need to work to include the router-config.json during the graph build step here, but we should thoroughly test the bike-sharing functionality the first time we deploy using this graph building process, and we'll need to include that file if we ever work on #1275.