Skip to content

Services

Roman Nikitin edited this page Jan 23, 2021 · 1 revision

Platform consists of several modules (also can be called microservices):

  • TCP flag receiver (eventlet tcp server on port 31337). For each connection send team token in the first line, then flags, each in a new line.

  • HTTP flag receiver accepts flags and supports Hackerdom's checksystem protocol. Flags are submitted in batches of no more than 100 flags in PUT requests with JSON-encoded array of flags in body. Team token must be provided in X-TeamToken request header.

  • Celerybeat sends round start events to celery.

  • Celery is running checkers distributably. Can be scaled using docker command: ./control.py scale_celery -i N to run N instances (assuming system is already started). One instance for ~60 team-tasks is recommended (so, if there're 80 teams and 4 tasks, run 5-6 instances of celery). control.py has "worker" subcommand, which starts only Celery to unload the main machine.

  • Flower is a beautiful celery monitoring app.

  • Redis acts as a fast passthrough cache for the most frequent operations (check if the flag exists, etc).

  • RabbitMQ acts as a broker for Celery and some inter-service communication (e.g. flag metrics are sent from receivers to monitoring service).

  • Postgres is a persistent game storage that provides a guaranteed data safety in case of failures.

  • client_api and admin_api provide data for Vue.js frontend. Admin API is also used for internal authentication.

  • events is a Socket.io server for all game events & live scoreboard reloads.

  • monitoring is a Prometheus metrics exporter (currently only various flag statistics are available at /api/metrics).

  • Nginx acts as a routing proxy that acts as a single external entry point for all services. It also provides load balancing for http services and rate limiting for flag receivers.

  • Initializer starts with the system, waits for the database to become available (all other containers wait for the initializer to finish its job) then drops old tables and initializes database. From that point, changing global configuration (game hardness, round time, etc) is useless, as it's already copied to the database. If changes are required, connect to the postgres container directly and run psql command (read the reference). For default database name and user (system_db and system_admin) use ./control.py rd exec postgres psql -U system_admin system_db. After changes in database you need to drop the cache. Easy way is to run FLUSHALL in redis, but that can lead to a round of unavailable scoreboard.

Please note that teams & tasks configuration can be updated during the live game on the admin panel, accessible at /admin.

It's not recommended to change the round time and start time after the system started, as it'll require at least to restart celerybeat service. This functionality is unintended and not tested.

Clone this wiki locally