Skip to content

Sprint 62

Jump to bottom
drotheram edited this page Sep 1, 2022 · 15 revisions

Started July 11, 2022; Due by Sept 1, 2022 (Review on Sept 1, 2022)

Sprint Goals:

RiskProfiler

  • RiskProfiler QA/QC script (Drew)

    • Integrate Lighthouse check - Drew
    • Investigate WCAG Compliance - Drew
      • NRCan has an internal resource - Joost
      • Waiting on French pages
      • Run on English Pages - Venessa Greebe to follow up with internal NRCan resources
    • Link-checker in gh-actions - riskprofiler repo - Drew/Damon
      • Follow up on link-checker dead links
    • Deploy H7 Static Site Final User Engagement version - Anthony
    • Create list of known issues - Damon/Julie

  • Fix Sauid/Building Map/Chart Data discrepancy bug - Drew/Will - Done

  • User engagement

  • Move riskprofiler-cms code, issues, content etc. to riskprofiler (Anthony) > backlog Hard deadline before October release

  • Review remaining translations (Drew, Joost, Phil, Will) - in progress

  • Fully automate riskprofiler static website generation in Docker WordPress instance (Anthony)

ElasticSearch - Kibana - PostGIS

  • v1.4.3 updates (Will)

    • 900913 hexgrids (possibly a nice to have - consider push to v1.4.4) - In progress switching to EPSG:3857
    • Create v1.4.3 ES indices and dump to S3 (Drew) - Done

  • Document Required data sets for version/subversion release (Drew)

  • Test AWS CLI script to build opendrr-api docker-compose build and dump db to S3 bucket (Anthony/Drew) - Blocked technical bugs

  • Contract for pygeoapi versioning (Drew/Joost) > Sprint 63

  • Break up add_data.sh (#214)

    • Determine persistent storage of pre-PSRA/DSRA db dump (Anthony/Drew/Will)
      • pre-PSRA/DSRA version on S3
      • 2 Gb versioned release assets chunks
    • Consider splitting db dump by schema
    • break up into PSRA-DSRA dependencies and actual PSRA/DSRA processing
    • add config parameters for processing PSRA(true/false) and DSRA(named scenarios)

  • DSRA Dashboards in Kibana (Drew, Damon)

    • DFC troubleshoot WebGL error in Kibana maps (Drew/Arash)
    • DFC examples in code for direct data access via PyGeoAPI - wiki or python notebook

Data Dissemination

  • Publish FGP records (Joost) - waiting on Charles FGP blocking

    • Send revised data files and MXD's to FGP (Will, Phil)
      • Physical Exposure - To be released alongside social fabric
      • Social Fabric - Will sent on Feb 11
      • PSRA - waiting on changes to seismic risk index > sprint 62
      • DSRA 1KM, 5KM, 25KM - DG signoff pending
    • Review revised FGP data layers (Murray, Phil, Tiegan, Joost, Will)
      • Physical Exposure
      • Social Fabric - Waiting on release of supporting publication
      • PSRA - waiting on changes to seismic risk index > sprint 62
      • DSRA
    • Prep legend files (MXD's) for FGP data See issue
      • Physical Exposure
      • Social Fabric
      • PSRA - French language legends in progress > sprint 62
      • DSRA (Carol)
    • Update API endpoints to prod in FGP metadata - To be released in October - Joost

  • GPKG generation via GitHub Actions for release assets opendrr-api (Damon, Anthony) > Blocked

    • Blocked by diskspace limits on GitHub Actions - Stack need to be optimized for disk usage - Blocked
    • might be able to unblock with changes to add_data.sh - using db dump instead of building from scratch

  • Generate layer tiles (Joost, Damon, Anthony) - Done with SAUID, all but one CSD done, will work on hex grid layers next.

    • List of v1.4.3 vector tiles:

      • Document vector tile generation workflow with comprehensive list of source data (Damon/Anthony/Joost)
      • PSRA aggregations Sauid/CSD hexgrid (at all resolutions)
      • 900913 priority (4326 if time remaining)

    • Automate vector tiles for DSRA scenarios (Possible for 3857, Investigating if possible for 900913 & 4326 (Damon, Anthony)
      * document and report bug to GDAL

    • Automate vector tile backup on GitHub and serve the vector tiles on GitHub Pages (10GB limit) (Anthony)

    • Create 3857 Hexgrid geometries (Will)

OpenDRR Administrative

  • Security Documentation (Joost) - pending Director signoff

    • FGP CSV document

  • Use of compressed release assets to reduce Git LFS data storage and bandwidth usage (Anthony) > sprint 62

  • Document Process for adding new DSRA scenario in earthquake-scenarios Wiki (Damon, Drew, Will, Anthony)

Demos:

  • Simplified procedure to update RiskProfiler website from H7 (Anthony)
  • 3857 Bugs (Will)

Scrum Notes:

July 19

Damon

  • Vector tiles

Anthony

  • 900913

Will

  • 900913 boundaries repo
  • 900913 model factory

August 2

Damon

  • Vector tiles, investigating some issues

Anthony

  • 900913, testing boundaries.backup, github actions processes

Will

  • waiting on Anthony's edit for add_data.sh for boundaries, sql scripts ready to test on newest model_factory test

August 9

Anthony

  • New OpenDRR/boundaries-db repo for the sole purpose of storing the PostgreSQL database dump (7-Zip 2000 MiB multi-volumes split into 100 MiB chunks), and
  • Updated add_data.sh etc. to import the database from OpenDRR/boundaries-db
  • New private repo OpenDRR/earthquake-scenarios-2022-jrimando-google-drive
  • TODO: Put up initializations/ files
  • TODO: Generate static web pages (in beta) for RiskProfiler from H7's 2022-07-26 version

Will

  • tested new build opendrr-api/model-factory, 1st build failed @DSRA, 2nd build successful up to db build. Didn't test postgis to ES but no changes since 1.4.3
  • current build incorporates 900913 postgis to ES as default, and if we do want to add _4326 hexgrids, will need to add additional scripts to incorporate
  • TODO: review new db, make edits if nessessary
  • TODO: create db backup, and pre dsra/psra backup

August 11

Anthony

  • Put up initializations/ files on OpenDRR/earthquake-scenarios-2022-jrimando-google-drive
  • TODO: Generate static web pages (in beta) for RiskProfiler from H7's 2022-07-26 version

August 16

Anthony

  • Create GitHub workflows (named sync.yml) automate the pulling of updates of OpenDRR/h7-framework and OpenDRR/h7-riskprofiler repos from H7
  • TODO: Create GitHub workflow sync.yml for OpenDRR/riskprofiler-cms too (#8)
  • TODO: Generate static web pages (in beta) for RiskProfiler from H7's 2022-07-26 version

August 23

Damon Struggling to get Docker Volumes working with GeoServer Still Creating Sauid Tiles GitHub Pages in NHSL Repo (PR from May)

Anthony Simplifying process of updating RiskProfiler - almost all automatic Looking at updating python .env file

Drew Just back from leave - catching up on emails Looking into inconsistencies in RiskProfiler maps and plots

August 25

Anthony:

Damon: Some work on NHSL - Link update

Drew: Investigating discrepancy between ES Charts and Vector Tile values

August 30

Anthony: Good morning! As I am serving at a Funeral Mass, here is my scrum note for 2022-08-30: Fixed h7-riskprofiler repo syncing problem: sync.yml: "set -euxo pipefail" to unhide errors so that git cherry-pick, when piped to tee, won't fail silently again https://github.com/OpenDRR/h7-riskprofiler/commit/072fcb625b61e44936c567d495cd5e4e2f4cebbb Created a pull request for Phil Evans: "Add French translation for Google basemap attribution" https://github.com/phil-evans/riskprofiler/pull/1/files Tagged v1.0-API_v1.4.0-20220825 release for RiskProfiler for fixing basemap language. (Also, got distracted and worked on .deb and snap packages for Hugo v0.102.1 release 😅)

Damon: More work on GH actions to generate tiles - added input More research on Docker Volumes - blocked on trying to get geoserver to work with it

Will: Back from leave Working with Anthony to get 900913 hexgrid Started building latest stack PostGIS loads hexgrids into 900914 projection with ogr2ogr

Drew: Tracked down data mismatch issue. ES was converting float to int field type but still displaying values as float Loaded new v1.4.3 indices, updated pygeoapi template and updated pygeoapi config Waiting for H7 to update endpoint versions (preferably versionless) Addressing additional issues in RiskProfiler repo prior to launch (Translations, Disclaimer etc.)

Sprint Planning | Sprint Review
Wikis: data | model-factory | opendrr-api | opendrr | python-env | riskprofiler-cms

Clone this wiki locally