Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deploy CloudFront distributions in front of CopWatch #307

Draft
wants to merge 52 commits into
base: develop
Choose a base branch
from

Conversation

copelco
Copy link
Member

@copelco copelco commented Oct 21, 2024

Deploy CloudFront distribution for app server.

https://app.clickup.com/t/86897a79g

TODO

  • Deploy staging CDN
  • Remove django-cache-machine
    • Remove cache_response
    • Remove caching package (django-cache-machine), imports, and referenced code
  • Temporarily deploy this branch to staging (edit github workflow)
  • Update cache primer to (1) query actual site and (2) queue a task per agency to speed it up
  • Determine if we need intermediate Let's Encrypt certs for origin hostnames
  • Determine how to deploy CDN CloudFormation stacks alongside K8s CloudFormation stack
  • Determine how to handle https://nccopwatch.org/resources pages if updated via Admin
  • Deploy production CDN

Related PRs (need to be merged)

Co-authored-by: Ronard <[email protected]>
@copelco
Copy link
Member Author

copelco commented Oct 21, 2024

@copelco copelco changed the title deploy staging CDN Deploy CloudFront distributions in front of CopWatch Oct 21, 2024
@copelco
Copy link
Member Author

copelco commented Oct 21, 2024

Testing notes

Look for x-cache header when requesting pages through CloudFront:

curl --user "forwardjustice:trafficstops" --head https://staging.nccopwatch.org/api/agency/80/stops/                                                          
HTTP/2 200 
content-type: application/json
content-length: 2406
date: Mon, 21 Oct 2024 14:13:13 GMT
allow: GET, HEAD, OPTIONS
x-frame-options: DENY
content-language: en
x-content-type-options: nosniff
x-xss-protection: 1; mode=block
referrer-policy: same-origin
strict-transport-security: max-age=15724800; includeSubDomains
vary: Accept-Encoding
x-cache: Hit from cloudfront
via: 1.1 771410cef18c0261fb48d1f92756f9e2.cloudfront.net (CloudFront)
x-amz-cf-pop: ATL59-P1
x-amz-cf-id: tSl4WzkS7MEz4qRCtuJmX7OIqFq5pWSfwfdD0T1VMbARB13_7cCFcA==
age: 3300

An example query of the origin directly:

curl --user "forwardjustice:trafficstops" --insecure --header "Host: staging.nccopwatch.org" --head https://staging-origin.nccopwatch.org/api/agency/80/stops/
HTTP/2 200 
date: Mon, 21 Oct 2024 15:10:12 GMT
content-type: application/json
content-length: 2406
vary: Accept, Accept-Language
allow: GET, HEAD, OPTIONS
x-frame-options: DENY
content-language: en
x-content-type-options: nosniff
x-xss-protection: 1; mode=block
referrer-policy: same-origin
strict-transport-security: max-age=15724800; includeSubDomains

Remove --head to see the JSON response.

Clear cache

Create invalidation on staging:

aws cloudfront create-invalidation --distribution-id E2OFFI0H5HY2N8 --paths "/*"

@github-actions github-actions bot temporarily deployed to staging October 23, 2024 17:46 Destroyed
@github-actions github-actions bot temporarily deployed to staging October 23, 2024 18:16 Destroyed
@github-actions github-actions bot temporarily deployed to staging October 24, 2024 14:44 Destroyed
@ronardcaktus
Copy link
Member

Slow queries with time

15 seconds

> time curl --user "forwardjustice:trafficstops" --insecure --header "Host: staging.nccopwatch.org" --head https://staging-origin.nccopwatch.org/api/agency/-1/search-rate/
HTTP/2 200 
date: Thu, 24 Oct 2024 14:51:19 GMT
content-type: application/json
content-length: 1793
vary: Accept, Accept-Language
allow: GET, HEAD, OPTIONS
expires: Mon, 23 Dec 2024 14:51:19 GMT
cache-control: max-age=5184000
x-frame-options: DENY
content-language: en
x-content-type-options: nosniff
x-xss-protection: 1; mode=block
referrer-policy: same-origin
strict-transport-security: max-age=15724800; includeSubDomains

curl --user "forwardjustice:trafficstops" --insecure --header  --head   0.00s user 0.01s system 0% cpu 15.756 total

@github-actions github-actions bot temporarily deployed to staging October 24, 2024 15:05 Destroyed
@github-actions github-actions bot temporarily deployed to staging October 24, 2024 17:07 Destroyed
@github-actions github-actions bot temporarily deployed to staging October 27, 2024 17:45 Destroyed
@github-actions github-actions bot temporarily deployed to staging October 28, 2024 15:22 Destroyed
@github-actions github-actions bot temporarily deployed to staging October 28, 2024 15:47 Destroyed
@github-actions github-actions bot temporarily deployed to staging October 28, 2024 19:45 Destroyed
@github-actions github-actions bot temporarily deployed to staging October 28, 2024 20:22 Destroyed
@github-actions github-actions bot temporarily deployed to staging October 30, 2024 17:28 Destroyed
@github-actions github-actions bot temporarily deployed to staging November 1, 2024 13:16 Destroyed
@github-actions github-actions bot temporarily deployed to staging November 1, 2024 13:21 Destroyed
@github-actions github-actions bot temporarily deployed to staging November 1, 2024 13:35 Destroyed
@github-actions github-actions bot temporarily deployed to staging November 1, 2024 14:15 Destroyed
@github-actions github-actions bot temporarily deployed to staging November 2, 2024 21:27 Destroyed
@github-actions github-actions bot temporarily deployed to staging November 4, 2024 14:28 Destroyed
@github-actions github-actions bot temporarily deployed to staging November 6, 2024 16:32 Destroyed
@github-actions github-actions bot temporarily deployed to staging November 6, 2024 16:40 Destroyed
@github-actions github-actions bot temporarily deployed to staging November 6, 2024 16:51 Destroyed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants