Skip to content

Commit

Permalink
fix: update jwt and remove some deprecations
Browse files Browse the repository at this point in the history
  • Loading branch information
Chris committed Aug 13, 2023
1 parent 8b652dd commit b48c006
Show file tree
Hide file tree
Showing 10 changed files with 130 additions and 73 deletions.
15 changes: 6 additions & 9 deletions .github/workflows/deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@ name: Continuous Integration and Deployment
on:
pull_request:
push:
branches:
- master
- beta
branches:
- master
- beta

jobs:
Integration_And_Deployment:
Expand All @@ -24,26 +24,23 @@ jobs:
- name: Install Top Level Dependencies
run: npm ci

- name: Install Dependencies
run: npm run bootstrap

- name: Lint packages
run: npm run lint

- name: Build Packages
run: npm run build
run: npm run build

- name: Test Packages
run: npm run test

- name: Deploy Prod
if: github.ref == 'refs/heads/master'
env:
CF_API_TOKEN: ${{ secrets.CF_API_TOKEN }}
CLOUDFLARE_API_TOKEN: ${{ secrets.CF_API_TOKEN }}
run: npm run deploy:prod

- name: Deploy Beta
if: github.ref == 'refs/heads/beta'
env:
CF_API_TOKEN: ${{ secrets.CF_API_TOKEN }}
CLOUDFLARE_API_TOKEN: ${{ secrets.CF_API_TOKEN }}
run: npm run deploy:beta
24 changes: 14 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,35 +1,39 @@
# Polyratings Monorepo

Lerna monorepo to contain all polyratings related code

## Packages
* [frontend](./packages/frontend/) - Frontend react app deployed at https://polyratings.dev
* [backend](./packages/backend/) - Backend typescript app deployed on Cloudflare Workers
* [cron](./packages/cron/) - nightly job that syncs data between environments as well as backs up the professor list to a separate git [repo](TODO://PROVIDE_URL)
* [eslint-config](./packages/eslint-config/) - shared eslint config that is enforced in all other packages

- [frontend](./packages/frontend/) - Frontend react app deployed at <https://polyratings.dev>
- [backend](./packages/backend/) - Backend typescript app deployed on Cloudflare Workers
- [cron](./packages/cron/) - nightly job that syncs data between environments as well as backs up the professor list to a separate git [repo](TODO://PROVIDE_URL)
- [eslint-config](./packages/eslint-config/) - shared eslint config that is enforced in all other packages

## Getting Ready for development
If you are not interested in developing the backend or cron packages you can skip to [setup](#Setup)

If you are not interested in developing the backend or cron packages you can skip to [setup](#setup)

In order to set up for development you have one of two options:

1. Following the instructions [here](./docs/deployment.md) to deploy polyratings to your personal cloudflare account. This will then allow you to do test deployments of your changes in an isolated environment.
2. Reaching out to `[email protected]` to receive credentials to the cloudflare account in order to be able to publish to the official dev environment. This option should only be taken if you are interested in working on polyratings in the long term and have demonstrated an interest through multiple previous code contributions.

## Setup

Since this repository is organized using lerna setup is a little different than standard js projects.

Install top level JS dependencies

```bash
npm install
```
This will install lerna locally to this repo. Then run:
```bash
npm run bootstrap
```
This will install all of the dependencies for all of the sub packages and sym link dependent packages. Finally run:

This will install lerna and all of the dependencies for all of the sub packages and sym link dependent packages. Finally run:

```bash
npm run build
```

This will build all of the projects and put shared files where they are supposed to be.

You can now start developing in your desired package. Follow the README in the specific package for specific information.
77 changes: 53 additions & 24 deletions docs/deployment.md
Original file line number Diff line number Diff line change
@@ -1,94 +1,123 @@
**Note: This document is prepared under the assumption that you are storing your source code in GitHub and using Cloudflare as your cloud provider. YMMV and under any other configurations we cannot guarantee knowledge or support.**

# Preliminary Setup

## Setting Up Source Code

First you'll need somewhere to store the Polyratings source. You can either fork the [existing GitHub Repository](https://github.com/Polyratings/polyratings) or copy into your own fresh repo.

## Create Your Cloudflare Account

You can follow the the [steps here](https://support.cloudflare.com/hc/en-us/articles/201720164-Creating-a-Cloudflare-account-and-adding-a-website) to setup your Cloudflare account and either register a new domain new or transfer an existing one to Cloudflare. For all intents and purposes it **is not** necessary for you to utilize anything more than the Cloudflare [Free Tier](https://www.cloudflare.com/plans/free/) unless you wish to utilize some of the analytics or access control tools/features.
## Acquiring Access to Perspective API

## Acquiring Access to Perspective API

Perspective API is used to perform sentiment analysis on new ratings as students enter them, and is an integral part to the operation of the site. You can follow the steps [here](https://developers.perspectiveapi.com/s/docs-get-started) to setup access. We will not be covering how to setup a Google Cloud Platform account or its associated project, but you can find that information [here.](https://console.cloud.google.com/freetrial?_ga=2.23528445.670071674.1647121117-896238666.1632636274)

# Cloudflare Workers

Setting up the different Cloudflare workers requires some manual setup (both through the Cloudflare dashboard and in the source code) and then can be finished using the available build scripts.

## Cloudflare Dashboard

### KV

[Cloudflare Workers KV](https://developers.cloudflare.com/workers/runtime-apis/kv/) is a Key-Value store that is accessible to the Cloudflare workers during runtime. Unfortunately, these stores must be setup **before** deploying the workers and (to the best of my knowledge) cannot be done programmatically.

To setup your KVs, navigate to the "Workers" tab from the Cloudflare Dashboard, and the select the "KV" subtab. In the center-right section of the page, you'll see a "Create namespace" button, become familiar with it because you'll need to create at least 4 separate KV namespaces (or more as each deployment environment--prod/beta/dev--requires 4 unique namespaces to operate).
**Namespaces to create:**
- POLYRATINGS_TEACHERS_`ENV`
- POLYRATINGS_RATING_QUEUE_`ENV`
- POLYRATINGS_PROFESSOR_QUEUE_`ENV`
- POLYRATINGS_USERS_`ENV`
- POLYRATINGS_REPORTS_`ENV`
Take note of the namespace IDs as you'll need them for later configuration.

- POLYRATINGS*TEACHERS*`ENV`
- POLYRATINGS*RATING_QUEUE*`ENV`
- POLYRATINGS*PROFESSOR_QUEUE*`ENV`
- POLYRATINGS*USERS*`ENV`
- POLYRATINGS*REPORTS*`ENV`
Take note of the namespace IDs as you'll need them for later configuration.

### Workers Usage Model
Navigate back to the "Overview" subtab and on the right side of the page, you'll see an option called "Default Usage Model" and it's probably set to "Unbound", click "Change" and set it to "Bundled."

Navigate back to the "Overview" subtab and on the right side of the page, you'll see an option called "Default Usage Model" and it's probably set to "Unbound", click "Change" and set it to "Bundled."

This is mostly done because under our own testing, there are multiple times where deploying wrangler environments will not actually set the "bounded" model as specified in the configuration, and it's a pain to go back and manually set that for each worker using the Cloudflare Dashboard.

### URLS and Endpoints

You'll need to figure out what URLS you want to listen on for your backend. Our current convention is to use `api-ENVIRONMENT.DOMAIN.COM` so for the actual Polyratings site it's `api-prod.polyratings.org`.

Once you've determined the endpoints you want, you'll need to setup some DNS records that allow Cloudflare (and Workers) to redirect requests to the proper handler. To do this, navigate to the "Websites" tab, click on the domain that will be hosting your deployment, select "DNS," and click the "Add record" button to and an "AAAA" type record where "name" is the URL path before `DOMAIN.COM` (i.e. `api-prod`) then the IPv6 address should be "100::" (yes, this technically means to discard the request, but we use this to enable Cloudflare proxying on the URL path, which then triggers the routing rules we will later configure).

## Cloudflare Wrangler

The next thing you'll need to do is install [Cloudflare Wrangler](https://developers.cloudflare.com/workers/cli-wrangler/install-update/) and [authenticate](https://developers.cloudflare.com/workers/cli-wrangler/authentication/) yourself. This allows you to perform deployments from the CLI (and make use of the build scripts we have).

## Configuring the Workers

Each of the Wrangler project's ([Backend Worker](#backend-worker) or [Cron Worker](#cron-worker)) are configured using a `wrangler.toml` file that declares the different environment configurations, deployment types, etc. There are values that are specific to your Cloudflare account and **must** be changed!

If you want a better understanding of the syntax and structure of the configuration file, you can find that information [here.](https://developers.cloudflare.com/workers/cli-wrangler/configuration/) The only configurations that are unique for each person/domain/etc. are the following:

**Top Level Declarations:**

- `zone_id` - The Cloudflare Zone that corresponds to the domain of the website (can be found on the right side of that domain's "Overview" Page)
- `account_id` - Your Cloudflare Account
- `account_id` - Your Cloudflare Account

**Environment-Specific Declarations:**

- `route` - The URL that specific environment will listen on. These are the URLS you setup in [URLS and Endpoints](#urls-and-endpoints).

### Backend Worker

In the Backend package, you'll need to modify the `kv_namespaces` field under each environment, changing the ID of each namespace to match the titles and environments of the namespaces you created in [KV](#kv).

Additionally, you're going to need to configure some [Secrets using Wrangler.](https://developers.cloudflare.com/workers/platform/environment-variables/#adding-secrets-via-wrangler) Specifically, we need to create the following secrets (they can either be the same per environment, or different so long as they are valid):

- `PERSPECTIVE_API_KEY` - API key received when following [Preliminary Setup](#preliminary-setup)
- `JWT_SIGNING_KEY` - @mfish33 (if you want to explain how to gen it)

#### Deploying/Publishing
After performing the above steps, you should be able to deploy the worker using wrangler:
`wrangler publish --env ENVIRONMENT_TO_DEPLOY`

After performing the above steps, you should be able to deploy the worker using wrangler:
`wrangler deploy --env ENVIRONMENT_TO_DEPLOY`

### Cron Worker

Note: You **only** need to deploy this worker if you have multiple backend environments (prod, beta, dev, etc.) and want to synchronize them daily. Otherwise you can ignore this step.

The Cron worker requires the following secrets to function:

- `POLYRATINGS_CI_USERNAME` - The username of a user in the `POLYRATINGS_USERS` KV.
- `POLYRATINGS_CI_PASSWORD` - The plaintext password of the user above.
- `CF_API_TOKEN` - A Cloudflare API token that has permission to access the backend workers. You can follow the steps [here](https://developers.cloudflare.com/api/tokens/create/) to generate that token.
- `CLOUDFLARE_API_TOKEN` - A Cloudflare API token that has permission to access the backend workers. You can follow the steps [here](https://developers.cloudflare.com/api/tokens/create/) to generate that token.

#### Deploying/Publishing
After performing the above steps, you should be able to deploy the worker using wrangler:
`wrangler publish --env ENVIRONMENT_TO_DEPLOY`
#### Executing Deployment

After performing the above steps, you should be able to deploy the worker using wrangler:
`wrangler deploy --env ENVIRONMENT_TO_DEPLOY`

## Populating the Database

We currently provide a [daily dump](https://github.com/Polyratings/polyratings-data) of the entire Polyratings professor dataset. While we don't currently have a utility that can automatically populate an arbitrary KV with that dataset, you can use Wrangler [bulk put](https://developers.cloudflare.com/workers/cli-wrangler/commands/#put-2) functionality and upload a JSON file containing a slightly modified version of those dumps to populate your KV.

# Cloudflare Pages

As far as I am aware, there is no easy programmatic/automatic way to create a Cloudflare Pages project, so you'll need to follow the steps below to automatically build and deploy the site from source.

### Configuration (When Modifying the Backend Package)
Before we can begin deploying the frontend, we'll need to modify the URLs that it points to. From the root of the Frontend package in `src/App.config.ts`, you'll need to modify the `remoteUrl` of the various `AppConfigurations` to match the base routes you created in [URLS and Endpoints](#urls-and-endpoints): `https://api-ENV.DOMAIN.COM`
## Configuration (When Modifying the Backend Package)

Before we can begin deploying the frontend, we'll need to modify the URLs that it points to. From the root of the Frontend package in `src/App.config.ts`, you'll need to modify the `remoteUrl` of the various `AppConfigurations` to match the base routes you created in [URLS and Endpoints](#urls-and-endpoints): `https://api-ENV.DOMAIN.COM`

## Deploying

Navigate to the "Pages" tab from the dashboard to begin and select "Create a Project." You can select "Connect GitHub" and follow the setup process to link your new repository with Cloudflare. You'll be redirected back to Cloudflare and confirm the repository you'll be using for Cloudflare Pages by selecting "Begin setup".

### Builds and Deployments Page
You'll need to fill out the page using the following settings:
* **Project name:** The name you want associated with this Pages project. Note: it **is not** the final URL that will point to your homepage.
* **Production branch**: This should usually be `master` unless you've changed the default branch of your repository.
* **Framework preset**: None
* **Build command**: `npm run bootstrap && npm run build`. This is because the monorepo is orchestrated by [Lerna](https://lerna.js.org/) so all of the installation, symlinking, and building is handled at the top level.
* **Build output directory**: `/packages/frontend/dist`

Then select "Save and Deploy." Your site should automatically deploy to `project-name.pages.dev` though if your project name is identical to another, there will be a unique identifier appended to the end of `project-name`.
You'll need to fill out the page using the following settings:

- **Project name:** The name you want associated with this Pages project. Note: it **is not** the final URL that will point to your homepage.
- **Production branch**: This should usually be `master` unless you've changed the default branch of your repository.
- **Framework preset**: None
- **Build command**: `npm install && npm run build`. This is because the monorepo is orchestrated by [Lerna](https://lerna.js.org/) so all of the installation, symlinking, and building is handled at the top level.
- **Build output directory**: `/packages/frontend/dist`

Then select "Save and Deploy." Your site should automatically deploy to `project-name.pages.dev` though if your project name is identical to another, there will be a unique identifier appended to the end of `project-name`.
12 changes: 8 additions & 4 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

15 changes: 11 additions & 4 deletions packages/backend/README.md
Original file line number Diff line number Diff line change
@@ -1,37 +1,44 @@
# Polyratings Backend

This package contains the source code required to run the backend for the Polyratings service.

## Cloudflare Workers

This backend was written to be deployed on the [Cloudflare Workers Service](https://workers.cloudflare.com/).
As such, the available resources and globals will be restricted to those provided by the
[Cloudflare Workers Runtime API](https://developers.cloudflare.com/workers/runtime-apis).

### Workers KV

[WorkersKV](https://developers.cloudflare.com/workers/runtime-apis/kv) is a provided runtime api that provides
access to a durable key-value storage that is backed by the Cloudflare Cache and CDN. This API is the basis for
the site's data-storage solution, primarily for its low-cost and speed.

## Architecture

The backend begins with a single entrypoint in [index.ts](src/index.ts) that sets up the trpc router and surrounding logic

* routers - provide handlers for each module professors, ratings, admin and auth
- routers - provide handlers for each module professors, ratings, admin and auth

* dao (Data Access Object) - provide an abstraction layer over the data and allow request handlers to have a nice api surface. This includes all interactions with kv. For example adding/removing professors or reviews
- dao (Data Access Object) - provide an abstraction layer over the data and allow request handlers to have a nice api surface. This includes all interactions with kv. For example adding/removing professors or reviews

* generated - contains generated config files that are consumed by the cron job. These files are just a typescript file that represents the wrangler.toml file. These files are generated whenever the build command is called
- generated - contains generated config files that are consumed by the cron job. These files are just a typescript file that represents the wrangler.toml file. These files are generated whenever the build command is called

* types - contains kv schema along with supporting type helpers
- types - contains kv schema along with supporting type helpers

## Building

To build this package, you simply need to run `npm run build` and [esbuild](https://esbuild.github.io/) will
bundle the entire project and its dependencies together. The build artifact from this process can be manually uploaded
to Cloudflare Workers.

## Testing

Automated testing is currently a work-in-progress as we are working on this project. It is our full intention to completely
automate the building, testing, and deployment of this project within some type of CI/CD orchestration.

## Deployment

This package makes use of [Wrangler](https://developers.cloudflare.com/workers/cli-wrangler) a Command-line Interface
provided by Cloudflare which can automatically build and deploy your workers project to the Cloudflare network.

Expand Down
Loading

0 comments on commit b48c006

Please sign in to comment.