Merge pull request #2957 from guardian/mm/docs-2024-10-30

Documentation | Bring existing documentation up to date.

README.md

@@ -6,7 +6,7 @@ profile (dot) theguardian (dot) com
 
 Gateway is the frontend to sign-in and registration at the Guardian at [profile.theguardian.com](https://profile.theguardian.com).
 
-Need help? Contact the Identity team on [Digital/Identity](https://chat.google.com/room/AAAAFdv9gK8).
+Need help? Contact the Identity & Trust team on [P&E/Identity & Trust](https://chat.google.com/room/AAAAFdv9gK8).
 
 ## Architecture/Overview
 

docs/architecture.md

@@ -18,11 +18,13 @@ Many other client side applications at the Guardian now are React app based, whi
 
 Gateway is primarily a [TypeScript](https://www.typescriptlang.org/), [React](https://reactjs.org/), and [Express.js](https://expressjs.com/) application, utilising [the Guardian Source Design System](https://theguardian.design/) components, and [Emotion](https://emotion.sh) CSS-in-JS library for UI/design. We use [Jest](https://jestjs.io/) for unit testing, and [Cypress](https://www.cypress.io/) for integration tests and E2E tests.
 
+We also heavily integrate with [Okta Customer Identity Solution](https://okta.com), who is our Identity backend and provider, and their APIs. For documentation on how we interact with Okta, see the [Okta Documentation](./okta) folder, and specific documentation on the new(er) [Okta IDX API](./okta/idx/README.md).
+
 ## Browser Support
 
 **Our core line in the sand is that core functionality and features must NOT require any client-side JavaScript.** To this end Gateway is set up as a completely SSR (Server-side Rendered) React application.
 
-This has the added benefit that since we're only serving HTML, we can target a wider range of browsers compared to doing client side hydration, pretty much any browser thats supports [TLS 1.2](https://caniuse.com/#feat=tls1-2) will get functional support. As far as styling/css go, we target the recommended browsers from [guardian/dotcom-rendering](https://github.com/guardian/dotcom-rendering/blob/master/docs/principles/browser-support.md#recommended-browsers).
+This has the added benefit that since we're only serving HTML, we can target a wider range of browsers compared to doing client side hydration, pretty much any browser thats supports [TLS 1.2](https://caniuse.com/#feat=tls1-2) will get functional support. As far as styling/css go, we target the recommended browsers from [guardian/dotcom-rendering](https://github.com/guardian/dotcom-rendering/blob/main/dotcom-rendering/docs/principles/browser-support.md#browser-support-principles).
 
 However there will be some JavaScript code that needs to run client side, such as for analytics, consent etc. And hydration can be used to enhance functionality for the user.
 

docs/dependency-upgrades.md

@@ -13,7 +13,25 @@
 > Antoine de Saint-Exupéry, _The Little Prince_
 
 We should, if possible, update our NPM packages weekly, for they too will decay
-and reveal security vulnerabilities. Here is a general workflow for so doing:
+and reveal security vulnerabilities.
+
+## Automatic dependency upgrades
+
+Every week on Monday at 08:30 UTC, Dependabot will create a number of PRs for
+updating our dependencies.
+
+These will need to be reviewed and merged.
+
+In some cases dependabot will not be able to upgrade a package due to tests
+failing. In this case, you'll will have to manually review, and potentially
+upgrade the dependencies manually. The instructions for this are below.
+
+## Manual dependency upgrades
+
+We usually have to do manual dependency upgrades when performing major version
+upgrades, or when dependabot is unable to upgrade a package due to failing tests.
+
+Here is a general workflow for so doing:
 
 1. Dependabot will add some PRs for necessary dependency upgrades to our PRs
    list. This is your call to action, your batsignal!

docs/development.md

@@ -236,6 +236,16 @@ You can export multiple stories from each file, for example to show how the comp
 
 ## State Management
 
+Within Gateway state will take one of two forms.
+
+1. Managing some data/data between the server and client on a per request basis.
+   - See [Request State Locals and Client State](#request-state-locals-and-client-state) for more information.
+2. Managing some user data/data between requests.
+   - Use query parameters for simple data that needs to persist between requests.
+     - See [Query Params](#query-params) for more information.
+   - Use the encrypted state cookie for more complex data that needs to persist between requests.
+     - See [Encrypted State Cookie](#encrypted-state-cookie) for more information.
+
 ### Request State Locals and Client State
 
 Sometimes data is needed by the client to render a specific component, e.g. an error. Using SSR with additional client side hydration we
@@ -393,6 +403,10 @@ export const getPersistableQueryParams = (params: QueryParams): PersistableQuery
 
 This file also exposes an `addQueryParamsToPath` method which can be used to append query parameters to a given path/string with the correct divider (`?`|`&`). By default it filters out parameters that do not persist from the `QueryParams` object and then turns it into a query string. If you want to include an parameter that doesn't persist, you can manually opt into providing a value as the 3rd argument to the method.
 
+The `parseExpressQueryParams` method in [`src/server/lib/queryParams.ts`](../src/server/lib/queryParams.ts) will parse and validate the query parameters from a request. This will also need to be updated when adding new query parameters, to make sure we only allow the expected parameters are available to use.
+
+The query params for a given request will be available on the `RequestState` (`res.locals.queryParams`) on the [server](#server), and the `ClientState` on the [client](#client).
+
 #### Server
 
 You can access this server side on the `ResponseWithRequestState` object as `res.locals.queryParams`. For example you could get the `returnUrl` using:
@@ -467,24 +481,55 @@ const TestComponent = ({ queryString, clientId, error }: Props) => {
 
 ```
 
+### Encrypted State Cookie
+
+In some cases we need to preserve user data, or user state data between requests in order to be able to modify behaviour of a given request/page. This is done using the encrypted state cookie.
+
+The type is defined in [`EncryptedState`](../src/shared/model/EncryptedState.ts) interface. This is used to determine what data is stored in the cookie. The `EncryptedState` interface should only include properties that need to persist between requests, and should not include any properties that are only needed for a single request. The data should also be as small as possible, as the cookie has a maximum size limit.
+
+The data is encrypted and the cookie signed in order to prevent tampering with the data, and prevent it being readable by an actor. The cookie is also set to be HttpOnly, so it cannot be accessed by JavaScript, and Secure, so it can only be sent over HTTPS.
+
+To set/update the cookie, use the methods in [`src/server/lib/encryptedStateCookie.ts`](../src/server/lib/encryptedStateCookie.ts). The `setEncryptedStateCookie` method is used to set the cookie, and overwrite any existing cookie. The `updateEncryptedStateCookie` method is used to update the cookie, and merge the new data with the existing data. The `clearEncryptedStateCookie` method is used to clear the cookie. Use `readEncryptedStateCookie` to read the cookie and get the data.
+
+When using the cookie, make sure to remove data from the cookie when it is no longer needed, to avoid the cookie growing too large.
+
+Example of usage:
+
+```ts
+router.get('/some-route', (req: Request, res: Response) => {
+	const encryptedState = readEncryptedStateCookie(req);
+
+	// do something with the encrypted state
+	const email = encryptedState.email;
+
+	// update/set the encrypted state to remove the email, and add a new value
+	updateEncryptedStateCookie(res, {
+		email: undefined,
+		passcodeUsed: true,
+	});
+
+  ...
+});
+```
+
 ## Styling
 
 Styling is done in JS (or TSX in our case) using the [Emotion](https://emotion.sh) CSS-in-JS library, which allows for the definitions of styles at the component level, which means once rendered, the html sent to the client only contains the CSS required for that page.
 
-It's also used as [the Guardian Source Design System](https://theguardian.design/) components are built using Emotion too, allowing the use for those components in our project.
+It's also used as [the Guardian Source Design System](https://theguardian.design/) components are built using Emotion too, allowing the use for those components in our project, through the [`@guardian/source`](https://github.com/guardian/csnx/tree/main/libs/@guardian/source) and [`@guardian/source-development-kitchen`](https://github.com/guardian/csnx/tree/main/libs/%40guardian/source-development-kitchen) packages.
 
 Example of styling and adding it to a `p` tag using Emotion and Source:
 
 ```tsx
 import React from 'react';
 import { css } from '@emotion/react';
-import { textSans, neutral } from '@guardian/source/foundations';
+import { textSans15 } from '@guardian/source/foundations';
 
 // style the tag using the css string literal
 const p = css`
-	color: ${neutral[100]};
+	${textSans15};
+	color: var(--color-text);
 	margin: 0;
-	${textSans.small()};
 `;
 
 // example component with the css attribute to add the styling
@@ -495,6 +540,12 @@ Try to keep the styling as close to the component as possible to the component b
 
 Shared styles used by multiple components can be added to and imported from the [src/client/styles/Shared.ts](../src/client/styles/Shared.ts) file.
 
+Gateway also supports theming, specifically a light and dark mode, which is done through the [`@media (prefers-color-scheme: dark)`](https://developer.mozilla.org/en-US/docs/Web/CSS/@media/prefers-color-scheme) media query.
+
+The theme is set in the [src/client/styles/Theme.tsx](../src/client/styles/Theme.tsx), and uses CSS variables to define the colours for each theme. The theme is then applied using the [Global Styles](https://emotion.sh/docs/globals) from Emotion, to the [`MinimalLayout`](../src/client/layouts/MinimalLayout.tsx) component, that is used as the base layout for all pages.
+
+Therefore rather than defining colours directly, it is recommended to use the CSS variables defined in the theme, as shown in the example above.
+
 ## Environment Variables
 
 As mentioned in the setup guide, some environment variables are required to start the application. However this section focuses on adding or removing an environment variable.

docs/gateway-flows.md

@@ -1,16 +1,20 @@
 # Flow diagrams for common Gateway routes
 
-These flow diagrams are a WIP, mostly used to help visualise some complex functions in Gateway for the purposes of mocking Okta flows in testing.
+**_Note: These diagrams are outdated and describe behaviour for the older Okta Classic API. For more up to date flows using the newer Okta IDX API, see the [IDX Documentation](./okta/idx/README.md)_**
 
 ## Sign in
 
+_Note: See [IDX Sign In](./okta/idx/sign-in-idx.md) for the most up to date flow_
+
 ```mermaid
 flowchart TD
   A[GET /signin] --> B[POST /signin] --> oktaSignInController --> authenticateWithOkta --> getUserGroups --> performAuthorizationCodeFlow
 
 ```
 
-## Register
+## Register / Create Account
+
+_Note: See [IDX Create Account](./okta/idx/create-account-idx.md) for the most up to date flow for new users_
 
 ```mermaid
 flowchart TD
@@ -30,7 +34,7 @@ flowchart TD
   redirectToEmailSent[302 /register/email-sent]
 ```
 
-## Resend registration email
+### Resend registration email
 
 ```mermaid
 flowchart TD
@@ -41,6 +45,8 @@ flowchart TD
 
 ## Reset password
 
+_Note: See [IDX Reset Password](./okta/idx/reset-password-idx.md) for the most up to date flow_
+
 ```mermaid
 flowchart TD
   get[GET /reset-password] --> post[POST /reset-password] --> sendChangePasswordEmailController --> sendEmailInOkta --> getUser --> userFound{User found?} -- Yes --> userStatus{User status?}

docs/okta/login-page-interception.md

@@ -4,7 +4,7 @@ When performing the OAuth Authorization Code flow with Okta, while navigating to
 
 If they do, then the user will be redirected back to the client app with the `authorization_code` parameter, and the SDK will then exchange this for the OAuth tokens.
 
-If they don't then the user will be prompted to sign in/authenticate. However this is done by Okta showing their own hosted login page, which has a lack of customisation options, and it doesn't provide us full control over the user experience, or doesn't allow us to do specific things during the process.
+If they don't, or the `prompt=login` parameter is included in the request, then the user will be prompted to sign in/authenticate. However this is done by Okta showing their own hosted login page, which has a lack of customisation options, and it doesn't provide us full control over the user experience, or doesn't allow us to do specific things during the process.
 
 Examples of things we currently do that we can't do with the Okta hosted login page:
 
@@ -52,7 +52,7 @@ opt no existing session - interception happens here
   note over Browser: Load HTML, execute JS,<br>redirect to Gateway
   Browser->>Gateway: Request /signin?fromUri={fromUri}&clientId={clientId}&...
   Gateway->>Browser: Load /signin?fromUri={fromUri}&clientId={clientId}&...
-  note over Browser: User sign in with<br>email+password/social/set password<br>session set in browser<br>redirect to fromURI
+  note over Browser: User authenticates<br>i.e sign in, create account,<br>or reset password<br><br>Okta session set in browser<br><br>Perform redirect to fromURI
   Browser->>Okta: Request fromUri
 end
 Okta->>Browser: Redirect request to app with the `auth_code` parameter<br/>oauth redirect_uri