DoS Test Database

Table of Contents

• DoS Test Database
  • Table of Contents
  • Overview
  • Quick Start
    • Development Requirements
    • Local Environment Configuration
    • Local Project Setup

Overview

This project provides a way to build a Core DoS database from the test data.

Quick Start

Development Requirements

• macOS operating system provisioned with the curl -L bit.ly/make-devops-macos-setup | bash command
• iTerm2 command-line terminal and Visual Studio Code source code editor, which will be installed automatically for you in the next steps

Local Environment Configuration

Clone the repository

git clone [project-url]
cd ./[project-dir]

The following is equivalent to the curl -L bit.ly/make-devops-macos-setup | bash command. If that step has already been done it can be omitted at this point

make macos-setup

There are essential configuration options that must be set before proceeding any further. As a minimum the following command will ensure that tooling like docker and git are going to operate as expected, including local secret scanning and code formatting are enabled

make setup

Please, ask one of your colleagues for the AWS account numbers used by the project. The next command will prompt you to provide them. This information can be sourced from a properly set up project by running make show-configuration | grep ^AWS_ACCOUNT_ID_

make devops-setup-aws-accounts

Generate and trust a self-signed certificate that will be used locally to enable encryption in transit

make trust-certificate

Local Project Setup

Start up a local version of the Core DoS test database

tx-mfa # nonprod
make download
make build
make start log

Build a reusable database

make image-create

This produces dtdb/database Docker image that can be started up using the following command

docker run -it --name db-dos \
  --publish 5432:5432 \
  --detach \
  000000000000.dkr.ecr.eu-west-2.amazonaws.com/uec-tools/dtdb/database:latest
docker logs --follow db-dos

You may need to replace 000000000000 with your AWS account number.

Connect to the database

• Host: localhost
• Port: 5432
• Database name: pathwaysdos_dev
• Username: release_manager
• Password: postgres

Contributing

Here is the list of the development practices that have to be followed by the team and the individual members:

• Only use single canonical branch main. Any intermediate branch significantly increases the maintenance overhead of the repository.
• Apply the git rebase workflow and never merge from main to a task branch. Follow the squash-rebase-merge pattern to keep the history linear and clean.
• Cryptographically sign your commits using gpg to ensure its content have not been tampered with.
• Format the summary message of your pull request (merge request) using the following pattern "JIRA-XXX Summary of the change being made" for complines and clarity as well as to enable tooling to produce release notes automatically.
• Announce your PR/MR on the development Slack channel to allow any team member to review it and to share the knowledge. A change can be merged only if all comments have been addressed and it has been approved by at least one peer. Make good use of paring/mobbing/swarming practices for collaborative coding.

Before starting any work, please read CONTRIBUTING.md for more detailed instructions.

Development

• Describe how to
  • Connect to a local database
  • Interact with mock components
  • Switch each individual component to the dev mode
• Code formatting
• Code quality
• Reference the TODO.md file
• Provide guidance on how to use feature toggles and branching by abstraction

Testing

List all the type of test suites included and provide instructions how to execute them

• Unit
• Integration
• Contract
• End-to-end
• Performance
• Security
• Smoke

How to run test suite in the pipeline

Test data and mock services

• How the test data set is produced
• Are there any mock services in place

Manual check

Here are the steps to perform meaningful local system check:

• Log in to the system using a well known username role

Deployment

Artefact Versioning

E.g. semantic versioning vs. timestamp-based

CI/CD Pipelines

List all the pipelines and their purpose

• Development
• Test
• Cleanup
• Production (deployment)

Reference the jenkins/README.md file

Deployment From the Command-line

make deploy PROFILE=dev

Secrets

Where are the secrets located, i.e. AWS Secrets Manager, under the $(PROJECT_ID)-$(PROFILE)/deployment secret name and variable $(DEPLOYMENT_SECRETS) should be set accordingly.

AWS Access

To be able to interact with a remote environment, please make sure you have set up your AWS CLI credentials and MFA to the right AWS account using the following command

tx-mfa

Architecture

Diagrams

System Context Diagram

Include an image of the C4 model System Context diagram exported as a .png file from the draw.io application.

Container Diagram

Include an image of the C4 model Container diagram exported as a .png file from the draw.io application.

Component Diagram

Include an image of the C4 model Component diagram exported as a .png file from the draw.io application.

Processes and Data Flow

Include an image of the Processes and Data Flow diagram

Infrastructure

Include an image of the Infrastructure diagram. Please, be aware that any sensitive information that can be potentially misused either directly or indirectly must not be stored and accessible publicly. This could be IP addresses, domain names or detailed infrastructure information.

Networking

Include an image of the Networking diagram. Please, be aware that any sensitive information must not be stored and accessible publicly. This could be IP addresses, domain names or detailed networking information.

Integration

Interfaces

Document all the system external interfaces

• API documentation should be generated automatically

Dependencies

Document all the system external dependencies and integration points

Data

What sort of data system operates on and processes

• Data set
• Consistency and integrity
• Persistence

Authentication and authorisation

• Default user login for testing
• Different user roles
• Authosrisation type
• Authentication method

It is recommended that any other documentation related to the aspect of security should be stored in a private workspace.

Technology Stack

What are the technologies and programing languages used to implement the solution

Key Architectural Decisions

Link or include the abbreviated list of the ADRs

System Quality Attributes

• Accessibility, usability
• Resilience, durability, fault-tolerance
• Scalability, elasticity
• Consistency
• Performance
• Interoperability
• Security
• Supportability

Guiding Principles

List of the high level principles that a product /development team must adhere to:

• The solution has to be coded in the open - e.g. NHSD GitHub org
• Be based on the open standards, frameworks and libraries
• API-first design
• Test-first approach
• Apply the automate everything pattern
• AWS-based cloud solution deployable to the NHSD CPaaS Texas platform
• Use of the Make DevOps automation scripts (macOS and Linux)

Operation

Error Handling

• What is the system response under the erroneous conditions

Observability

• Logging
  • Indexes
  • Format
• Tracing
  • Correlation ID
• Monitoring
  • Dashboards
• Alerting
  • Triggers
  • Service status
• Fitness functions
  • What do we measure?

What are the links of the supporting systems?

Auditing

Are there any auditing requirements in accordance with the data retention policies?

Backups

• Frequency and type of the backups
• Instructions on how to recover the data

Cloud Environments

List all the environments and their relation to profiles

• Development
  • Profile: dev
  • URL address: https://?.k8s-dev.texasplatform.uk/
  • Username: [email protected]
  • Password: stored in the AWS Secrets Manager ?
• Test
• Demo
• Live

Describe how to provision and deploy to a task branch environment

Runbooks

List all the operational runbooks

Product

Communications

• Slack channels
  • Development, e.g. [service-name]-development
  • CI/CD and data pipelines, processes, e.g. [service-name]-automation
  • Service status, e.g. [service-name]-status
• Email addresses in use, e.g. [service.name]@nhs.net

All of the above can be service, product, application or even team specific.

Documentation

• Sprint board link
• Backlog link
• Roadmap link
• Risks register link
• Documentation workspace link