From eef42ae5e1352812507e25d7ebad62ce1e995974 Mon Sep 17 00:00:00 2001 From: Chris Butler Date: Thu, 8 Aug 2024 10:49:49 +1000 Subject: [PATCH 1/5] chore(proposals): release modernization proposal skeleton Signed-off-by: Chris Butler --- proposals/release-modernization.md | 121 +++++++++++++++++++++++++++++ 1 file changed, 121 insertions(+) create mode 100644 proposals/release-modernization.md diff --git a/proposals/release-modernization.md b/proposals/release-modernization.md new file mode 100644 index 0000000..102fdd6 --- /dev/null +++ b/proposals/release-modernization.md @@ -0,0 +1,121 @@ +--- +title: Release process modernization and standardization for OSCAL compass +x-trestle-template-version: 0.0.1 +authors: + - butler54 +description: TBD +begin-design-discussions: 2024-08-08 #Insert +status: accepted #deferred, rejected, withdrawn or replaced - PR's are not accepted. Status is based on main. Rejected is unlikely to exist except where a clear record is required +--- + +**Checklist** + + + Add extra fields for management here + +- [x] Template passes validation with `trestle` +- [ ] Proposal cuts across multiple `oscal-compass` projects + + + +## Summary/Abstract + +Provide a high-level summary of the proposed change. Keep it short. + +This design process establishes a series of standard practices for planning and recording features or changes proposed for the project. The intent is to provide the project a consistent means to track decisions for historical reference and future plans while fully capturing how the feature or change will come to be. Additional benefits include making the process of building the release changelog easier and faster when the feature or change is ready to be released. It is a combination of a feature and effort-tracking document, a product requirements document, and a design document. + +## Background + +### Motivation and problem space + +Describe the need, problem, and motivation for the feature or change and any context required to understand the motivation. + +### Impact and desired outcome + +Describe any potential impact this feature or change would have. Readers should be able to understand why the feature or change is important. Briefly describe the desired outcome if the change or feature were implemented as designed. + +### Prior discussion and links + +Often these proposals start as an issue, forum post, email and it's helpful to link to those resources in this section to provide context and credit the right people for the idea. + +It is vital for projects to be able to track the chain of custody for a proposed enhancement from conception through implementation which can sometimes be difficult to do in a single Github issue, especially when it is a larger design decision or cuts across multiple areas of the project. + +The purpose of the design proposal processes is to reduce the amount of "siloed knowledge" in a community. By moving decisions from a smattering of mailing lists, video calls, slack messages, GitHub exchanges, and hallway conversations into a well tracked artifact, the process aims to enhance communication and discoverability. + +## User/User Story (Optional) + +Define who your the intended users of the feature or change are. Detail the things that your users will be able to do with the feature if it is implemented. Include as much detail as possible so that people can understand the "how" of the system. This can also be combined with the prior sections. + +## Goals + +List the desired goal or goals that the design is intended to achieve. These goals can be leveraged to structure and scope the design and discussions and may be reused as the "definition of done" - the criteria you use to know the implementation of the design has succeeded in accomplishing its goals. + +## Non-Goals + +Describe what is out of scope for the design proposal. Listing non-goals helps to focus discussion and make progress. + +## Proposal + +This is where we get down to the specifics of what the proposal actually is. It should have enough detail that reviewers can understand exactly what you're proposing, but should not include things like API designs or implementation. This section should expand on the desired outcome and include details on how to measure success. + +## Design Details + +This section should contain enough information to allow the following to occur: +* potential contributors understand how the feature or change should be implemented +* users or operators understand how the feature of change is expected to function and interact with other components of the project +* users or operators can take action to pre-plan any needed changes within their architecture that impacted by the upcoming feature or change if it's approved for implementation +* decisions or opinions on a specific approach are fully discussed and explained +* users, operators, and contributors can gain a comprehensive understanding of compatibility of the feature or change with past releases of the project. + +This may include API specs (though not always required), code snippets, data flow diagrams, sequence diagrams, etc. + +If there's any ambiguity about HOW your proposal will be implemented, this is the place to discuss them. This can also be combined with the proposal section above. It should also address how the solution is backward compatible and how to deal with these incompatibilities, possibly with defaulting or migrations. It may be useful to refer back to the goals and non-goals to assist in articulating the "why" behind your approach. + +## Impacts / Key Questions + +List crucial impacts and key questions, some of which may still be open. They likely require discussion and are required to understand the trade-offs of the design. During the lifecycle of a design proposal, discussion on design aspects can be moved into this section. After reading through this section, it should be possible to understand any potentially negative or controversial impact of the design. It should also be possible to derive the key design questions: X vs Y. + +This will also help people understand the caveats to the proposal, other important details that didn't come across above, and alternatives that could be considered. It can also be a good place to talk about core concepts and how they relate. It can be helpful to explicitly list the pros and cons of each decision. Later, this information can be reused to update project documentation, guides, and Frequently Asked Questions (FAQs). + +### Pros +Pros are defined as the benefits and positive aspects of the design as described. It should further reinforce how and why the design meets its goals and intended outcomes. This is a good place to check for any assumptions that have been made in the design. +### Cons +Cons are defined as the negative aspects or disadvantages of the design as described. This section has the potential to capture outstanding challenge areas or future improvements needed for the project and could be referenced in future PRs and issues. This is also a good place to check for any assumptions that have been made in the design. +## Risks and Mitigations + +Describe the risks of this proposal and how they can be mitigated. This should be broadly scoped and describe how it will impact the larger ecosystem and potentially adopters of the project; such as if adopters need to immediately update, or support a new port or protocol. It should include drawbacks to the proposed solution. + +### Security Considerations + +When attempting to identify security implications of the changes, consider the following questions: +* Does the change alter the permissions or access of users, services, components - this could be an improvement or downgrade or even just a different way of doing it? +* Does the change alter the flow of information, events, and logs stored, processed, or transmitted? +* Does the change increase the 'surface area' exposed - meaning, if an operator of the project or user were to go rogue or be uninformed in its operation, do they have more areas that could be manipulated unfavorably? +* What existing security features, controls, or boundaries would be affected by this change? + +This section can also be combined into the one above. + + +## Future Milestones (Optional) + +List things that the design will enable but that are out of scope for now. This can help understand the greater impact of a proposal without requiring to extend the scope of a proposal unnecessarily. + +## Implementation Details (Optional) + +Some projects may desire to track the implementation details in the design proposal. Some sections may include: + +### Testing Plan + +An overview on the approaches used to test the implementation. + +### Update/Rollback Compatibility + +How the design impacts update compatibility and how users can test rollout and rollback. + +### Scalability + +Describe how the design scales, especially how changes API calls, resource usage, or impacts SLI/SLOs. + +### Implementation Phases/History + +Describe the development and implementation phases planned to break up the work and/or record them here as they occur. Provide enough detail so readers may track the major milestones in the lifecycle of the design proposal and correlate them with issues, PRs, and releases occurring within the project. \ No newline at end of file From 29f395b2208df03892f1b40c87c55846bd15b6e1 Mon Sep 17 00:00:00 2001 From: Chris Butler Date: Thu, 8 Aug 2024 10:52:25 +1000 Subject: [PATCH 2/5] chore(proposals): Update title Signed-off-by: Chris Butler --- proposals/release-modernization.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/release-modernization.md b/proposals/release-modernization.md index 102fdd6..ecf057d 100644 --- a/proposals/release-modernization.md +++ b/proposals/release-modernization.md @@ -1,5 +1,5 @@ --- -title: Release process modernization and standardization for OSCAL compass +title: Modernisation and standardisation of release management across OSCAL compass x-trestle-template-version: 0.0.1 authors: - butler54 From 30b01d6a134811c44bb8025a8386dfb161afac24 Mon Sep 17 00:00:00 2001 From: Chris Butler Date: Fri, 30 Aug 2024 08:59:51 -0700 Subject: [PATCH 3/5] feat: Filling out background Signed-off-by: Chris Butler --- proposals/release-modernization.md | 76 +++++++++++++++++++++++------- 1 file changed, 60 insertions(+), 16 deletions(-) diff --git a/proposals/release-modernization.md b/proposals/release-modernization.md index ecf057d..0ee4202 100644 --- a/proposals/release-modernization.md +++ b/proposals/release-modernization.md @@ -14,49 +14,79 @@ status: accepted #deferred, rejected, withdrawn or replaced - PR's are not accep Add extra fields for management here - [x] Template passes validation with `trestle` -- [ ] Proposal cuts across multiple `oscal-compass` projects +- [x] Proposal cuts across multiple `oscal-compass` projects ## Summary/Abstract -Provide a high-level summary of the proposed change. Keep it short. +OSCAL compass has grown to a number of projects. Today all of these projects are software (not a service) which is being built and distributed via package registries. +CI & release infrastructure has been developed and coded individually over time matching the capabilities of emerging systems. The result has been duplication and variation between projects. In addition to this some aspects, such as the use of `setuptools` are aging. + +Here we are proposing to: +1. Rationalize on a modernized set of build configuration and management infrastructure, as much as is realistically possible. +2. Establish a set of common pipelines that are reused. Keeping the code as "DRY" as possible. +3. Continuing to use platforms that are freely available for open source projects. -This design process establishes a series of standard practices for planning and recording features or changes proposed for the project. The intent is to provide the project a consistent means to track decisions for historical reference and future plans while fully capturing how the feature or change will come to be. Additional benefits include making the process of building the release changelog easier and faster when the feature or change is ready to be released. It is a combination of a feature and effort-tracking document, a product requirements document, and a design document. ## Background ### Motivation and problem space -Describe the need, problem, and motivation for the feature or change and any context required to understand the motivation. +Today the CI and project release infrastructure is increasingly complex due to: + +1. An external forced march due to version changes in the OSCAL schema +2. Autogeneration tasks within the pipeline related to the schema. +3. Inter-project dependencies (e.g `compliance-trestle` is a dependency of [compliance-trestle-fedramp](https://github.com/oscal-compass/compliance-trestle-fedramp) and []`compliance-to-policy`](https://github.com/oscal-compass/compliance-to-policy). +4. General improvements required to match CNCF and broader industry trends. +5. Each project has built its' own pipelines. +6. Historically `compliance-trestle` restricted the toolset to python. However, this has led to choices who are not best in breed. +7. The pipelines are tightly coupled. The result is changes to the pipelines often induce unnecessary releases. + +In addition for this there are two areas where state of the art has evolved significantly: + +1. GitHub Actions: `compliance-trestle` was an early adopter of GitHub actions. The result is there are features which could streamline operations which have not been adopted. +2. Python package management: Python has had a number of options for package and release management. This has created multiple approaches and configuration files (`setup.py` vs `setup.cfg` vs `pyproject.toml` vs `requirements.txt` etc.) + + ### Impact and desired outcome -Describe any potential impact this feature or change would have. Readers should be able to understand why the feature or change is important. Briefly describe the desired outcome if the change or feature were implemented as designed. +Oscal compass codebase has common pipelines for CICD. The common pipelines + ### Prior discussion and links -Often these proposals start as an issue, forum post, email and it's helpful to link to those resources in this section to provide context and credit the right people for the idea. +- [Conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) +- [] +- [] +- [] -It is vital for projects to be able to track the chain of custody for a proposed enhancement from conception through implementation which can sometimes be difficult to do in a single Github issue, especially when it is a larger design decision or cuts across multiple areas of the project. +#### Current related issues -The purpose of the design proposal processes is to reduce the amount of "siloed knowledge" in a community. By moving decisions from a smattering of mailing lists, video calls, slack messages, GitHub exchanges, and hallway conversations into a well tracked artifact, the process aims to enhance communication and discoverability. +- []() +- [] -## User/User Story (Optional) -Define who your the intended users of the feature or change are. Detail the things that your users will be able to do with the feature if it is implemented. Include as much detail as possible so that people can understand the "how" of the system. This can also be combined with the prior sections. +## User/User Story (Optional) +- As a developer I want to minimize duplicate code in the cicd pipelines +- As a community I want to have consistentcy in CICD behaviour across different pipelines +- As a external contributor I want to be easily be able to run the GitHub actions pipelines +- As a developer I want a single place to configure the project tooling +- ## Goals List the desired goal or goals that the design is intended to achieve. These goals can be leveraged to structure and scope the design and discussions and may be reused as the "definition of done" - the criteria you use to know the implementation of the design has succeeded in accomplishing its goals. ## Non-Goals -Describe what is out of scope for the design proposal. Listing non-goals helps to focus discussion and make progress. +This proposal does not want to change the branching or releasing strategy where the teams are hitting a decent cadence. ## Proposal -This is where we get down to the specifics of what the proposal actually is. It should have enough detail that reviewers can understand exactly what you're proposing, but should not include things like API designs or implementation. This section should expand on the desired outcome and include details on how to measure success. + + ## Design Details @@ -78,15 +108,20 @@ List crucial impacts and key questions, some of which may still be open. They li This will also help people understand the caveats to the proposal, other important details that didn't come across above, and alternatives that could be considered. It can also be a good place to talk about core concepts and how they relate. It can be helpful to explicitly list the pros and cons of each decision. Later, this information can be reused to update project documentation, guides, and Frequently Asked Questions (FAQs). ### Pros + Pros are defined as the benefits and positive aspects of the design as described. It should further reinforce how and why the design meets its goals and intended outcomes. This is a good place to check for any assumptions that have been made in the design. + ### Cons Cons are defined as the negative aspects or disadvantages of the design as described. This section has the potential to capture outstanding challenge areas or future improvements needed for the project and could be referenced in future PRs and issues. This is also a good place to check for any assumptions that have been made in the design. + ## Risks and Mitigations Describe the risks of this proposal and how they can be mitigated. This should be broadly scoped and describe how it will impact the larger ecosystem and potentially adopters of the project; such as if adopters need to immediately update, or support a new port or protocol. It should include drawbacks to the proposed solution. ### Security Considerations + + When attempting to identify security implications of the changes, consider the following questions: * Does the change alter the permissions or access of users, services, components - this could be an improvement or downgrade or even just a different way of doing it? * Does the change alter the flow of information, events, and logs stored, processed, or transmitted? @@ -98,7 +133,9 @@ This section can also be combined into the one above. ## Future Milestones (Optional) -List things that the design will enable but that are out of scope for now. This can help understand the greater impact of a proposal without requiring to extend the scope of a proposal unnecessarily. +Currently `oscal compass` is mostly python with some go. +The focus of this proposal is to flush out details on the python ecosystem. +Parts will need to be extended for the go projects. ## Implementation Details (Optional) @@ -106,15 +143,22 @@ Some projects may desire to track the implementation details in the design propo ### Testing Plan -An overview on the approaches used to test the implementation. +Testing CICD is hard without having side effects. +In order to test this it is recommended that a fork is developed including **fully releasing** to a testing pypi project to ensure that, at least for `compliance trestle`, there are no broken releases. + ### Update/Rollback Compatibility -How the design impacts update compatibility and how users can test rollout and rollback. +There are three considerations: +1. GitHub actions workflow: This is effectively a hard cut-over for the maintaining team. Once the changes are introduced into develop the changes should be releases as soon as possible to avoid any issues relating to hot fixes. + +2. End users (installers): Technically there is no breaking change here. The buildtools which change are for develop time not install time. +3. Developers: Developers will need to update their environment. Rolling back will be difficult but not impossible. + ### Scalability -Describe how the design scales, especially how changes API calls, resource usage, or impacts SLI/SLOs. +There are no scalability concerns with this approach at this time. ### Implementation Phases/History From fddd2ed01751161ecaa46dd41ca23e9a04b7113c Mon Sep 17 00:00:00 2001 From: Chris Butler Date: Wed, 11 Sep 2024 11:05:19 +1000 Subject: [PATCH 4/5] chore: push for update Signed-off-by: Chris Butler --- proposals/release-modernization.md | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/proposals/release-modernization.md b/proposals/release-modernization.md index 0ee4202..9f3bdbe 100644 --- a/proposals/release-modernization.md +++ b/proposals/release-modernization.md @@ -46,7 +46,7 @@ Today the CI and project release infrastructure is increasingly complex due to: In addition for this there are two areas where state of the art has evolved significantly: 1. GitHub Actions: `compliance-trestle` was an early adopter of GitHub actions. The result is there are features which could streamline operations which have not been adopted. -2. Python package management: Python has had a number of options for package and release management. This has created multiple approaches and configuration files (`setup.py` vs `setup.cfg` vs `pyproject.toml` vs `requirements.txt` etc.) +2. Python package management: Python has had a number of options for package and release management. This has created multiple approaches and configuration files (`setup.py` vs `setup.cfg` vs `pyproject.toml` vs `yafp` etc.) @@ -86,9 +86,21 @@ This proposal does not want to change the branching or releasing strategy where ## Proposal +## Design Details -## Design Details +### Local lintning toolchain (`pre-commit`) + + + + + +### Python package management and configuration + + +### CICD + +#### Establish common build stages (across languages) This section should contain enough information to allow the following to occur: * potential contributors understand how the feature or change should be implemented From 76aa8d42e0bc5bb0172fce76028ba20e63442851 Mon Sep 17 00:00:00 2001 From: Chris Butler Date: Thu, 10 Oct 2024 16:05:03 +1100 Subject: [PATCH 5/5] chore: update Signed-off-by: Chris Butler --- proposals/release-modernization.md | 108 +++++++++++++++++++++++++++-- 1 file changed, 102 insertions(+), 6 deletions(-) diff --git a/proposals/release-modernization.md b/proposals/release-modernization.md index 9f3bdbe..a431fa6 100644 --- a/proposals/release-modernization.md +++ b/proposals/release-modernization.md @@ -52,8 +52,7 @@ In addition for this there are two areas where state of the art has evolved sign ### Impact and desired outcome -Oscal compass codebase has common pipelines for CICD. The common pipelines - +Oscal compass codebase has a consistent approach across all projects. Gates are consistent across project. ### Prior discussion and links @@ -64,9 +63,11 @@ Oscal compass codebase has common pipelines for CICD. The common pipelines #### Current related issues -- []() -- [] - +- [Merge bot for trestle](https://github.com/oscal-compass/compliance-trestle/issues/1716) +- [Python Sem ver issues](https://github.com/oscal-compass/compliance-trestle/issues/1590) +- [Container build for trestle](https://github.com/oscal-compass/compliance-trestle/issues/983) +- [Abstract github pipelines](https://github.com/oscal-compass/compliance-trestle/issues/1608) +- [Pipeline creation for informing release has been made](https://github.com/oscal-compass/compliance-trestle/issues/1503) ## User/User Story (Optional) @@ -74,7 +75,7 @@ Oscal compass codebase has common pipelines for CICD. The common pipelines - As a community I want to have consistentcy in CICD behaviour across different pipelines - As a external contributor I want to be easily be able to run the GitHub actions pipelines - As a developer I want a single place to configure the project tooling -- + ## Goals List the desired goal or goals that the design is intended to achieve. These goals can be leveraged to structure and scope the design and discussions and may be reused as the "definition of done" - the criteria you use to know the implementation of the design has succeeded in accomplishing its goals. @@ -85,6 +86,101 @@ This proposal does not want to change the branching or releasing strategy where ## Proposal +Exclusion + +![]() + + +### Release process + +### Exclusions + +1. https://github.com/oscal-compass/oscal-compass.github.io +2. https://github.com/oscal-compass/community + + + +### Conventional commits + +- All projects must use conventional commits + - A single conventional commit is associated with each + + +### Automated merges and releases + +- All projects, excluding the community project, use automated merges + - Projects with a version >= 1.0.0 require two reviews for merge to default branch and release gates. + - Projects with a version < 1.0.0 require one review, however, may require two at their own discretion + - Projects with a version < 0.1.0 are at the discretion of the user +- codeowners used for specific changes + +### Documentation websites + +- All documentation websites use `mkdocs-material` + - Plugins must be consistent +- +- Documentation websites must be versioned + - Latest documentation update occurs when merging to the default branch +- Release versions occur on a release tag + - Major and minor versions are saved in the dropdown. + - E.g. you can see 3.4 and 3.3 but 3.4.1 will overwrite the 3.4 release + +### SBoMs + + +### + + +### Language specific tools + + + +#### Python + +##### Standardize on `pyproject.toml` and `hatch` + +Python packaging and project definition has been inconsistent for many years. +The [python packaging authority](https://packaging.python.org/en/latest/flow/) has standardized on `pyproject.toml` +While `setup.py` is not deprecated, [certain elements of it's use are](https://packaging.python.org/en/latest/discussions/setup-py-deprecated/#is-setup-py-deprecated). +Today we are split, in compliance-trestle across `setup.py`, `setup.cfg`, `pyproject.toml` and some external files such as `.yapf-config` and `.pylintrc` +Rationalize on `pyproject.toml` to avoid split-brain scenarios. + +There are a number of tools for [python packaging and build backends](https://packaging.python.org/en/latest/key_projects/). Some are supported by the packing authority and some are not. + +[`hatch` and `hatchling`](https://hatch.pypa.io/latest/) appear [to be the most popular](https://github.com/pypa/hatch/graphs/commit-activity), modernized build tool supported by + + +#### Go + + +### Workflows + + + + +## CI Phases +1. Pull request review: + 1. Linting + 2. SAST + 3. + +2. Develop commit: + 1. Update documentation website with 'Latest' branch + + +Workflows required + +1. Package release +2. Website release +3. Quality gate +4. Tag release +5. + + + + + + ## Design Details