Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(docs): Add docs about architecture and fix structure #1229

Merged
merged 21 commits into from
Jan 24, 2025

Conversation

kamilprz
Copy link
Contributor

Description

The main goal of this PR was to document the Retina architecture, namely the data plane and the available control planes. While working on this, I also made some smaller changes to improve our consistency.

What this PR does:

Smaller changes

  • Change the favicon image being used to one with a better aspect ratio. See below for screenshots of before/after.
    • The current logo looks stretched in browser tabs and bookmarks due to the aspect ratio and not enough negative space.
  • Restructure the way our headings and pages are organised. (This meant a number of files had to be renamed/moved, whose content was not touched.)
    • For example currently if you click on "Captures" you navigate to a landing page/readme, but if you click on "Installation" it opens up a drop down of its pages. This is inconsistent. This PR makes all of the headings open up by default. See screenshots below for before/after.
  • Added subheadings to contributing page to make it easier to read.

Architecture docs

  • A new "Architecture" page is added to the newly introduced "Introduction" heading. This discusses the Data Plane and the available Control Planes.
    • As part of this a number of diagrams are introduced. Both the .png and the source for the diagram .excalidraw are included so that the diagrams can be versioned.
  • Update the "What is Retina?" page to be consistent with the landing page of retina.sh in terms of the feature descriptions. Also added a "What is Hubble?" subsection with a brief description.

Related Issue

#1055

There is also another PR opened for Hubble installation - #1223

Checklist

  • I have read the contributing documentation.
  • I signed and signed-off the commits (git commit -S -s ...). See this documentation on signing commits.
  • I have correctly attributed the author(s) of the code.
  • I have tested the changes locally.
  • I have followed the project's style guidelines.
  • I have updated the documentation, if necessary.
  • I have added tests, if applicable.

Screenshots (if applicable) or Testing Completed

I think its a bit redundant to put the Architecture Diagrams in here, as the files themselves are included. So check those out directly. Or open up a preview of the markdown page itself in GitHub.

Retina favicon image
image

Example of new file structure - showcasing that the heading now opens rather than having a landing page when clicked on.
image

Parity of feature highlight between retina.sh and "What is Retina?" page.
image


Please refer to the CONTRIBUTING.md file for more information on how to contribute to this project.

@kamilprz kamilprz requested a review from a team as a code owner January 16, 2025 13:04
Signed-off-by: Kamil <[email protected]>
@kamilprz
Copy link
Contributor Author

kamilprz commented Jan 16, 2025

Able to run make docs-prod and serve the page from /site.

Build

~/src/retina git:architecture-docs
> make docs-prod
docker run -i -p 3000:3000 -v /home/kamilp/src/retina:/retina -w /retina/ node:20-alpine npm install --prefix site && npm run build --prefix site

up to date, audited 1339 packages in 2s

347 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities
npm notice
npm notice New major version of npm available! 10.8.2 -> 11.0.0
npm notice Changelog: https://github.com/npm/cli/releases/tag/v11.0.0
npm notice To update run: npm install -g [email protected]
npm notice

> [email protected] build
> docusaurus build


   ------------------------------------------------------------------------------------------------------------------------------------------------------

                                                               Update available 3.6.1 → 3.7.0
                                                                                 .....

   ------------------------------------------------------------------------------------------------------------------------------------------------------

[INFO] [en] Creating an optimized production build...


✔ Client
  Compiled successfully in 20.53s

✔ Server


<w> [webpack.cache.PackFileCacheStrategy] Caching failed for pack: Error: EACCES: permission denied, mkdir '/home/kamilp/src/retina/site/node_modules/.cache/webpack/server-production-en'

● Client █████████████████████████ cache (99%) shutdown IdleFileCachePlugin
 serialize pack

✔ Server


<w> [webpack.cache.PackFileCacheStrategy] Caching failed for pack: Error: EACCES: permission denied, mkdir '/home/kamilp/src/retina/site/node_modules/.cache/webpack/client-production-en'
docusaurus-lunr-search:: Building search docs and lunr index file
docusaurus-lunr-search:: Start scanning documents in 2 threads
docusaurus-lunr-search:: Indexing time: 624.868ms
docusaurus-lunr-search:: indexed 38 documents out of 38
docusaurus-lunr-search:: writing search-doc.json
docusaurus-lunr-search:: writing search-doc-1737036699828.json
docusaurus-lunr-search:: writing lunr-index.json
docusaurus-lunr-search:: writing lunr-index-1737036699828.json
docusaurus-lunr-search:: End of process
[SUCCESS] Generated static files in "build".
[INFO] Use `npm run serve` command to test your build locally.
                                                                                                                                                     30.959s

Serve

~/src/retina git:architecture-docs
> cd site
~/src/retina/site git:architecture-docs
> npm run serve

> [email protected] serve
> docusaurus serve


   ------------------------------------------------------------------------------------------------------------------------------------------------------

                                                               Update available 3.6.1 → 3.7.0
                                                                                 .....

   ------------------------------------------------------------------------------------------------------------------------------------------------------

[SUCCESS] Serving "build" directory at: http://localhost:3000/

docs/01-Introduction/02-architecture.md Outdated Show resolved Hide resolved
docs/01-Introduction/02-architecture.md Outdated Show resolved Hide resolved
docs/01-Introduction/02-architecture.md Outdated Show resolved Hide resolved
docs/01-Introduction/02-architecture.md Outdated Show resolved Hide resolved
docs/01-Introduction/02-architecture.md Outdated Show resolved Hide resolved
docs/01-Introduction/02-architecture.md Outdated Show resolved Hide resolved
docs/01-Introduction/02-architecture.md Outdated Show resolved Hide resolved
docs/01-Introduction/02-architecture.md Outdated Show resolved Hide resolved
docs/01-Introduction/02-architecture.md Outdated Show resolved Hide resolved
docs/01-Introduction/02-architecture.md Outdated Show resolved Hide resolved
@kamilprz kamilprz linked an issue Jan 17, 2025 that may be closed by this pull request
nddq
nddq previously approved these changes Jan 22, 2025
docs/01-Introduction/01-intro.md Show resolved Hide resolved
@kamilprz kamilprz added this pull request to the merge queue Jan 23, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to a conflict with the base branch Jan 23, 2025
SRodi
SRodi previously approved these changes Jan 23, 2025
Copy link
Member

@SRodi SRodi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should add a section at the very start to explain very high level concepts and capture them in a high-level diagram. This should include data-plane, control-plane, and the types of data-plane. Also, what are the Kubernetes resources that contain the data plane and control-plane (i.e. agent daemonset, operator). Maybe add this in a separate PR as a second iteration? Approving anyway, great work @kamilprz thanks!

ritwikranjan and others added 2 commits January 24, 2025 12:21
…scripts and docs (microsoft#1259)

# Description

This pull request includes significant changes to transition from the
"legacy" to the "standard" directory and package structure. The changes
involve updates to Helm charts, Makefile commands, package imports, and
file paths.

Directory and package structure updates:

*
[`.github/workflows/release-charts.yaml`](diffhunk://#diff-711012dcc1f315d20ef98e04b14cbc0cb1a388641934c2151ac99f74ff11f62cL43-R43):
Updated the Helm package path from `legacy` to `standard`.
*
[`Makefile`](diffhunk://#diff-76ed074a9305c04054cdebb9e9aad2d818052b07091de1f20cad0bbac34ffb52L447-R447):
Updated multiple Helm install commands to use the `standard` directory
instead of `legacy`.
[[1]](diffhunk://#diff-76ed074a9305c04054cdebb9e9aad2d818052b07091de1f20cad0bbac34ffb52L447-R447)
[[2]](diffhunk://#diff-76ed074a9305c04054cdebb9e9aad2d818052b07091de1f20cad0bbac34ffb52L460-R460)
[[3]](diffhunk://#diff-76ed074a9305c04054cdebb9e9aad2d818052b07091de1f20cad0bbac34ffb52L477-R477)
[[4]](diffhunk://#diff-76ed074a9305c04054cdebb9e9aad2d818052b07091de1f20cad0bbac34ffb52L496-R496)
*
[`cmd/root.go`](diffhunk://#diff-ab967ab1a2f3a1b769106eeb7bfe892ef0e81d1d27811fa15be08e6749feee1fL9-R9):
Changed package imports and daemon initialization from `legacy` to
`standard`.
[[1]](diffhunk://#diff-ab967ab1a2f3a1b769106eeb7bfe892ef0e81d1d27811fa15be08e6749feee1fL9-R9)
[[2]](diffhunk://#diff-ab967ab1a2f3a1b769106eeb7bfe892ef0e81d1d27811fa15be08e6749feee1fL31-R31)
* Renamed files from `legacy` to `standard`:
  * `cmd/legacy/daemon.go` to `cmd/standard/daemon.go`
  * `cmd/legacy/daemon_linux.go` to `cmd/standard/daemon_linux.go`
  * `cmd/legacy/daemon_windows.go` to `cmd/standard/daemon_windows.go`
*
[`crd/Makefile`](diffhunk://#diff-414956f6f35c3bc4a93c4c08592ee22397d76d99a1b728e1e33c65075a2a99c8L7-R7):
Updated the `HELM_CRD_DIR` path to use the `standard` directory.

File deletions and renames:

* Deleted `deploy/legacy/prometheus/retina/create-cm.sh`.
* Renamed
`deploy/legacy/manifests/controller/helm/retina/templates/NOTES.txt` to
`deploy/standard/manifests/controller/helm/retina/templates/NOTES.txt`
with updated paths inside the file.

## Related Issue

This PR will close microsoft#1115 

## Checklist

- [x] I have read the [contributing
documentation](https://retina.sh/docs/contributing).
- [x] I signed and signed-off the commits (`git commit -S -s ...`). See
[this
documentation](https://docs.github.com/en/authentication/managing-commit-signature-verification/about-commit-signature-verification)
on signing commits.
- [x] I have correctly attributed the author(s) of the code.
- [x] I have tested the changes locally.
- [x] I have followed the project's style guidelines.
- [x] I have updated the documentation, if necessary.
- [x] I have added tests, if applicable.

## Screenshots (if applicable) or Testing Completed

Please add any relevant screenshots or GIFs to showcase the changes
made.

## Additional Notes

Add any additional notes or context about the pull request here.

---

Please refer to the [CONTRIBUTING.md](../CONTRIBUTING.md) file for more
information on how to contribute to this project.

---------

Signed-off-by: Ritwik Ranjan <[email protected]>
Co-authored-by: kamilprz <[email protected]>
# Description

Add `metricscardinality` to heartbeat. This will give visibility on the
number of time series being exposed by retina.

## Related Issue

microsoft#1040 

## Checklist

- [x] I have read the [contributing
documentation](https://retina.sh/docs/contributing).
- [x] I signed and signed-off the commits (`git commit -S -s ...`). See
[this
documentation](https://docs.github.com/en/authentication/managing-commit-signature-verification/about-commit-signature-verification)
on signing commits.
- [x] I have correctly attributed the author(s) of the code.
- [x] I have tested the changes locally.
- [x] I have followed the project's style guidelines.
- [x] I have updated the documentation, if necessary.
- [x] I have added tests, if applicable.

## Screenshots (if applicable) or Testing Completed

Metrics exported with heartbeat:

![image](https://github.com/user-attachments/assets/6fb4d76b-5780-4751-91c0-46e3c4f0fb85)


## Additional Notes

Metrics of types `histogram` and `summary` expose multiple time series
during a scrape. Code is counting according to number of time series
exposed at /metrics endpoint.


Ref: 
https://prometheus.io/docs/concepts/metric_types/#histogram
https://prometheus.io/docs/concepts/metric_types/#summary


---

Please refer to the [CONTRIBUTING.md](../CONTRIBUTING.md) file for more
information on how to contribute to this project.

Signed-off-by: Alex Castilio dos Santos <[email protected]>
@kamilprz kamilprz dismissed stale reviews from SRodi and nddq via 1d34c3e January 24, 2025 12:21
@kamilprz kamilprz added this pull request to the merge queue Jan 24, 2025
Merged via the queue into microsoft:main with commit af88054 Jan 24, 2025
27 checks passed
@kamilprz kamilprz deleted the architecture-docs branch January 24, 2025 13:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add documentation for Hubble control plane
6 participants