Skip to content

Deployment Details

Jump to bottom
JK edited this page Oct 28, 2024 · 3 revisions

Deployment Details

https://github.com/eclipse/openvsx

Core implementation

  • Java/Spring server app
  • Postgres DB
  • ElasticSearch
  • Blob storage (Azure or Google Cloud)
  • React/TypeScript UI

https://github.com/EclipseFdn/open-vsx.org

Deployment specific details for open-vsx.org deployment

https://github.com/open-vsx/publish-extensions

Scripts and Github actions for publishing

  • extensions.json file for list of extensions to auto-publish
  • Github action for nightly auto-publishing
  • log of results
  • Other Github actions for publishing extensions

Publishing an Extension

Packaging

  • Create a .vsix file with the vsce package command
  • Minimal package includes extension.js and package.json

Publishing

  • Click 'Publish' and drag and drop .vsix file in the UI.
  • Use the ovsx publish --p [personal access token] command. Requires a personal access token one generates from profile settings.
  • Because file processing is asynchronous, extension will temporarily show as inactive until file processing completes, assuming no errors.

Deployment

Basic implementation architecture

Documentation on architecture of implementation and instructions for building and deploying.

Database tables - Core tables

user_data
extension
extension_review
extension_version
file_resource
namespace
namespace_membership
namespace_social_links
personal_access_token

Recurring Jobs

  • update-download-counts - hourly
  • MonthlyAdminStatistics - monthly
  • ElasticSearchUpdateIndex - daily
  • DataMirror, if enabled - configurable

Background processes

  • File processing on publish
  • Namespace rename/merge
  • DB migration
  • Extension file removal
  • Generate/delete keypair for extension signing
  • Create/delete extension signature

Open-vsx.org Deployment

Production: open-vsx.org
Staging: staging.open-vsx.org (creds required for access)

Deployment details

  • Production Kube Cluster
    • 2 server pods (with up to 5 CPUs and 8 GB of RAM)
    • 3 elastic search pods (with up to 4 CPUs and 6 GB of RAM)
  • Staging Kube Cluster
    • 1 server pod (with up to 1 CPU and 2 GB of RAM)
    • 1 elastic search pod (with up to 1 CPU and 2 GB of RAM)
  • Spring Boot
    • v3.1.0
  • Java
    • eclipse-temurin:17.0.7_7-jdk
  • Postgres
    • version?
    • 39GB
  • Elastic
    • v8.7.1
    • 5MB
  • Azure Blob storage
    • 1.5TB

Monitoring and Alerting

  • Better Uptime

Deployment Process

  1. SRE creates a pull request in the eclipse/openvsx repository.
  2. GitHub runs Eclipse Contributor Agreement action to check that the committer has signed the Eclipse Contributor Agreement.
  3. GitHub action runs unit tests and integration tests.
  4. SRE merges the pull request into the eclipse/openvsx master branch.
  5. SRE creates new release in the eclipse/openvsx repository:
    1. SRE merges @dependabot pull requests to fix vulnerabilities.
    2. SRE drafts a new release and publishes it.
    3. GitHub action creates server and webui Docker images.
    4. if webui has changed, then SRE publishes new openvsx-webui version to NPM.
    5. if cli has changed, then SRE publishes new ovsx version to NPM.
    6. SRE updates release post with links to server and webui Docker images and, if changed, openvsx-webui and ovsx NPM packages.
  6. SRE updates DockerFile in amvanbaren/open-vsx.org repository with new openvsx-server Docker image version and, if changed, new openvsx-webui package version.
  7. if openvsx-webui version has changed, then SRE updates the openvsx-webui version in the website/package.json file and runs yarn install to update the yarn.lock file.
  8. SRE commits and pushes the changes to a new branch in the amvanbaren/open-vsx.org repository.
  9. SRE creates pull request to merge the new amvanbaren/open-vsx.org branch into EclipseFdn/open-vsx.org repository main branch.
  10. GitHub runs Eclipse Contributor Agreement action to check that the committer has signed the Eclipse Contributor Agreement.
  11. GitHub action kicks off Jenkins job to verify the new Docker image.
  12. SRE merges the pull request into the main branch.
  13. GitHub action kicks off Jenkins job to build the Docker image and deploy it to staging.
  14. SRE checks the staging server and does some quick smoke testing.
  15. SRE creates pull request in EclipseFdn/open-vsx.org repository to merge the main branch into the production branch.
  16. Independent review and approval of the pull request.
  17. SRE merges the pull request into the production branch.
  18. GitHub action kicks off Jenkins job to build the Docker image and deploy it to production.
  19. SRE monitors production for ~15 min to make sure the deployment succeeded and the server is stable.

Common Error Situations and Remedies

  1. ElasticSearch unresponsive:
    1. Delete ElasticSearch pods.
    2. In server application.yml set ovsx.elasticsearch.clear-on-start: true.
    3. Wait until all ElasticSearch pods have started.
    4. Deploy server application.
    5. In server application.yml remove ovsx.elasticsearch.clear-on-start: true.
  2. Open VSX server unresponsive:
    1. Delete Open VSX server pod.
    2. Wait until new pod has started.
    3. Repeat the previous steps until all Open VSX server pods are renewed.
  3. Too much load:
    1. Delete pod that is under load.
    2. Wait until new pod has started.

Status

Site status and 90 day history are available at https://status.open-vsx.org/. These are a series of https://uptime.betterstack.com monitors. Site admins have access to an API for access to historical data.

Outages

Outages show up on the status page and as an incident report at https://status.open-vsx.org/incidents.

Handling an Outage

Once an outage is detected, an incident is created at https://status.open-vsx.org/incidents. The support acknowledges the incident on that page to indicate they are aware of the issue and are working it. If it's a lengthy outage, status reports may be appropriate.

One the outage is resolved, the support team documents the cause and actions taken.

Planned Outages

Planned outages are communicated with:

Clone this wiki locally