Skip to content

Latest commit

 

History

History
318 lines (220 loc) · 13.4 KB

DEVELOPMENT.md

File metadata and controls

318 lines (220 loc) · 13.4 KB
Raw

Development

This doc explains the development workflow so you can get started contributing to Skaffold!

Requirements

You must install these tools:

  1. go: The language skaffold is built in (version >= go 1.14)
  2. git: For source control
  3. make: For building skaffold.

Getting started

First you will need to setup your GitHub account and create a fork:

  1. Create a GitHub account
  2. Setup GitHub access via SSH
  3. Create and checkout a repo fork

Once you have those, you can iterate on skaffold:

  1. Build your dev version of skaffold
  2. Verify changes locally
  3. Run skaffold tests
  4. Build docs if you are making doc changes

When you're ready, you can create a PR!

You may also be interested in contributing to the docs.

Checkout your fork

To make local changes to skaffold and eventually submit a pull request to the repo, we recommend creating your own fork.

Once you've done this, clone your fork to your local machine:

git clone [email protected]:${YOUR_GITHUB_USERNAME}/skaffold.git
cd skaffold
git remote add upstream [email protected]:GoogleContainerTools/skaffold.git
git remote set-url --push upstream no_push

Adding the upstream remote sets you up nicely for regularly syncing your fork.

IDE setup

Skaffold uses go modules and we commit to the vendor directory all our dependencies to make CI faster.
We recommend checking out Skaffold outside of the GOPATH.

JetBrains Goland:

  1. File > Open... (choose your skaffold directory)
  2. ensure Go > Go modules > Vendoring mode is checked

JetBrains IntelliJ with Go plugin:

  1. File > Open... (choose your skaffold directory)

Visual Studio Code with Go plugin:

  1. File > Open...

Making a config change

Some changes to the skaffold code require a change to the skaffold config. These changes require a few extra steps:

  • Open the latest Config at pkg/skaffold/schema/latest/config.go and inspect the comment at L25

  • If the line mentions the config version is not released, proceed making your changes.

    // This config version is not yet released, it is SAFE TO MODIFY the structs in this file.

  • If the line mentions the config version is released then,

    // !!! WARNING !!! This config version is already released, please DO NOT MODIFY the structs in this file.

    • Run ./hack/new_version.sh to create a new version.

    • Run make test to verify changes.

    • Commit these generated changes, and submit a PR.

Once you've done this, merge or rebase your development branch with config changes, including the new config change. Any new config changes belong in pkg/skaffold/schema/latest/config.go. Do not edit the older config versions.

  • Be sure and update the documentation in pkg/skaffold/schema/<previous_config_version>/upgrade.go with any additions, removals, or updates you make to the config.

  • In case of backwards compatibility issues, update the Upgrade() method from the previous config version appropriately. This is usually required when a previously existing field in the config is changed, but not when a new field is added.

Note: the Upgrade() method is called by skaffold automatically for older config versions. This can also be done manually by users by running skaffold fix.

  • Finally, before committing your final changes and opening your pull request, be sure and run make test locally. This will regenerate the JSON schemas for the skaffold config with your new changes. Commit the resulting changes autogenerated by the scripts.

For more details behind the logic of config changes see the Skaffold config management doc.

Making changes to the Skaffold API

We build the API directly through gRPC, which gets translated into REST API through a reverse proxy gateway library. When adding new message types, make changes to proto/v1/skaffold.proto, and when adding new enum types, make changes to proto/enums/enums.proto. When changing either of these files, you can run ./hack/generate-proto.sh to generate the equivalent Go code.

Adding actionable error messages to code.

Skaffold has a built-in framework to provide actionable error messages for user to help bootstrap skaffold errors.

Also, v1.19.0 onwards, skaffold is collecting failure error codes to help the team get more insights into common failure scenarios.

To take advantage of this framework, contributors can simply use the ErrDef struct to throw meaningful actionable error messages and improve user experience.

e.g In this example PR,

  1. The contributor created 3 distinct error codes in skaffold.proto 
     // Docker build error when listing containers.
 INIT_DOCKER_NETWORK_LISTING_CONTAINERS = 122;
 // Docker build error indicating an invalid container name (or id).
 INIT_DOCKER_NETWORK_INVALID_CONTAINER_NAME = 123;
 // Docker build error indicating the container referenced does not exists in the docker context used.
 INIT_DOCKER_NETWORK_CONTAINER_DOES_NOT_EXIST = 124
    The INIT in this case stands for skaffold INIT phase which includes, parsing of skaffold config and creating a skaffold runner. The other valid phases are BUILD, DEPLOY, STATUSCHECK. Complete list here
  2. Run hack/generate_proto.sh. These will generate go code and structs for the newly added proto fields. 
     git status
    modified:   docs/content/en/api/skaffold.swagger.json
    modified:   docs/content/en/docs/references/api/grpc.md
    modified:   proto/skaffold.pb.go
    modified:   proto/skaffold.proto
  3. The contributor then used these error codes when creating an error in their proposed code change. They used the constructor sErrors.NewError in pkg/skaffold/errors to inantiate an object of struct ErrDef. ErrDef implements the golang error interface. 
     err :=  sErrors.NewError(fmt.Errorf(errMsg), 
   proto.ActionableErr{
     Message: errMsg,
     ErrCode: proto.StatusCode_INIT_DOCKER_NETWORK_INVALID_CONTAINER_NAME,
 	  Suggestions: []*proto.Suggestion{
   	    {
 		  SuggestionCode: proto.SuggestionCode_FIX_DOCKER_NETWORK_CONTAINER_NAME,
 		  Action:         "Please fix the docker network container name and try again",
 	    },
 	  },
   }))

With above two changes, skaffold will now show a meaning full error message when this error condition is met.

skaffold dev 
invalid skaffold config: container 'does-not-exits' not found, required by image 'skaffold-example' for docker network stack sharing.
Please fix the docker network container name and try again.

Building skaffold

To build with your local changes you have two options:

  1. Build the skaffold binary:

    make
./out/skaffold version

    You can then run this binary directly, or copy/symlink it into your path.

  2. Build and install the skaffold binary:

    make install
skaffold version

    This will install skaffold via go install (note that if you have manually downloaded and installed skaffold to /usr/local/bin, this will probably take precedence in your path over your $GOPATH/bin).

    If you are unsure if you are running a released or locally built version of skaffold, you can run skaffold version - output which includes dirty indicates you have built the binary locally.

Verifying local changes

If you are iterating on skaffold and want to see your changes in action, you can:

  1. Build skaffold
  2. Use the quickstart example

Testing skaffold

skaffold has both unit tests and integration tests.

Unit Tests

The unit tests live with the code they test and can be run with:

make test

In case you see a linter error such as:

make test
RUN hack/linter.sh
ERRO Running error: no such linter "gocritic"

re-run the hack/install_golint.sh script to upgrade golangci-lint.

Integration tests

The integration tests live in integration. They can be run with:

make integration

These tests require a Docker daemon, a Kubernetes cluster and all the tools used by every Builder and Deployer, such as kubectl, bazel, java, kustomize...

A way to run the integration tests without installing those tools and without depending on a Kubernetes cluster is to install kind and run:

make integration-in-kind

Running a subset of integration tests

You can select specific integration tests to run via the INTEGRATION_TEST_ARGS env var:

INTEGRATION_TEST_ARGS="-run=TestDev/" make integration

Running GCP specific integration tests

Another set of the integration tests require a GCP project because they will push to a GCR registry or use Cloud Build to build artifacts. Those tests can be run with:

GCP_ONLY=true make integration

These tests will be kicked off by reviewers for submitted PRs.

Building skaffold docs

The latest version of the skaffold site is based on the Hugo theme of the github.com/google/docsy template.

Testing docs locally

Before creating a PR with doc changes, we recommend that you locally verify the generated docs with:

make preview-docs

Once PRs with doc changes are merged, they will get automatically published to the docs for the latest build to https://skaffold-latest.firebaseapp.com. which at release time will be published with the latest release to https://skaffold.dev.

Previewing the docs on the PR

Mark your PR with docs-modifications label. Our PR review process will answer in comments in ~5 minutes with the URL of your preview and will remove the label.

Testing the Skaffold binary release process

Skaffold release process works with Google Cloud Build within our own project k8s-skaffold and the skaffold release bucket, gs://skaffold.

In order to be able to iterate/fix the release process you can pass in your own project and bucket as parameters to the build.

We continuously release builds under gs://skaffold/builds. This is done by triggering cloudbuild.yaml on every push to master.

To run a build on your own project:

gcloud builds submit --config deploy/cloudbuild.yaml --substitutions=_RELEASE_BUCKET=<personal-bucket>,COMMIT_SHA=$(git rev-parse HEAD) --project <personalproject>

We release stable versions under gs://skaffold/releases. This is done by triggering cloudbuild-release.yaml on every new tag in our Github repo.

To test a release on your own project:

gcloud builds submit --config deploy/cloudbuild-release.yaml --substitutions=_RELEASE_BUCKET=<personal-bucket>,TAG_NAME=testrelease_v1234 --project <personalproject>

To just run a release without Google Cloud Build only using your local Docker daemon, you can run:

make -j release GCP_PROJECT=<personalproject> RELEASE_BUCKET=<personal-bucket>

Creating a PR

When you have changes you would like to propose to skaffold, you will need to:

  1. Ensure the commit message(s) describe what issue you are fixing and how you are fixing it (include references to issue numbers if appropriate)
  2. Add unit tests. Unit test coverage should increase or stay the same with every PR.
  3. Create a pull request

Please follow our small Pull Requests guidelines for quicker response time.

Reviews

Each PR must be reviewed by a maintainer. This maintainer will add the kokoro:run label to a PR to kick off the integration tests, which must pass for the PR to be submitted.