-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #8 from kubeshop/f1ames/feat/deployment
Add Helm chart for deployment
- Loading branch information
Showing
26 changed files
with
2,093 additions
and
317 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,101 @@ | ||
name: Test | ||
|
||
on: | ||
pull_request: | ||
push: | ||
branches: | ||
- main | ||
|
||
concurrency: ${{ github.workflow }}-${{ github.ref }} | ||
|
||
jobs: | ||
test-skaffold: | ||
name: Skaffold | ||
runs-on: ubuntu-latest | ||
steps: | ||
- name: Setup Skaffold | ||
run: curl -Lo skaffold https://storage.googleapis.com/skaffold/releases/latest/skaffold-linux-amd64 && sudo install skaffold /usr/local/bin/ | ||
|
||
- name: Setup Node.js | ||
uses: actions/setup-node@v2 | ||
with: | ||
node-version: 18 | ||
|
||
- name: Checkout Repo | ||
uses: actions/checkout@v2 | ||
|
||
- name: Start Minikube | ||
run: minikube start --extra-config=apiserver.enable-admission-plugins=ValidatingAdmissionWebhook | ||
|
||
- name: Deploy Admission Controller | ||
run: ./scripts/deploy.sh | ||
|
||
- name: Preview namespace state | ||
run: kubectl -n monokle-admission-controller get all,CustomResourceDefinition,ValidatingWebhookConfiguration,secrets | ||
|
||
- name: Install Dependencies | ||
working-directory: ./tests/ | ||
run: npm ci | ||
|
||
- name: Test | ||
working-directory: ./tests/ | ||
run: npm test | ||
|
||
test-helm: | ||
name: Helm | ||
runs-on: ubuntu-latest | ||
strategy: | ||
fail-fast: true | ||
matrix: | ||
k8s: [v1.25.15, v1.26.10, v1.27.7, v1.28.3] | ||
helm: [v3.11.3, v3.12.3, v3.13.1] | ||
|
||
steps: | ||
- name: Setup Node.js | ||
uses: actions/setup-node@v2 | ||
with: | ||
node-version: 18 | ||
|
||
- name: Setup Helm | ||
uses: azure/setup-helm@v3 | ||
with: | ||
version: ${{ matrix.helm }} | ||
|
||
- name: Debug helm | ||
run: helm version | ||
|
||
- name: Checkout Repo | ||
uses: actions/checkout@v2 | ||
|
||
- name: Start Minikube | ||
run: minikube start --kubernetes-version=${{ matrix.k8s }} --extra-config=apiserver.enable-admission-plugins=ValidatingAdmissionWebhook | ||
|
||
- name: Build local images (init container) | ||
run: eval $(minikube -p minikube docker-env) && minikube image build -t admission-webhook-init -f ./Dockerfile . | ||
working-directory: ./admission-controller/init | ||
|
||
- name: Build local images (server container) | ||
run: eval $(minikube -p minikube docker-env) && minikube image build -t admission-webhook -f ./Dockerfile . | ||
working-directory: ./admission-controller/server | ||
|
||
- name: Helm install | ||
run: | | ||
eval $(minikube -p minikube docker-env) | ||
helm package ./helm --version 0.0.0 | ||
helm install monokle-ac monokle-admission-controller-0.0.0.tgz \ | ||
--set namespace=mac-test \ | ||
--set image.init.pullPolicy=Never \ | ||
--set image.init.overridePath=admission-webhook-init \ | ||
--set image.server.pullPolicy=Never \ | ||
--set image.server.overridePath=admission-webhook | ||
- name: Preview namespace state | ||
run: kubectl -n mac-test get all,CustomResourceDefinition,ValidatingWebhookConfiguration,secrets | ||
|
||
- name: Install Dependencies | ||
working-directory: ./tests/ | ||
run: npm ci | ||
|
||
- name: Test | ||
working-directory: ./tests/ | ||
run: MONOKLE_NAMESPACE=mac-test npm test |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -63,3 +63,6 @@ test/types/index.js | |
|
||
# compiled app | ||
dist | ||
|
||
# other tmp files | ||
scripts/install.yaml |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,181 @@ | ||
# Contributing and development | ||
|
||
## Prerequisites | ||
|
||
* Minikube (or any other K8s cluster running) | ||
* kubectl | ||
* helm | ||
* Skaffold | ||
* nodejs | ||
|
||
## Running | ||
|
||
Start Minikube (or any other cluster of your choice): | ||
|
||
```bash | ||
minikube start --extra-config=apiserver.enable-admission-plugins=ValidatingAdmissionWebhook | ||
``` | ||
|
||
### Deploying (via Skaffold) | ||
|
||
```bash | ||
./scripts/deploy.sh | ||
``` | ||
|
||
After changes in `./admission-controller` can be reloaded with: | ||
|
||
```bash | ||
skaffold dev -f scripts/skaffold.yaml | ||
``` | ||
|
||
You can also do manual clean-up and re-run `deploy.sh` script again: | ||
|
||
```bash | ||
kubectl delete all -n monokle-admission-controller --all && \ | ||
kubectl delete validatingwebhookconfiguration.admissionregistration.k8s.io/monokle-admission-controller-webhook && \ | ||
kubectl delete namespace monokle-admission-controller && \ | ||
kubectl delete namespace nstest1 && \ | ||
kubectl delete namespace nstest2 && \ | ||
kubectl delete crd policies.monokle.io && \ | ||
kubectl delete crd policybindings.monokle.io | ||
``` | ||
|
||
### Deploying (via Helm + Minikube registry) | ||
|
||
Deploying with helm requires having local docker registry (for locally build images). THis can be done with Minikube: | ||
|
||
```bash | ||
eval $(minikube -p minikube docker-env) | ||
``` | ||
|
||
And then building images: | ||
|
||
```bash | ||
cd admission-controller/init | ||
minikube image build -t admission-webhook-init -f ./Dockerfile . | ||
|
||
cd admission-controller/server | ||
minikube image build -t admission-webhook -f ./Dockerfile . | ||
|
||
docker images | ||
``` | ||
|
||
```bash | ||
helm install monokle-ac ./helm \ | ||
--set image.init.pullPolicy=Never \ | ||
--set image.init.overridePath=admission-webhook-init \ | ||
--set image.server.pullPolicy=Never \ | ||
--set image.server.overridePath=admission-webhook | ||
``` | ||
|
||
To uninstall: | ||
|
||
```bash | ||
helm uninstall monokle-ac | ||
``` | ||
|
||
### Checking deployment state | ||
|
||
Namespaced resources (webhook server) will be deployed to dedicated `monokle-admission-controller` namespace, to watch it you can run: | ||
|
||
```bash | ||
watch kubectl -n monokle-admission-controller get all,CustomResourceDefinition,ValidatingWebhookConfiguration,secrets | ||
``` | ||
|
||
After it runs, the result should be something like: | ||
|
||
```bash | ||
NAME READY STATUS RESTARTS AGE | ||
pod/monokle-admission-controller-server-6958c9bbf8-jvkvk 1/1 Running 0 5m11s | ||
|
||
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE | ||
service/monokle-admission-controller-server ClusterIP 10.99.122.106 <none> 443/TCP 5m11s | ||
|
||
NAME READY UP-TO-DATE AVAILABLE AGE | ||
deployment.apps/monokle-admission-controller-server 1/1 1 1 5m11s | ||
|
||
NAME DESIRED CURRENT READY AGE | ||
replicaset.apps/monokle-admission-controller-server-6958c9bbf8 1 1 1 5m11s | ||
|
||
NAME CREATED AT | ||
customresourcedefinition.apiextensions.k8s.io/policies.monokle.io 2023-10-12T12:16:04Z | ||
customresourcedefinition.apiextensions.k8s.io/policybindings.monokle.io 2023-10-12T12:16:04Z | ||
|
||
NAME WEBHOOKS AGE | ||
validatingwebhookconfiguration.admissionregistration.k8s.io/monokle-admission-controller-webhook 1 5m11s | ||
|
||
NAME TYPE DATA AGE | ||
secret/default-token-w56nz kubernetes.io/service-account-token 3 5m39s | ||
secret/monokle-admission-controller-tls kubernetes.io/tls 2 5m1s | ||
secret/monokle-policies-sa-token-fcpld kubernetes.io/service-account-token 3 5m49s | ||
``` | ||
|
||
For getting info about CRDs: | ||
|
||
```bash | ||
kubectl get crd | ||
kubectl describe crd policies.monokle.io | ||
kubectl describe crd policybindings.monokle.io | ||
``` | ||
|
||
#### Init container logs | ||
|
||
The `monokle-admission-controller-server` has one init container which is responsible for certificate creation/renewal and propagation into cluster. Logs from it can be viewed with: | ||
|
||
```bash | ||
kubectl -n monokle-admission-controller logs pod/monokle-admission-controller-server-... -c init | ||
``` | ||
|
||
## Testing | ||
|
||
Create test namespaces first: | ||
|
||
```bash | ||
kubectl create namespace nstest1 | ||
kubectl create namespace nstest2 | ||
``` | ||
|
||
First you need to create policy resource, for example: | ||
|
||
```bash | ||
kubectl apply -f examples/policy-sample-1.yaml | ||
kubectl apply -f examples/policy-sample-2.yaml | ||
``` | ||
|
||
Then it needs to be bind to be used for validation. Either without scope (globally to all, but ignored namespaces) or with `matchResource` field: | ||
|
||
```bash | ||
kubectl apply -f examples/policy-binding-sample-1.yaml | ||
kubectl apply -f examples/policy-binding-sample-2.yaml | ||
kubectl apply -f examples/policy-binding-sample-3.yaml | ||
``` | ||
|
||
You can inspect deployed policies with: | ||
|
||
```bash | ||
kubectl get policy | ||
kubectl describe policy | ||
|
||
kubectl get policybinding | ||
kubectl describe policybinding | ||
``` | ||
|
||
Then you can try to create sample resource and see webhook response: | ||
|
||
```bash | ||
kubectl apply -f examples/pod-valid.yaml | ||
kubectl apply -f examples/pod-warning.yaml | ||
kubectl apply -f examples/pod-errors.yaml | ||
``` | ||
|
||
## Refs | ||
|
||
* https://kubernetes.io/blog/2019/03/21/a-guide-to-kubernetes-admission-controllers/ | ||
* https://github.com/stackrox/admission-controller-webhook-demo/tree/master | ||
* https://www.witodelnat.eu/blog/2021/local-kubernetes-development | ||
* https://minikube.sigs.k8s.io/docs/tutorials/using_psp/ | ||
* https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/ | ||
* https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/ | ||
* https://kubernetes.io/docs/concepts/workloads/pods/init-containers/ | ||
* https://kubernetes-client.github.io/javascript/index.html | ||
* https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/ |
Oops, something went wrong.