This repository demonstrates a possible GitOps workflow using Argo CD and Tekton. We are using Argo CD to setup our Kubernetes clusters dev and prod (in the following we will only use the dev cluster) and Tekton to build and update our example application.
$ brew install kn # knative
$ brew install knative-sandbox/kn-plugins/admin - optional
$ brew install knative-sandbox/kn-plugins/quickstart
$ brew install hey # loadtest$ kn quickstart minikube
$ minikube tunnel --profile knative
$ http://hello.default.127.0.0.1.sslip.io/
$ kn revision list$ minikube addons enable dashboard --profile=knative
$ minikube -p knative addons enable metrics-server
$ minikube dashboard --url --profile knativeIn the next step we have to to install Argo CD by running the following command:
kubectl create namespace argocd
kustomize build clusters/argocd/dev | k apply -f -$ helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
$ helm repo update
$ helm install prometheus prometheus-community/kube-prometheus-stack -n default -f values.yaml
$ kubectl port-forward -n default svc/prometheus-operated 9090
$ helm repo add grafana https://grafana.github.io/helm-charts
$ kubectl apply -f https://raw.githubusercontent.com/knative-sandbox/monitoring/main/grafana/dashboards.yaml
$ kubectl --namespace default port-forward $POD_NAME 3000
$ kubectl get secret --namespace default prometheus-grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echoNote : May manually load from grafana dashboard
$ https://github.com/argoproj/argo-cd/blob/master/examples/dashboard.jsonkubectl create namespace argo-rollouts
kubectl apply -n argo-rollouts -f https://github.com/argoproj/argo-rollouts/releases/latest/download/install.yamlAfter some minutes we have to verify that Argo CD is running:
kubectl get pods -n argocdargocd-application-controller-75b4dcd7bb-97wm8 1/1 Running 0 3m50s
argocd-dex-server-996685b6d-zffmd 1/1 Running 0 3m50s
argocd-redis-99fb49846-pqn7p 1/1 Running 0 3m50s
argocd-repo-server-5c76bd686b-79gjn 1/1 Running 0 3m49s
argocd-server-67885bdcff-5q5cz 1/1 Running 0 3m49sTo deploy our manifests to the cluster we are using the app of apps pattern. For that we have to create a new Application, which manages all other applications (including Argo CD):
kubectl apply -f clusters/apps/dev.yamlNow we have to add our Ingresses to the /etc/hosts file:
sudo echo "`minikube ip --profile=dev` argocd-dev.fake grafana-dev.fake prometheus-dev.fake tekton-dev.fake server-dev.fake" | sudo tee -a /etc/hostsNow we can visit argocd-dev.fake and login with the default admin user. The initial password is autogenerated to be the pod name of the Argo CD API server. This can be retrieved with the command:
#kubectl get pods -n argocd -l app.kubernetes.io/name=argocd-server -o name | cut -d'/' -f 2
kubectl port-forward svc/argocd-server -n argocd 8080:443
kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -dIn the UI of Argo CD we can now see all deployed applications:
This example also deploys the Prometheus Stack via the kube-prometheus-stack Helm chart. We are using the Flux Helm Operator instead of the Argo CD to deploy the Helm chart. When the Helm chart was successfully synced Prometheus is available at prometheus-dev.fake and Grafana at grafana-dev.fake.
In the next step we can login to Grafana with the admin user and the password admin. Then we can import the example Dashboard for Argo CD. The dashboard can be found in the GitHub repository of the Argo CD project at https://github.com/argoproj/argo-cd/blob/master/examples/dashboard.json.
NOTE: To deploy Tekton using Argo CD we had slightly adjusted the following files:
We had to remove the
preserveUnknownFieldsfield from all CRDs and we had to change theapp.kubernetes.io/instancelabel fromdefaulttotektonfor all manifests.
We are using Tekton Pipelines to build our example application and to update the deployed Docker image. The required tasks and pipelines are deployed by Argo CD, but we have to manually provide the credentials for Docker Hub and GitHub. To create the secrets we have to run the following commands:
kubectl create secret docker-registry docker-registry-secret --docker-server="https://index.docker.io/v1/" --docker-username=<DOCKER_USERNAME> --docker-password=<DOCKER_PASSWORD>
kubectl annotate secret docker-registry-secret tekton.dev/docker-0=https://index.docker.io/v1/
kubectl create secret generic github-secret --type="kubernetes.io/basic-auth" --from-literal=username=<GITHUB_USERNAME> --from-literal=password=<GITHUB_PASSWORD>
kubectl annotate secret github-secret tekton.dev/git-0=https://github.comIn a production ready setup, we should use the Vault Secrets Operator or a similar tool to manage our secrets.
When we have created the secrets, we can run our pipeline, to build a new Docker image for the development and production environment:
cat <<EOF | kubectl create -f -
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
name: build-and-deploy-dev-1
annotations:
argocd.argoproj.io/compare-options: IgnoreExtraneous
spec:
pipelineRef:
name: build-and-deploy-pipeline
params:
- name: gitUrl
value: https://github.com/bilalislam/gitops-using-argo-cd-and-tekton
- name: gitRevision
value: dev
- name: imageUrl
value: ninjafx/gitops-using-argo-cd-and-tekton
- name: serviceName
value: server
- name: filePath
value: clusters/server/dev/server-deploy.yaml
- name: yamlField
value: spec.template.spec.containers[0].image
serviceAccountName: pipeline-account
workspaces:
- name: git-source
persistentVolumeClaim:
claimName: git-source-pvc
EOFcat <<EOF | kubectl create -f -
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
name: build-and-deploy-prod-1
annotations:
argocd.argoproj.io/compare-options: IgnoreExtraneous
spec:
pipelineRef:
name: build-and-deploy-pipeline
params:
- name: gitUrl
value: https://github.com/bilalislam/gitops-using-argo-cd-and-tekton
- name: gitRevision
value: main
- name: imageUrl
value: ninjafx/gitops-using-argo-cd-and-tekton
- name: serviceName
value: server
- name: filePath
value: clusters/server/prod/server-deploy.yaml
- name: yamlField
value: spec.template.spec.containers[0].image
serviceAccountName: pipeline-account
workspaces:
- name: git-source
persistentVolumeClaim:
claimName: git-source-pvc
EOFNow we can visit tekton-dev.fake to see the results of our above pipeline runs.
When the pipeline ran successfully, we can also watch the Pods in the server Namespace, were we can see that our example application is automatically updated after some time:
k get pods -w --namespace serverNAME READY STATUS RESTARTS AGE
server-cbfbc758f-7w6lx 1/1 Running 0 14s
server-cbfbc758f-7w6lx 1/1 Terminating 0 24s
server-cbfbc758f-7w6lx 0/1 Terminating 0 25s
server-cbfbc758f-7w6lx 0/1 Terminating 0 26s
server-cbfbc758f-7w6lx 0/1 Terminating 0 26s
server-5b5c778558-2krtw 0/1 Pending 0 0s
server-5b5c778558-2krtw 0/1 Pending 0 0s
server-5b5c778558-2krtw 0/1 ContainerCreating 0 0s
server-5b5c778558-2krtw 0/1 Running 0 1s
server-5b5c778558-2krtw 1/1 Running 0 8sWe can also visit server-dev.fake were we should see the greeting Hello World. When we change line 25 in the cmd/server/server.go file and merge our changes to the dev branch and rerun our pipeline, we should also see an updated greeting.
Instead of manually running our pipeline, we can also use Tekton Triggers in the future to run our pipeline on each commit to the dev and main branch.
