-
Notifications
You must be signed in to change notification settings - Fork 262
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Vaibhav Thakur
committed
Aug 13, 2023
0 parents
commit 4bbcfb5
Showing
189 changed files
with
51,296 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,70 @@ | ||
# Some important production-grade Kubernetes Ops Services | ||
|
||
## Monitoring and Alerting | ||
|
||
1. [Standalone Prometheus](https://github.com/Thakurvaibhav/k8s/tree/master/monitoring#standalone-prometheus-setup) | ||
2. [Clustered and Scalable Prometheus using Thanos](https://github.com/Thakurvaibhav/k8s/tree/master/monitoring#highly-available-and-scalable-clustered-prometheus-setup-using-thanos) | ||
|
||
#### BLOG | ||
|
||
[Kubernetes Multi-Cluster Monitoring using Prometheus and Thanos](https://medium.com/faun/kubernetes-multi-cluster-monitoring-using-prometheus-and-thanos-7549a9b0d0ae) | ||
|
||
[Prodcution Grade Kubernetes Monitoring using Prometheus](https://medium.com/faun/production-grade-kubernetes-monitoring-using-prometheus-78144b835b60) | ||
|
||
## Logging | ||
|
||
1. [Filebeat](https://github.com/Thakurvaibhav/k8s/tree/master/logging#elasticsearch-logging-stack) | ||
2. [Fluentd](https://github.com/Thakurvaibhav/k8s/tree/master/logging#elasticsearch-logging-stack) | ||
3. [FluentBit](https://github.com/Thakurvaibhav/k8s/tree/master/logging#elasticsearch-logging-stack) | ||
|
||
## CI/CD | ||
|
||
1. [Spinnaker-Halyard](https://github.com/Thakurvaibhav/k8s/tree/master/ci-cd/spinnaker-halyard#halyard-kubernetes-manifests-which-can-be-used-to-manage-production-grade-spinnaker-installations) | ||
|
||
#### BLOG | ||
|
||
[Continuous Delivery Pipeline for Kubernetes using Spinnaker](https://medium.com/faun/continuous-delivery-pipeline-for-kubernetes-using-spinnaker-225fe9c9a6e6) | ||
|
||
## AWS Resource Access | ||
|
||
1. [KIAM](https://github.com/Thakurvaibhav/k8s/tree/master/aws-resource-access/kiam#install-kiam) | ||
2. [Kube2IAM](https://github.com/Thakurvaibhav/k8s/tree/master/aws-resource-access/kube2iam#install-kube2iam) | ||
|
||
#### BLOG | ||
|
||
[AWS Resource Access from Kubernetes](https://www.metricfire.com/blog/kubernetes-on-aws-resources) | ||
|
||
## Ingress Controllers | ||
|
||
1. [Nginx Ingress Controller](https://github.com/Thakurvaibhav/k8s/tree/master/ingress-controllers/nginx#ingress-controllers-for-aws-and-gke-based-kubernetes-clusters) | ||
2. [Traefik Ingress Controller](https://github.com/Thakurvaibhav/k8s/tree/master/ingress-controllers/traefik#traefik-ingress-controller) | ||
|
||
## Custom Metrics Auto-Scaling | ||
|
||
1. [Prometheus-Adapter](https://github.com/Thakurvaibhav/k8s/tree/master/custom-metrics-hpa#kubernetes-horizontal-pod-auto-scaling-using-prometheus-metrics) | ||
|
||
## Databases | ||
|
||
1. [MongoDB](https://github.com/Thakurvaibhav/k8s/tree/master/databases/mongodb#prodcution-grade-mongo-db-set-up) | ||
2. [Elasticsearch](https://github.com/Thakurvaibhav/k8s/tree/master/databases/elasticsearch#prodcution-grade-elastic-search-db-set-up) | ||
|
||
#### BLOG | ||
|
||
[Scaling MongoDB on Kubernetes](https://medium.com/faun/scaling-mongodb-on-kubernetes-32e446c16b82) | ||
[Highly Available and Scalable Elasticsearch on Kubernetes](https://medium.com/faun/https-medium-com-thakur-vaibhav23-ha-es-k8s-7e655c1b7b61) | ||
|
||
## AWS ADDONS | ||
|
||
1. [Cluster Autoscaler](https://github.com/Thakurvaibhav/k8s/tree/master/aws-addons/cluster-autoscaler#cluster-autoscaler-for-kubernetes-on-aws) | ||
|
||
## Distributed KV Stores | ||
|
||
1. [Consul](https://github.com/Thakurvaibhav/k8s/tree/master/consul#consul-helm-chart) | ||
|
||
## Distirbuted Secret Stores | ||
|
||
1. [Vault](https://github.com/Thakurvaibhav/k8s/tree/master/vault#vault-helm-chart) | ||
|
||
## Ingress Certificate Manager | ||
|
||
1. [Cert-manager](https://github.com/Thakurvaibhav/k8s/tree/master/cert-manager#letsencrypt-certificate-manager-for-kuberntes-on-gke) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,82 @@ | ||
# Cluster Autoscaler for Kubernetes on AWS | ||
|
||
|
||
## Add labels to node groups and modify IAM policy to Scale-In Scale-Out | ||
|
||
`kops edit ig nodes` | ||
|
||
Now Add the new labels in cloudLabels key. | ||
|
||
``` | ||
spec: | ||
cloudLabels: | ||
k8s.io/cluster-autoscaler/k8s.mydomain.com: "" | ||
k8s.io/cluster-autoscaler/enabled: "" | ||
k8s.io/cluster-autoscaler/node-template/label: "" | ||
kubernetes.io/cluster/k8s.mydomain.com: owned | ||
... | ||
minSize: 2 | ||
maxSize: 5 | ||
``` | ||
|
||
`kops edit cluster` | ||
|
||
Now add the policy. | ||
|
||
``` | ||
... | ||
kind: Cluster | ||
spec: | ||
additionalPolicies: | ||
node: | | ||
[ | ||
{ | ||
"Effect": "Allow", | ||
"Action": [ | ||
"autoscaling:DescribeAutoScalingGroups", | ||
"autoscaling:DescribeAutoScalingInstances", | ||
"autoscaling:SetDesiredCapacity", | ||
"autoscaling:DescribeLaunchConfigurations", | ||
"autoscaling:DescribeTags", | ||
"autoscaling:TerminateInstanceInAutoScalingGroup" | ||
], | ||
"Resource": ["*"] | ||
} | ||
] | ||
... | ||
``` | ||
|
||
### Review updates | ||
|
||
`kops update cluster` | ||
|
||
### Apply updates | ||
|
||
`kops update cluster --yes` | ||
|
||
### Check if rolling-update is needed | ||
|
||
`kops rolling-update cluster` | ||
|
||
### Perform rolling-update if required | ||
|
||
`kops rolling-update cluster --yes` | ||
|
||
|
||
## Install Cluster Autosacaler | ||
|
||
``` | ||
helm install --name cluster-autoscaler \ | ||
--namespace kube-system \ | ||
--set image.tag=v1.14.6 \ | ||
--set autoDiscovery.clusterName=k8s.mydomain.com \ | ||
--set extraArgs.balance-similar-node-groups=false \ | ||
--set extraArgs.expander=random \ | ||
--set rbac.create=true | ||
--set rbac.pspEnabled=true \ | ||
--set awsRegion=us-east-2 \ | ||
--set nodeSelector."node-role\.kubernetes\.io/master"="" \ | ||
--set tolerations[0].effect=NoSchedule \ | ||
--set tolerations[0].key=node-role.kubernetes.io/master \ | ||
--set cloudProvider=aws stable/cluster-autoscaler | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,173 @@ | ||
# Install kiam | ||
|
||
## Creating IAM Roles | ||
|
||
1. Create the IAM role called `kiam-server` | ||
|
||
2. Enable `Trust Relationship` between the newly created role and role attached to Kubernetes cluster master nodes. | ||
- Go to the newly created role in AWS console and Select `Trust relationships` tab | ||
- Click on `Edit trust relationship` | ||
- Add the following content to the policy: | ||
``` | ||
{ | ||
"Sid": "", | ||
"Effect": "Allow", | ||
"Principal": { | ||
"AWS": "<ARN_KUBERNETES_MASTER_IAM_ROLE>" | ||
}, | ||
"Action": "sts:AssumeRole" | ||
} | ||
``` | ||
|
||
3. Add inline policy to the `kiam-server` role | ||
``` | ||
{ | ||
"Version": "2012-10-17", | ||
"Statement": [ | ||
{ | ||
"Effect": "Allow", | ||
"Action": [ | ||
"sts:AssumeRole" | ||
], | ||
"Resource": "*" | ||
} | ||
] | ||
} | ||
``` | ||
|
||
4. Create the IAM role (let's call it `my-role`) with appropriate access to AWS resources. | ||
|
||
5. Enable `Trust Relationship` between the newly created role and role attached to Kiam server role. | ||
- Go to the newly created role in AWS console and Select `Trust relationships` tab | ||
- Click on `Edit trust relationship` | ||
- Add the following content to the policy: | ||
``` | ||
{ | ||
"Sid": "", | ||
"Effect": "Allow", | ||
"Principal": { | ||
"AWS": "<ARN_KIAM-SERVER_IAM_ROLE>" | ||
}, | ||
"Action": "sts:AssumeRole" | ||
} | ||
``` | ||
|
||
6. Enable Assume Role for Master Pool IAM roles. Add the following content as inline policy to Master IAM roles: | ||
``` | ||
{ | ||
"Version": "2012-10-17", | ||
"Statement": [ | ||
{ | ||
"Effect": "Allow", | ||
"Action": [ | ||
"sts:AssumeRole" | ||
], | ||
"Resource": "<ARN_KIAM-SERVER_IAM_ROLE>" | ||
} | ||
] | ||
} | ||
``` | ||
|
||
|
||
## Deploying Cert-Manager | ||
|
||
1. Deploy cert-manager | ||
- Install the CustomResourceDefinition resources separately | ||
|
||
`kubectl apply -f https://raw.githubusercontent.com/jetstack/cert-manager/release-0.8/deploy/manifests/00-crds.yaml` | ||
- Create the namespace for cert-manager | ||
|
||
`kubectl create namespace cert-manager` | ||
- Label the cert-manager namespace to disable resource validation | ||
|
||
`kubectl label namespace cert-manager certmanager.k8s.io/disable-validation=true` | ||
- Add the Jetstack Helm repository | ||
|
||
`helm repo add jetstack https://charts.jetstack.io` | ||
- Update your local Helm chart repository cache | ||
|
||
`helm repo update` | ||
- Install the cert-manager Helm chart | ||
|
||
`helm install --name cert-manager --namespace cert-manager --version v0.8.0 jetstack/cert-manager` | ||
- This set-up is enough for kiam to work. However detailed Steps can be found [here](https://cert-manager.readthedocs.io/en/latest/getting-started/install/kubernetes.html#steps) | ||
|
||
2. Generate CA private key and self-signed certificate for kiam agent-server TLS | ||
- `openssl genrsa -out ca.key 2048` | ||
- `openssl req -x509 -new -nodes -key ca.key -subj "/CN=kiam" -out kiam.cert -days 3650 -reqexts v3_req -extensions v3_ca -out ca.crt` | ||
- Save the CA key pair as a secret in Kubernetes | ||
``` | ||
kubectl create secret tls kiam-ca-key-pair \ | ||
--cert=ca.crt \ | ||
--key=ca.key \ | ||
--namespace=cert-manager | ||
``` | ||
- Deploy cluster issuer, certificate and issue the certificate | ||
``` | ||
kubectl apply -f kiam/namespace.yaml | ||
kubectl apply -f kiam/certificate.yaml | ||
``` | ||
|
||
- Test if certificates are issued correctly | ||
``` | ||
kubectl -n kiam get secret kiam-agent-tls -o yaml | ||
kubectl -n kiam get secret kiam-server-tls -o yaml | ||
``` | ||
|
||
3. Annotating Resources | ||
- Add the IAM role's name to Deployment as an annotation | ||
``` | ||
apiVersion: extensions/v1beta1 | ||
kind: Deployment | ||
metadata: | ||
name: mydeployment | ||
namespace: default | ||
spec: | ||
... | ||
minReadySeconds: 5 | ||
template: | ||
annotations: | ||
iam.amazonaws.com/role: my-role | ||
spec: | ||
containers: | ||
... | ||
``` | ||
- Add role annotation to the namespace in which pods will run | ||
``` | ||
apiVersion: v1 | ||
kind: Namespace | ||
metadata: | ||
name: default | ||
annotations: | ||
iam.amazonaws.com/permitted: ".*" | ||
``` | ||
The default is not to allow any roles. You can use a regex as shown above to allow all roles or can even specify a particular role per namespace. | ||
|
||
## Deploying KIAM | ||
|
||
1. Deploy the KIAM server (this will run as a DS on all master nodes) | ||
``` | ||
kubectl apply -f kiam-server.yaml | ||
``` | ||
2. Deploy the KIAM agent | ||
``` | ||
kubectl apply -f kiam-agent.yaml | ||
``` | ||
|
||
## Testing Access | ||
|
||
1. Deploy test-pod | ||
``` | ||
kubectl apply -f test-deploy.yaml | ||
``` | ||
|
||
2. Exec into the pod and run | ||
``` | ||
curl 169.254.169.254/latest/meta-data/iam/security-credentials/ | ||
``` | ||
You should get `myrole` as the response. | ||
|
||
|
||
|
||
|
||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,39 @@ | ||
apiVersion: certmanager.k8s.io/v1alpha1 | ||
kind: ClusterIssuer | ||
metadata: | ||
name: kiam-ca-issuer | ||
namespace: kiam | ||
spec: | ||
ca: | ||
secretName: kiam-ca-key-pair | ||
--- | ||
apiVersion: certmanager.k8s.io/v1alpha1 | ||
kind: Certificate | ||
metadata: | ||
name: kiam-agent | ||
namespace: kiam | ||
spec: | ||
secretName: kiam-agent-tls | ||
issuerRef: | ||
name: kiam-ca-issuer | ||
kind: ClusterIssuer | ||
commonName: kiam | ||
--- | ||
apiVersion: certmanager.k8s.io/v1alpha1 | ||
kind: Certificate | ||
metadata: | ||
name: kiam-server | ||
namespace: kiam | ||
spec: | ||
secretName: kiam-server-tls | ||
issuerRef: | ||
name: kiam-ca-issuer | ||
kind: ClusterIssuer | ||
commonName: kiam | ||
dnsNames: | ||
- kiam-server | ||
- kiam-server:443 | ||
- localhost | ||
- localhost:443 | ||
- localhost:9610 | ||
--- |
Oops, something went wrong.