Skip to content

Commit

Permalink
add config sketch for setting up vector on minikube
Browse files Browse the repository at this point in the history
  • Loading branch information
gmega committed Jan 7, 2025
1 parent 48a1d1a commit d417f55
Show file tree
Hide file tree
Showing 3 changed files with 65 additions and 0 deletions.
2 changes: 2 additions & 0 deletions k8s/local/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Configurations which can be applied to a local k8s cluster like Minikube or Kind so that experiment logs can be collected for later parsing. This is a simple version of what one would do in a real cluster: use a log collector and ship everything onto external storage or a service like Logstash.

25 changes: 25 additions & 0 deletions k8s/local/codex-workflow-logs-pv.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# We need a persistent volume for Vector to collec the logs, which will then be accessible
# to the log parser.

apiVersion: v1
kind: PersistentVolume
metadata:
name: codex-workflow-logs-pv
spec:
capacity:
storage: 10Gi
accessModes:
- ReadWriteMany
hostPath:
path: /mnt/codex-workflow-logs
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: codex-workflow-logs-pvc
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 10Gi
38 changes: 38 additions & 0 deletions k8s/local/vector-agents-values.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
role: "Agent"
service:
enabled: false

extraVolumes:
- name: vector-logs
persistentVolumeClaim:
claimName: codex-workflow-logs-pvc

extraVolumeMounts:
- name: vector-logs
mountPath: /vector-logs

customConfig:
data_dir: /var/lib/vector
sources:
benchmark-experiments-source:
type: kubernetes_logs
extra_label_selector: "app.kubernetes.io/name=codex-benchmarks"
node_annotation_fields:
node_labels: ""
pod_annotation_fields:
container_id: ""
container_image: ""
container_image_id: ""
pod_node_name: ""
pod_owner: ""
pod_uid: ""
pod_ip: ""
pod_ips: ""

sinks:
output:
type: file
inputs: [benchmark-experiments-source]
path: "/vector-logs/benchmarks-%Y-%m-%d.jsonl"
encoding:
codec: json

0 comments on commit d417f55

Please sign in to comment.