Kubernetes
Prerequisites
This section provides all required dependencies to install and start the worker with Kubernetes.
Dependency 1: Kubernetes
Kubernetes will be used for downloading the worker and managing the lifecycle of the application.
You can refer to Kubernetes’ official documentation on installing and configuring the service.
Dependency 2: Redis
Redis will be used for storing workloads, created by the server, that will be run by a worker.
You can refer to Redis’s official documentation on installing and configuring the service.
Installation
This section provides an example of installing the worker with Kubernetes.
This example only shows a subset of all possible configuration options.
Step 1: Create a Worker Secret and ConfigMap
You will need to store some env vars in a Secret, and the rest can go in a ConfigMap.
Note:
Determine which worker auth method to use for worker-server communication before writing theConfigMap. Details of the two offerings can be found here.apiVersion: v1
kind: Secret
metadata:
name: vela-worker
namespace: default
data:
# these values are base64 encoded
# this value should only be specified if using the server-worker trusted symmetric token auth method.
VELA_SERVER_SECRET: PHNoYXJlZC1zZWNyZXQ+
# VELA_SERVER_SECRET: <shared-secret>
Do not store any passwords in the ConfigMap. The ConfigMap is more convenient for everything else.
apiVersion: v1
kind: ConfigMap
metadata:
name: vela-worker
namespace: default
data:
# This might be "http://vela-server:8080" if vela-server is also deployed in k8s.
VELA_SERVER_ADDR: https://vela-server.example.com
VELA_QUEUE_DRIVER: redis
VELA_EXECUTOR_DRIVER: linux
VELA_RUNTIME_DRIVER: kubernetes
VELA_RUNTIME_NAMESPACE: default
VELA_RUNTIME_PODS_TEMPLATE_NAME: pipeline-pods-template
# do not define VELA_WORKER_ADDR here. See "Create a Worker Deployment" below.
# VELA_RUNTIME_CONFIG is not needed in-cluster.
# We'll use the auto-mounted ServiceAccount Token.
And then load them in your cluster.
$ kubectl apply -f worker-secret.yaml
$ kubectl apply -f worker-configmap.yaml
Note:
For a full list of configuration options, please see the worker reference.Step 2: Load the Pipeline Pods Template CRD
Download Vela’s “Pipeline Pods Template” Custom Resource Definition (CRD).
Be sure to replace v0.14.0 with the version you’re installing.
$ curl https://raw.githubusercontent.com/go-vela/worker/v0.14.0/runtime/kubernetes/generated/go-vela.github.io_pipelinepodstemplates.yaml -o go-vela.github.io_pipelinepodstemplates.yaml
And then add the CRD to your cluster.
$ kubectl apply -f go-vela.github.io_pipelinepodstemplates.yaml
Step 3: Create a Pipeline Pods Template
The Pipeline Pods Template allows you to define annotations, securityContext, dnsConfig
and other settings that the worker should add to every Pipeline’s Pod.
apiVersion: go-vela.github.io/v1alpha1
kind: PipelinePodsTemplate
metadata:
name: pipeline-pods-template # this should match VELA_RUNTIME_PODS_TEMPLATE_NAME
namespace: default
spec:
# spec.template is a subset of what is possible to define in a Deployment's "spec.template".
template:
metadata:
# custom annotations to add to all pipeline pods
annotations:
example.com/owner: devops
#labels:
#spec:
# dnsConfig: {}
# dnsPolicy: ClusterFirst
# nodeSelector: {}
# tolerations: []
# affinity: {}
# # These gets applied to all containers in the Pipeline Pod.
# container:
# securityContext:
# capabilities:
# drop: ["ALL"]
# add: []
And then your PipelinePodsTemplate to your cluster.
$ kubectl apply -f pipeline-pods-template.yaml
Step 5: Create a ServiceAccount, Role, and RoleBinding
The Worker needs access to Kubernetes APIs. It uses an auto-mounted ServiceAcount token to do this.
Here is a ServiceAccount:
apiVersion: v1
kind: ServiceAccount
metadata:
name: vela-worker
namespace: default
Here’s the Role you’ll need:
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: vela-worker
namespace: default
rules:
- apiGroups: [""]
resources: [pods/log]
verbs: [get, list, watch]
- apiGroups: [""]
resources: [pods]
verbs: [create, patch, get, list, update, watch, delete]
- apiGroups: ["go-vela.github.io"]
resources: [pipelinepodstemplate]
verbs: [get, list, watch]
And use this RoleBinding:
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: vela-worker
namespace: default
subjects:
- kind: ServiceAccount
name: vela-worker
namespace: default
roleRef:
kind: Role
name: vela-worker
apiGroup: rbac.authorization.k8s.io
Then apply each of these to your cluster:
$ kubectl apply -f worker-serviceaccount.yaml
$ kubectl apply -f worker-role.yaml
$ kubectl apply -f worker-role-binding.yaml
Step 5: Create a Worker Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
name: vela-worker
namespace: default
labels:
app.kubernetes.io/name: vela-worker
spec:
replicas: 1 # Increase this to deploy more worker pods.
selector:
matchLabels:
app.kubernetes.io/name: vela-worker
serviceAccount: vela-worker
containers:
- name: worker
image: target/vela-worker:v0.13.0
ports:
- {name: http, port: 8080, protocol: TCP}
livenessProbe:
httpGet: {path: /health, port: 8080, scheme: HTTP}
env:
- {name: VELA_WORKER_ADDR_SCHEME, value: http}
- {name: VELA_WORKER_ADDR_PORT, value: "8080"}
- name: VELA_WORKER_POD_NAME
valueFrom:
fieldRef: {fieldPath: metadata.name}
- name: VELA_WORKER_POD_IP
valueFrom:
fieldRef: {fieldPath: status.podIP}
# using the pod name does not get a dns entry without a lot of unnecessary effort.
# So, here we use status.podIP instead of metadata.name
- name: VELA_WORKER_ADDR
value: $(VELA_WORKER_ADDR_SCHEME)://$(WORKER_POD_IP):$(VELA_WORKER_ADDR_PORT)
envFrom:
- configMapRef:
name: vela-worker
- secretRef:
name: vela-worker
And then load it in your cluster.
$ kubectl apply -f worker-deployment.yaml
Step 6: Verify the Worker Deployment
Ensure the worker started up successfully and is running as expected by inspecting details with kubectl.
$ kubectl describe deployment vela-worker
$ kubectl get pods -l app.kubernetes.io/name=vela-worker
You can also check the worker logs with stern. The following command will tail all of the logs
for pods that start with vela-worker-:
$ stern vela-worker-