Kubernetes

Description

This plugin enables the ability to manage resources in Kubernetes in a Vela pipeline.

Source Code: https://github.com/go-vela/vela-kubernetes

Registry: https://hub.docker.com/r/target/vela-kubernetes

Usage

NOTE:

Users should refrain from using latest as the tag for the Docker image.

It is recommended to use a semantically versioned tag instead.

Sample of applying Kubernetes files:

steps:
  - name: kubernetes
    image: target/vela-kubernetes:latest
    pull: always
    parameters:
      action: apply
      files: [ kubernetes/common, kubernetes/dev/deploy.yml ]

Sample of pretending to apply Kubernetes files:

steps:
  - name: kubernetes
    image: target/vela-kubernetes:latest
    pull: always
    parameters:
      action: apply
+     dry_run: true
      files: [ kubernetes/common, kubernetes/dev/deploy.yml ]

Sample of patching containers in Kubernetes files:

steps:
  - name: kubernetes
    image: target/vela-kubernetes:latest
    pull: always
    parameters:
      action: patch
      files: [ kubernetes/common, kubernetes/dev/deploy.yml ]
      containers:
        - name: sample
          image: alpine:latest

Sample of pretending to patch containers in Kubernetes files:

steps:
  - name: kubernetes
    image: target/vela-kubernetes:latest
    pull: always
    parameters:
      action: patch
+     dry_run: true
      files: [ kubernetes/common, kubernetes/dev/deploy.yml ]
      containers:
        - name: sample
          image: alpine:latest

Sample of watching the status of resources:

steps:
  - name: kubernetes
    image: target/vela-kubernetes:latest
    pull: always
    parameters:
      action: status
      statuses: [ sample ]

GKE clusters authentication

Google Kubernetes Engine (GKE) clusters version 1.26 and up require use of new "gke-gcloud-auth-plugin", see blog.

Download Google service account credentials:

gcloud beta secrets versions access 1 --secret [Secret Name] --project [Google Project Name] --out-file [Secret Name]-gsa-key.json

Upload Google service account secret to vela native secrets store:

vela add secret --secret.engine native --secret.type org --org MYORGNAME --name k8s-gsa-key --value @k8s-gsa-key.json -event deployment --event pull_request --event push --event tag --event comment

Create kubeconfig file and make sure to use user gke-gcloud-auth-user for your cluster:

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: XXXXXXXXXXX
    server: https://ip of K8s API Server
  name: clustername
contexts:
- context:
    cluster: clustername
    user: gke-gcloud-auth-user
  name: clustername

current-context: ""
kind: Config
preferences: {}
users:
- name: gke-gcloud-auth-user
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1beta1
      args:
      - --use_application_default_credentials
      command: gke-gcloud-auth-plugin
      env: null
      installHint: Install gke-gcloud-auth-plugin for use with kubectl by following
        https://cloud.google.com/blog/products/containers-kubernetes/kubectl-auth-changes-in-gke
      interactiveMode: IfAvailable
      provideClusterInfo: true

Upload kubeconfig file to vela native secretrs store:

vela add secret --secret.engine native --secret.type org --org MYORGNAME --name kube_config_secret --value @kube_config_secret.file -event deployment --event pull_request --event push --event tag --event comment

Configure Vela file:

secrets:
  - name: kube_config_secret
    key: ORGNAME/kube-config
    type: org
  - name: gsa_key
    key: ORGNAME/k8s-gsa-key
    type: org

  k8s-gsa-setup:
    steps:
    - name: setup-gke-access
      image: alpine:latest
      ruleset:
        branch: master
        event: [pull_request, push, deployment]
      pull: always
      environment:
        GOOGLE_APPLICATION_CREDENTIALS: "/vela/secrets/.kube/gsa-key.json" # <== change name as needed
      secrets:
        - source: gsa_key
          target: GSA_KEY_FILE
      commands:
        - mkdir -p /vela/secrets/.kube/
        - echo "$GSA_KEY_FILE" > $GOOGLE_APPLICATION_CREDENTIALS

  k8s-plugin-dry-run:
    needs: [ k8s-gsa-setup ]
    steps:
    - name: dry-run
      image: target/vela-kubernetes:latest
      ruleset:
        branch: master
        event: [ pull_request ]
      environment:
        GOOGLE_APPLICATION_CREDENTIALS: "/vela/secrets/.kube/gsa-key.json" # <== change name as needed
        USE_GKE_GCLOUD_AUTH_PLUGIN: True # <== required
      secrets:
        - source: kube_config_secret
          target: kube_config
      parameters:
        action: apply
        dry_run: true
        context: context-name   # <== change as needed
        files: [ Kubernetes/manifests/service.yaml ]

  k8s-plugin-apply:
    needs: [ k8s-gsa-setup ]
    steps:
    - name: run-apply
      image: target/vela-kubernetes:latest
      ruleset:
        branch: master
        event: [ push ]
      environment:
        GOOGLE_APPLICATION_CREDENTIALS: "/vela/secrets/.kube/gsa-key.json" # <== change name as needed
        USE_GKE_GCLOUD_AUTH_PLUGIN: True # <== required
      secrets:
        - source: kube_config_secret
          target: kube_config
      parameters:
        action: apply
        context: context-name   # <== change as needed
        files: [ Kubernetes/manifests/service.yaml ]

Secrets

NOTE: Users should refrain from configuring sensitive information in your pipeline in plain text.

Internal

Users can use Vela internal secrets to substitute these sensitive values at runtime:

steps:
  - name: kubernetes
    image: target/vela-kubernetes:latest
    pull: always
+   secrets: [ kube_config ]
    parameters:
      action: apply
      files: [ kubernetes/common, kubernetes/dev/deploy.yml ]
-     config: |
-     ---
-     apiVersion: v1
-     kind: Config

This example will add the secrets to the kubernetes step as environment variables:

KUBE_CONFIG=<value>

External

The plugin accepts the following files for authentication:

Parameter	Volume Configuration
`config`	`/vela/parameters/kubernetes/config`, `/vela/secrets/kubernetes/config`

Users can use Vela external secrets to substitute these sensitive values at runtime:

steps:
  - name: kubernetes
    image: target/vela-kubernetes:latest
    pull: always
+   secrets: [ kube_config ]
    parameters:
      action: apply
      files: [ kubernetes/common, kubernetes/dev/deploy.yml ]
-     config: |
-     ---
-     apiVersion: v1
-     kind: Config

This example will read the secret values in the volume stored at /vela/secrets/

Parameters

NOTE:

The plugin supports reading all parameters via environment variables or files.

Any values set from a file take precedence over values set from the environment.

The following parameters are used to configure the image:

Name	Description	Required	Default	Environment Variables
`action`	action to perform against Kubernetes	`true`	`N/A`	`PARAMETER_ACTION` `KUBERNETES_ACTION`
`cluster`	Kubernetes cluster from the configuration file	`false`	`N/A`	`PARAMETER_CLUSTER` `KUBERNETES_CLUSTER`
`context`	Kubernetes context from the configuration file	`false`	`N/A`	`PARAMETER_CONTEXT` `KUBERNETES_CONTEXT`
`config`	content of configuration file for communication with Kubernetes	`true`	`N/A`	`PARAMETER_CONFIG` `KUBERNETES_CONFIG` `KUBE_CONFIG`
`log_level`	set the log level for the plugin	`true`	`info`	`PARAMETER_LOG_LEVEL` `KUBERNETES_LOG_LEVEL`
`namespace`	Kubernetes namespace from the configuration file	`false`	`N/A`	`PARAMETER_NAMESPACE` `KUBERNETES_NAMESPACE`
`path`	path to configuration file for communication with Kubernetes	`false`	`N/A`	`PARAMETER_PATH` `KUBERNETES_PATH`
`version`	version of the `kubectl` CLI to install	`false`	`v1.24.0`	`PARAMETER_VERSION` `KUBERNETES_VERSION`

Config

The content passed to the config parameter is expected to be a full Kubernetes Config object as would be found in ~/.kube/config. Run kubectl config view to get an idea of what could be in this config file.

Consider this a minimal example of step with inline config:

name: k8s apply
image: target/vela-kubernetes:latest
ruleset:
	event: push
	branch: [main]
secrets: [K8S_TOKEN]
parameters:
	action: apply
	files:
  # These may not be necessary, but explicitness is good practice.
	cluster: mycluster
	context: mycontext
	namespace: mynamespace
		- k8s/myapp.yaml
	config: |
		---
		apiVersion: v1
		kind: Config
		clusters:
			- name: mycluster
			  cluster:
					server: https://kubernetes.example.com:6443
		users:
			- name: myuser
				user:
					token: "${K8S_TOKEN}"
		contexts:
			- name: mycontext
			  context:
					cluster: mycluster
					namespace: mynamespace
					user: myuser
		current-context: mycontext

Note the interpolation of the K8S_TOKEN secret. Doing this can ease management of large secrets with multiple "sub-secrets" even when they don't change very often.

Actions

These next sections define the parameters that must or can be passed when a particular action is active.

Apply

The following parameters are used to configure the apply action:

Name	Description	Required	Default	Environment Variables
`dry_run`	enables pretending to perform the apply (`true`/`client`, `false`/`none`, `server`)	`false`	`false`	`PARAMETER_DRY_RUN` `KUBERNETES_DRY_RUN`
`files`	list of Kubernetes files or directories to apply	`true`	`N/A`	`PARAMETER_FILES` `KUBERNETES_FILES`
`output`	set the output for the apply	`false`	`N/A`	`PARAMETER_OUTPUT` `KUBERNETES_OUTPUT`

Patch

The following parameters are used to configure the patch action:

Name	Description	Required	Default	Environment Variables
`containers`	containers from the files to patch	`true`	`N/A`	`PARAMETER_CONTAINERS` `KUBERNETES_CONTAINERS`
`dry_run`	enables pretending to perform the patch	`false`	`false`	`PARAMETER_DRY_RUN` `KUBERNETES_DRY_RUN`
`files`	list of Kubernetes files or directories to patch	`true`	`N/A`	`PARAMETER_FILES` `KUBERNETES_FILES`
`output`	set the output for the patch	`false`	`N/A`	`PARAMETER_OUTPUT` `KUBERNETES_OUTPUT`

Status

The following parameters are used to configure the status action:

Name	Description	Required	Default	Environment Variables
`statuses`	list of Kubernetes resources to watch status on	`true`	`N/A`	`PARAMETER_STATUSES` `KUBERNETES_STATUSES`
`timeout`	total time allowed to watch Kubernetes resources	`true`	`5m`	`PARAMETER_TIMEOUT` `KUBERNETES_TIMEOUT`
`watch`	enables watching until the resource completes	`false`	`true`	`PARAMETER_WATCH` `KUBERNETES_WATCH`

Template

COMING SOON!

Troubleshooting

You can start troubleshooting this plugin by tuning the level of logs being displayed:

steps:
  - name: kubernetes
    image: target/vela-kubernetes:latest
    pull: always
    parameters:
      action: apply
      files: [ kubernetes/common, kubernetes/dev/deploy.yml ]
+     log_level: trace

Below are a list of common problems and how to solve them:

Description​

Usage​

GKE clusters authentication​

Secrets​

Internal​

External​

Parameters​

Config​

Actions​

Apply​

Patch​

Status​

Template​

Troubleshooting​