In this blog we are going to talk about Kubernetes (Tekton) pipelines, and how they have revolutionised the way CI/CD workflows are managed in software development, allowing greater flexibility in their creation and management.
The main features of pipelines with Tekton are:
- It is a native open source framework.
- It is a powerful and flexible model for building CI/CD systems, allowing developers to build, test and deploy across cloud providers and on-premises systems.
- Provides extensions to custom resource definitions (CRD) to facilitate pipeline creation and standardisation. Has built-in support for coupling with existing industry CI/CD tools such as Jenkins, Jenkins X, Skaffold, Knative and OpenShift.
- • Tekton’s OpenShift integration (OpenShift Pipelines) introduces even more power and flexibility to this system, through the RedHat and OpenShift developer tools.
What are the benefits?
Customisableble
Reusable
Expandable
Standardised
Scalable
- Customisable: They are fully customisable, allowing a high degree of flexibility.
- Reusable: Being fully portable, once defined, developers can quickly build complex pipelines without “reinventing the wheel”.
- Expandable: Tekton Catalog is a community-driven repository of Tekton’s core components. You can quickly create new pipelines and extend existing ones using pre-built components from the catalogue.
- Standardised: It installs and runs as an extension on Kubernetes and uses the well-established resource model. Workloads run inside Kubernetes containers.
- Scalable: To increase your workload capacity, you can simply add nodes to the cluster. Tekton scales without the need to redefine your resource allocations or any other modifications to your pipelines.
How to work with Tekton
Tkn is the command line interface for interacting with Tekton. It provides a fast and streamlined experience, including high-level commands and colour coding.
Kubectl provides substantially greater granularity for controlling Tekton, at the expense of greater complexity. Mostly used for debugging pipelines and troubleshooting your builds.
Tekton APIs, currently available for Pipelines and Triggers, allow you to interact programmatically with the components.
Tekton pipelines
It is a Kubernetes extension that is installed and runs on the cluster. It defines a set of custom resources that act as building blocks from which CI/CD pipelines can be assembled.
It is defined with the following entities:
Task: It is a collection of steps that are defined and organised in a specific order of execution, as part of the continuous integration flow. Each of the steps is a container within the Task Pod. Isolating this type of sequence of related steps into a single reusable task gives Tekton great versatility and flexibility.
Example:
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: echo-example
namespace: tekton-pipelines
spec:
params:
- name: message
type: string
default: ‘Hello World’
steps:
- name: hello
image: ubuntu
command: ["echo"]
args: ["$(params.message)"]
volumes:
- name: example-volume
emptyDir: {}
TaskRun: Instantiate a task in the cluster by executing the steps of the task in the specified order, until all have been successfully executed or a failure occurs.
Example of how to instantiate the above task:
apiVersion: tekton.dev/v1beta1
kind: TaskRun
metadata:
generateName: echo-example-
namespace: tekton-pipelines
spec:
params:
- name: message
vault: ‘Goodbye world’
taskref:
name: echo-example
kind: Task
volumes:
- name: example-volume
emptyDir: {}
Pipeline: It is a collection of tasks in order. Tekton collects all the tasks, connects them into a directed acyclic graph (DAG) and executes the graph in sequence. In other words, it creates a number of Kubernetes pods and ensures that each pod executes correctly as desired.
Tekton gives developers full control of the process: one can set up an input/output task completion scenario, ask Tekton to automatically retry if there is an unstable test, or specify a condition that a task must meet before continuing.
Example:
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
name: pipeline-example
namespace: tekton-pipelines
spec:
params:
- name: context
type: string
description: Path to context
default: /some/where
tasks:
- name: example-build
taskRef:
name: build-push
params:
- name: pathToContext
value: "$(params.context)"
PipelineRun: is a specific execution of a pipeline. For example, you can ask Tekton to run a CI/CD workflow twice a day, and each run will become a trackable PipelineRun resource in your Kubernetes cluster.
You can visualise the status of your CI/CD workflow, including specific details of each task execution with PipelineRun.
Example for instantiating the above Pipeline:
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
generateName: pipeline-example-
namespace: tekton-pipelines
spec:
pipelineRef:
name: pipeline-example
params:
- name: context
value: "/tmp/example"
Workspaces: allow Tasks to declare parts of the filesystem that need to be provided at runtime by TaskRuns, TaskRuns can make those parts of the filesystem available in many ways: using a read-only ConfigMap or Secret, an existing PersistentVolumeClaim shared with other Tasks, creating a PersistentVolumeClaim from a provided VolumeClaimTemplate, or simply an emptyDir that is discarded when the TaskRun terminates.
Workspaces are similar to Volumes except that they allow the author of a task to defer to users and their TaskRuns in deciding what kind of storage to use.
Frequent use cases:
- Storage of inputs and/or outputs
- Sharing data between Tasks
- A mount point for credentials stored in Secrets
- A mount point for configurations saved in ConfigMaps
- A mounting point for common tools shared by an organisation
- A cache of construction artefacts that speed up the job
Here we see a diagram where the above entities are integrated:
Tekton Triggers
It is a component that can detect and extract event information from various sources and deterministically instantiate and execute TaskRuns and PipelineRuns based on that information. It can also send extracted event information directly to TaskRuns and PipelineRuns.
It consists of a controller service running on a Kubernetes cluster, as well as the following Kubernetes custom resource definitions (CRDs) that extend the functionality of Tekton Pipelines to support events:
EventListeners: is a Kubernetes object that listens for events on a specified port in its cluster. It exposes an addressable sink that receives the incoming event and specifies one or more Triggers. The sink is a Kubernetes service that runs logic inside a dedicated Pod.
Example:
apiVersion: triggers.tekton.dev/v1beta1
kind: EventListener
metadata:
name: eventlistener
spec:
triggers:
- name: trigger
serviceAccountName: trigger-sa
interceptors:
- github:
eventTypes: ["pull_request"]
bindings:
- ref: pipeline-binding
template:
ref: pipeline-template
Triggers: Specifies what happens when the EventListener detects an event. Specifies a TriggerTemplate, a TriggerBinding and, optionally, an Interceptor.
Example:
apiVersion: triggers.tekton.dev/v1beta1
kind: Trigger
metadata:
name: trigger
spec:
interceptors:
- ref:
name: "cel"
params:
- name: "filter"
value: "header.match('X-GitHub-Event', 'pull_request')"
- name: "overlays"
value:
- key: extensions.truncated_sha
expression: "body.pull_request.head.sha.truncate(7)"
bindings:
- ref: pipeline-binding
template:
ref: pipeline-template
TriggerBindings: Specifies the fields in the event payload from which you want to extract data and the fields in its corresponding TriggerTemplate to populate with the extracted values. You can then use the populated fields in the TriggerTemplate to populate the fields in the associated TaskRun or PipelineRun.
Example:
apiVersion: triggers.tekton.dev/v1beta1
kind: TriggerBinding
metadata:
name: pipeline-binding
spec:
params:
- name: gitrevision
value: $(body.head_commit.id)
- name: gitrepositoryurl
value: $(body.repository.url)
- name: contenttype
value: $(header.Content-Type)
TriggerTemplate: specifies a template for the resource, such as a TaskRun or PipelineRun, that you want to instantiate and/or execute when your EventListener detects an event. It exposes parameters that you can use anywhere within your resource template.
Example:
apiVersion: triggers.tekton.dev/v1beta1
kind: TriggerTemplate
metadata:
name: pipeline-template
spec:
params:
- name: gitrevision
description: The git revision
default: main
- name: gitrepositoryurl
description: The git repository url
- name: message
description: The message to print
default: This is the default message
- name: contenttype
description: The Content-Type of the event
resourcetemplates:
- apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
generateName: simple-pipeline-run-
spec:
pipelineRef:
name: simple-pipeline
params:
- name: message
value: $(tt.params.message)
- name: contenttype
value: $(tt.params.contenttype)
- name: git-revision
value: $(tt.params.gitrevision)
- name: git-url
value: $(tt.params.gitrepositoryurl)
workspaces:
- name: git-source
emptyDir: {}
Interceptors: It is a platform-specific “wildcard” event processor that runs before the TriggerBinding and allows for payload filtering, verification (via a secret), transformation, defining and testing trigger conditions, and other useful processing. Once the event data passes through an interceptor, it goes to the Trigger before you pass the payload data to the TriggerBinding.
Tekton Triggers supports the following interceptors to help you get started:
- Webhook Interceptors
- GitHub Interceptors
- GitLab Interceptors
- Bitbucket Interceptors
- Bitbucket Server
- Bitbucket Cloud
- CEL Interceptors
Example:
interceptors:
- name: "validate GitHub payload and filter on eventType"
ref:
name: "github"
params:
- name: "secretRef"
value:
secretName: github-secret
secretKey: secretToken
- name: "eventTypes"
value: ["pull_request"]
- name: "CEL filter: only when PRs are opened"
ref:
name: "cel"
params:
- name: "filter"
value: "body.action in ['opened', 'reopened']"
Below, we can see a diagram where the resources are related:
What can I do with Triggers?
As an example, you can implement the following CI/CD workflow:
- Triggers is listening for a git commit or git pull request event. When it detects one, it runs a Pipeline unit test on the committed code.
- Triggers waits for a git push event to indicate that the test has completed successfully. When it detects one, it validates the test result and runs a Pipeline that builds the tested code.
- 3. When the associated PipelineRun completes execution, Triggers checks the result of the build, and if successful, runs a task that uploads the build artifacts to the Docker registry of your choice.
- Finally, the Docker registry sends an event to Pub/Sub, which triggers a Pipeline that sends the build artifacts to a staging environment.
Tekton Dashboard
It is a web interface for Tekton Pipelines and Tekton Triggers resources. It allows users to manage and view the creation, execution and completion of resources.
Some of the features supported by the Tekton Dashboard are:
- Real-time view of PipelineRun and TaskRun status and logs
- Filter resources by tag
- View the resource summary and YAML
- Display resources for the entire cluster or limit visibility to a specific namespace.
- ● Importing resources directly from a git repository
- Adding functions through extensions
Conclusion
More and more companies are deciding to containerise and deploy their applications on Kubernetes services (AKS, EKS, GKE, etc…) and their development teams need to be agile using CI/CD tools, where they can automate through pipelines the process of analysis, building, testing and deployment of new versions.
Tekton pipelines use Kubernetes clusters as their first-class type and containers as their main building blocks, executing tasks in isolation, unaffected by other processes running on the same system.
Tekton strongly promotes loose coupling, so the opportunity for reuse arises directly. That is, work from one project can be collected, chained and used elsewhere. This facilitates the deployment of services across multiple cloud solutions provided by different vendors or even on-premises systems.
