Pipelines

Pipelines on Northflank allow you to create and manage complex Continuous Integration and Continuous Delivery workflows. Essentially, Pipelines manage the process of building, deploying and promoting images in different stages. Together with the configuration options of Build and Deployment services, Pipelines offer an incredible amount of flexibility to support your development process requirements.

If you are new to Pipelines on Northflank we recommend reading the Pipeline basics section.

We will learn how to create, manage, and configure Pipelines in this section, covering:

  • Basic Pipeline concepts
  • Creating a Pipeline
  • Adding Deployment services and linking Build services
  • Using stages and promoting builds
  • Changing linked builds, moving Deployment services, editing stages and workflows

Pipeline Basics

You should be familiar with the following concepts before creating or using Pipelines:

Build and Deployment services

A Deployment Service deploys images from a Build Service or an external registry. It can only be present in one Pipeline at a time. It will display this Pipeline in the service overview.

A Build Service (automatically) builds images from Git repositories. It can be used in conjunction with multiple Deployment Services. These appear in the respective Build Service overview.

Linking repositories and Deploying images

There are several ways to provide images to a Deployment Service. You can:

  • link a branch or pull request from a Build Service
  • select a built image to deploy from a specific commit in a Build Service
  • take an image directly from an external container registry

Linking a branch or pull request means that your Deployment service will use the latest build from that branch or pull request, provided you have configured the build rules to monitor it and that build latest is enabled.

Selecting a built image means your Deployment service will deploy that specific build from a commit and will not update automatically.

Stages and Promotion

By default, you can place Deployment services in three stages: Development, Staging, and Production.

You can link a Deployment Service in one stage with another service in the next stage. This allows you to then promote image builds to the next stage.

Creating a Pipeline

To create a Pipeline:

  1. Select your Project
  2. Choose Pipeline from the Create new drop-down menu in the top right corner of the dashboard
  3. Enter a Name and Description for your Pipeline
  4. Click Create Pipeline

Using a Pipeline

In this section we cover the steps to complete specific tasks within a Pipeline.

To create an effective Pipeline you will need to have at least one Build service and multiple Deployment services.

Adding Deployment services

If you have not yet created any Deployment services, or you have used all your Deployment services in another Pipeline, you will need to create a new Deployment service.

  1. Select your Project
  2. Open the Pipelines overview and select the Pipeline you want to use
  3. If the Pipeline sidebar is not open, click the button on the right of the screen to show it
  4. Click and drag the Deployment service from the Pipeline sidebar to the stage you want to add it to

If you have a lot of Deployment services you can use the search bar to find the one you want.

Linking to a branch or pull request

If you do not have any Build services, or you want to build from a new repository, you will need to create a new Build service.

In order to link a branch or pull request, you need to have added at least one Deployment service to the Pipeline.

  1. Select your Project
  2. Open the Pipelines overview and select the Pipeline you want to use
  3. If the Pipeline sidebar is not open, click the button on the right of the screen to show it
  4. Find the Build service you want to deploy from and click on the Deploy button
  5. Find the branch or pull request you wish to link and click on the Link button
  6. Choose the Deployment service(s) you want to use in the Pipeline and select Link service
  7. Click Done or close the overlay when finished or go back to select another branch or pull request to link

If you have a lot of Build services you can use the search bar to find the one you want.

Deploying a build

You need to have added at least one Deployment service to your Pipeline to deploy a build.

  1. Select your Project
  2. Open the Pipelines overview and select the Pipeline you want to use
  3. If the Pipeline sidebar is not open, click the button on the right of the screen to show it
  4. Find the Build service you want to deploy from and click on the Deploy button
  5. Find the branch or pull request you wish to deploy a build from and click on the Deploy button
    • If you do not have any builds for that branch or pull request, you will need to start a new build
  6. Click Select build on the build you want to deploy and click Deploy to service on the Deployment service(s) you want to deploy the build to
  7. Click Done or close the overlay when finished or go back to select another build to link

Linking Stages and Promoting builds

Stages

You can manage your Pipeline's stages by adding Deployment services to different stages.

You can move a Deployment service to a different stage by clicking the options button (three vertical dots) on the Deployment service in the Pipeline and selecting the Move to <stage> option for the stage you want to move it to.

Linking Deployment services

To link Deployment services navigate to the Pipeline you want to edit:

  1. Click the Link service button on the first Deployment service in the chain you want to create (for example if you want to link a service in Development to a service in Staging, click the button on the service in Development)
  2. Your selected Deployment service will now be highlighted in green
  3. Click on the Deployment service(s) in the next stage you want to link to
  4. Click again on the Link service button on the first Deployment service you were linking from to exit the link editor for that service (clicking anywhere except a Deployment service that can be linked/unlinked to your selected will also exit the editor)

You must have at least two Deployment services added to different, but sequential, stages. You cannot link services across multiple stages, links can only be made between adjacent stages.

Unlinking Deployment services

To unlink Deployment services navigate to the Pipeline you want to edit:

  1. Click the Link service button on the first Deployment service in the chain you want to unlink (for example if you have a service in Staging linked to a service in Production, click the button on the service in Staging)
  2. Your selected Deployment service will now be highlighted in green
  3. Click the Unlink service button (a red x) on the Deployment service(s) you want to unlink
  4. Click again on the Link service button on the first Deployment service you were unlinking to exit the link editor for that service (clicking anywhere except a Deployment service that can be linked/unlinked to your selected will also exit the editor)

You can remove all links from Deployment service by clicking the options button (three vertical dots) on the Deployment service in the Pipeline and selecting Unlink all.

Promoting builds

You can promote a build from a Deployment service in one stage to a linked Deployment service in a subsequent stage by clicking the Promote button on the first service in the link.

This will replace the currently deployed build on the linked Deployment service.

Deploying the latest build

Toggling the Deploy latest setting on a Deployment service in a Pipeline will switch between deploying a specific build or linking to a branch or pull request.

If deploy latest is enabled, the Deployment service will use a successful build from the most recent commit to the branch or pull request.

If deploy latest is disabled, the Deployment service will use the specific build deployed to it, or the last build automatically deployed when Deploy latest was disabled.

Removing Deployment services

To remove a Deployment service from a Pipeline click the options button (three vertical dots) on the service you want to remove and select Remove.

This will not unlink its Build service or external deployment, nor pause the Deployment service. You can edit or disable the Deployment service itself from its own dashboard.

Deleting a Pipeline

You can delete a Pipeline at any time, even if it has services added to it, by clicking Delete on the Pipelines list.

This will remove the Pipeline but it will not affect the Deployment services nor the linked Build services in any way.

You can edit or disable the services itself from their respective dashboards. See Build services and Deployment services for more details.

© 2020 Northflank Ltd. All rights reserved.

TermsPrivacy

contact@northflank.com