Set up a preview environment

Northflank's preview environment feature facilitates the automatic provisioning of resources and deployment of builds when you push commits to Git branches.

You can configure a preview environment template for each of your pipelines, and define Git triggers to create new preview environments when you push a commit to specific pull requests and branches.

A new preview environment will be created when you push a commit to a branch that matches your trigger rules, or, if the branch already has a preview environment, the existing preview environment will be updated. You can also use a webhook to trigger a new preview environment, or create one manually.

Preview environments are defined using Northflank templates to provision resources and to build and deploy your commits. The simplest way to create your preview template is to copy the necessary resource specifications you require from your existing project resources and adapt them using references and arguments.

You can use trigger references, node references, and arguments to programmatically provision resources for your preview environment, and resources created by the preview environment are tagged for easy identification and automatic cleanup.

To create a new preview environment template open an existing pipeline in your project, or create a new one, then click the configure button in the preview environment section.

First, view the settings for the template and add at least one trigger.

You can automatically provision preview environments using Git triggers, or by using the webhook trigger. The values from the triggers can be used in your template to build and deploy the desired commit using build source nodes.

Configure a Git trigger

Git triggers will create a new preview environment using the template when a commit is pushed to a branch or pull request that matches the trigger. If a preview environment already exists for the branch or pull request, the template will re-run to update necessary resources, and subsequently build and deploy the new commit.

Click add trigger to create a new Git trigger. Enter a reference for your Git trigger, a string by which it can be accessed in the preview environment template, and select the repository you want to use.

You can now define pull request and branch rules. Branch rules will run the template when a commit is pushed to a branch matching the given rules. Pull request rules will run the template whenever a pull request is opened for a branch matching the given rules, or a commit is pushed to a branch matching the given rules with an open pull request.

You can also add path rules and ignore flags in order to only run the preview environment template on changes to specific files, or to skip runs when a certain commit message is included.

Configure a webhook trigger

You can also use a webhook trigger to create preview environments. Enable the webhook trigger and trigger it by making either a GET or POST request. You can include query parameters at the end of the webhook URL to pass values as arguments.

For example:

https://api-platform.northflank.com/webhooks/release-flows/<TOKEN>?branch=<BRANCH>&sha=<SHA>

will create a preview environment and pass the values of branch and sha as arguments to the preview environment template.

Reference triggers in your template

Git trigger references take the format ${refs.<git-trigger-name>.<key>} and can return the following values:

Webhook trigger query parameters are passed directly to your template as arguments.

For example, triggering a webhook endpoint with the query parameters branch and sha

https://api-platform.northflank.com/webhooks/release-flows/<TOKEN>?branch=<BRANCH>&sha=<SHA>

will make the values accessible in your template as ${args.branch} and ${args.sha}.

The build source node uses an existing build service to build a commit when it is triggered by the selected Git trigger. You can add multiple build source nodes to handle multiple repositories.

The build source node will use the branch and commit specified in the trigger

It has the following configurable settings:

An example of a build source node specification, which obtains the branch and sha to build from the Git trigger.

{
  "kind": "BuildSource",
  "ref": "build",
  "spec": {
    "type": "service",
    "id": "<build service>",
    "branch": "${refs.<git-trigger>.branch}",
    "sha": "${refs.<git-trigger>.sha}",
    "defaults": {
      "branch": "main"
    },
    "reuseExistingBuilds": true,
    "buildOverrides": {
      "buildArguments": {}
    }
  },
  "condition": "success"
}

When you run a preview environment manually you can override the branch and commit for each build source, if required.

Open the visual page and drag a build source node into the workflow, if you are using a git trigger. Enter a reference for the node so that you can easily deploy the build later in the template, then select the build service to use to build the commit.

Now select the trigger you configured previously, and choose a default branch from the repository specified by the trigger. You can also choose a default commit to build, otherwise it will use the latest commit. The defaults will be used if you have another trigger and build source configured in the template.

You can enable create new build to start a new build every time the template runs, otherwise the build source will use the existing build for a commit if one is already available.

Selecting wait for completion will ensure the build has finished before running the next node or workflow in the template.

You can add nodes for the resources required for your environment, such as addons, jobs, secret groups, and deployment services.

You can use references and arguments to programmatically generate names and deploy builds for your preview environment. The generated name for the preview environment can be accessed via args.previewId and can be used to name resources, for example database-${args.previewId}. Build IDs from build sources can be used to deploy builds to services and jobs, and are accessed using the reference given to a build source node, in the format ${refs.<build-source-reference>.id}. Nodes with a reference will be available as drop-down options for other nodes in your template, in the visual editor forms.

You can also use template functions in preview environment templates.

When you have finished creating your template, consisting of your build sources, required resources, and any other nodes you may require, click save preview template. Northflank will now create a preview environment for each branch that matches your Git triggers when they receive a new commit.

You can see an example preview environment template at the end of this section.

You can securely store secrets and inject them into your preview environment by adding them as arguments on the preview environment's settings page. Secrets stored as preview environment arguments are encrypted at rest and accessible in your template using the arguments object: ${args.<argument-name>}.

Share secrets

You can share secrets from a development or staging environment with your preview environment by using secret groups that are restricted by tag. Add the relevant tag to preview environment resources and they will also inherit secrets from the restricted group.

You can create a new secret group in your preview environment and link it to an existing addon to share connection details. The new secret group will be restricted to resources tagged with the preview environment ID by default.

Preview environments will also inherit from any unrestricted secret groups in your project.

Share resources

You can use existing resources in your preview environments, such as build services, databases, and jobs, by adding the relevant nodes to your preview environment template and selecting the resource from your project. Build source nodes use existing build services, you can run a job with overrides, or execute a command in a running service.

You can provide the connection details for an existing database via a secret group, so that your preview environment can use the database in your development environment, for example.

Alternatively you can back up and create a fork of an existing database, so that the original database in your permanent environment is unaffected by any changes (for example migrations) in the preview branch.

With your Git trigger and preview environment template configured, Northflank will automatically create new preview environments for branches that match your branch and pull request trigger rules. If you have configured a webhook trigger, a new preview environment will be created when you send a request to the webhook URL, and you can use the branch and sha query parameters to trigger a build for the desired commit.

You can pause triggers for a preview environment in the header of the preview environment settings. While paused, Git triggers will be inactive and requests to the webhook will return the HTTP status 202, with a message explaining the trigger is paused. Preview environments can still be created manually using the UI while triggers are paused.

When a new preview environment is created it will be assigned a randomly-generated name, and all resources created by the preview environment will be tagged with the environment name. You are unable to assign or manage tags created by a preview environment, and they will be deleted when the environment is deleted.

You can see a list of preview environments created in your pipeline, with the repository, branch, and status for each environment. Click through to a preview environment to see the status of the template run and resources associated with the environment.

Resources

Resources for a preview environment, including services, jobs, addons, and secret groups, will be added to the list as they are created by the template. You can view the status of resources created by the preview environment, pause and resume them, and click through to manage them where required.

View runs

You can view a list of template runs for the preview environment, and click through to see the status and results of specific nodes in the template. A new template run for a preview environment will be triggered if a new commit is pushed to the same branch.

Settings

You can delete your preview environment from the settings page.

Manually create a preview environment

You can manually create a new preview environment by clicking the create preview button in your pipeline. Manually creating a preview environment allows you to select the branch and commit to use for each build source. Click create to run the preview environment template with your selected branch.

CI/CD is disabled for any manually-created environments, meaning it will not be updated when commits are pushed to the previewed branch. You can update the preview environment by creating a new preview with the same branch, which will run the preview template again.

If the preview environment has been created by a pull request trigger, the environment will be deleted when the pull request is closed or merged. A preview environment created by a branch trigger will be deleted if the branch is deleted. You can manually delete a preview environment from its settings page.

Deleting a preview environment will remove all the resources generated by the template and remove the preview environment entirely from your pipeline. If a new commit is pushed that matches the Git trigger, a new preview environment will be created for the branch.

Preview environment tags will only be deleted when a preview environment is automatically or manually deleted.

This example preview environment template starts with a parallel workflow containing a build source node and a backup node, which triggers a snapshot of an existing addon. The existing addon ID is provided in the argument ${args.SOURCE_ADDON}.

The build source uses a Git trigger called feature which is configured to run when a pull request is created or updated for any branch in the given repository with feature/* in the name. The trigger is defined in the triggers array at the end of the template.

After the backup has completed a new addon is deployed as a fork of the existing addon, using the addon ID and backup ID as the source. A secret group is then created with connection details for the preview environment addon. These secrets are restricted to resources that have the preview environment tag.

Finally, a deployment service is deploying the build triggered by the build source node.

{
  "apiVersion": "v1.1",
  "spec": {
    "kind": "Workflow",
    "spec": {
      "type": "sequential",
      "steps": [
        {
          "kind": "Workflow",
          "spec": {
            "type": "parallel",
            "steps": [
              {
                "kind": "BuildSource",
                "spec": {
                  "type": "service",
                  "id": "builder",
                  "branch": "${refs.feature.branch}",
                  "sha": "${refs.feature.sha}",
                  "defaults": {
                    "branch": "main"
                  },
                  "reuseExistingBuilds": true,
                  "buildOverrides": {
                    "buildArguments": {}
                  }
                },
                "condition": "success",
                "ref": "build"
              },
              {
                "kind": "AddonBackup",
                "spec": {
                  "addonId": "${args.SOURCE_ADDON}",
                  "backupType": "snapshot"
                },
                "condition": "success",
                "ref": "backup"
              }
            ]
          }
        },
        {
          "kind": "Addon",
          "spec": {
            "tlsEnabled": true,
            "billing": {
              "replicas": 1,
              "storage": 4096,
              "storageClass": "ssd",
              "deploymentPlan": "nf-compute-50"
            },
            "typeSpecificSettings": {
              "postgresqlConnectionPoolerReplicas": 2,
              "postgresqlReadConnectionPoolerReplicas": 2
            },
            "tags": [
              "${args.previewId}"
            ],
            "type": "postgresql",
            "name": "${args.previewId}-database",
            "version": "15",
            "externalAccessEnabled": false,
            "ipPolicies": [],
            "pitrEnabled": false,
            "source": {
              "addonId": "${args.SOURCE_ADDON}",
              "backupId": "${refs.backup.id}"
            }
          },
          "ref": "database"
        },
        {
          "kind": "SecretGroup",
          "spec": {
            "type": "secret",
            "secretType": "environment-arguments",
            "priority": 10,
            "tags": [
              "${args.previewId}"
            ],
            "secrets": {
              "variables": {},
              "files": {}
            },
            "restrictions": {
              "restricted": true,
              "tags": [
                "${args.previewId}"
              ],
              "nfObjects": []
            },
            "addonDependencies": [
              {
                "addonId": "${refs.database.id}",
                "keys": [
                  {
                    "keyName": "HOST",
                    "aliases": []
                  },
                  {
                    "keyName": "POSTGRES_URI",
                    "aliases": []
                  },
                  {
                    "keyName": "USERNAME",
                    "aliases": []
                  },
                  {
                    "keyName": "PASSWORD",
                    "aliases": []
                  },
                  {
                    "keyName": "DATABASE",
                    "aliases": []
                  }
                ]
              }
            ],
            "name": "Database connection"
          }
        },
        {
          "kind": "DeploymentService",
          "spec": {
            "deployment": {
              "instances": 1,
              "storage": {
                "ephemeralStorage": {
                  "storageSize": 1024
                },
                "shmSize": 64
              },
              "docker": {
                "configType": "default"
              },
              "internal": {
                "id": "${refs.build.nfObjectId}",
                "branch": "${refs.build.branch}",
                "buildSHA": "${refs.build.sha}"
              }
            },
            "tags": [
              "${args.previewId}"
            ],
            "runtimeEnvironment": {},
            "runtimeFiles": {},
            "billing": {
              "deploymentPlan": "nf-compute-50"
            },
            "name": "${args.previewId}-deploy",
            "ports": []
          }
        }
      ]
    }
  },
  "triggers": [
    {
      "vcsService": "github",
      "repoUrl": "",
      "branchRestrictions": [],
      "prRestrictions": [
        "feature/*"
      ],
      "pathIgnoreRules": [],
      "isAllowList": false,
      "ciIgnoreFlagsEnabled": true,
      "ciIgnoreFlags": [
        "[skip ci]",
        "[ci skip]",
        "[no ci]",
        "[skip nf]",
        "[nf skip]",
        "[northflank skip]",
        "[skip northflank]"
      ],
      "ref": "feature",
      "manualOnly": false,
      "vcsLinkId": "",
      "id": ""
    }
  ],
  "arguments": {
    "SOURCE_ADDON": "development-database"
  }
}