Log tailing

Retrieve logs for a service, job, or another resource with GET /v1/logs. Include the resource IDs as query parameters and use an API token with permission to read its logs.

The HTTP and WebSocket examples below use a team-scoped API token. With an organization-scoped token, use /v1/teams/{teamId}/logs and replace {teamId} with the target team ID.

Choose one target from this table. Do not combine target IDs from different rows.

Omit projectId for OpenTofu jobs and cluster pods. Container scan logs require permission to view scan reports. Scans from service builds also require permission to view service observability.

To read harness logs with an API token, the harness must use Team privacy. API tokens cannot access private harness logs, even for the creator. See harness privacy.

For addon backup logs, include backupId. For restore logs, include both backupId and restoreId. These IDs select backup or restore automatically. Otherwise, logSource defaults to runtime. An explicit logSource must match the selected IDs.

For build logs, set logSource=build. Add buildId to select a specific build. For a specific job run, include jobId and runId.

Retrieve a log range

HTTP requests use queryType=range by default.

Set NORTHFLANK_API_TOKEN to an API token with access to the target. Replace the project and service IDs with your own values. This example retrieves up to 100 service log lines that contain error from the last 10 minutes:

curl --get 'https://api.northflank.com/v1/logs' \
  --header "Authorization: Bearer $NORTHFLANK_API_TOKEN" \
  --data-urlencode 'projectId=default-project' \
  --data-urlencode 'serviceId=site-api' \
  --data-urlencode 'duration=600' \
  --data-urlencode 'lineLimit=100' \
  --data-urlencode 'textIncludes=error'

The response contains log entries in data and pagination information in pagination. Each entry includes containerId, log, and ts.

Use these filters to limit the results:

Stream live logs

To stream logs, open a WebSocket connection to wss://api.northflank.com/v1/logs with queryType=tail and the same target parameters. For example:

wss://api.northflank.com/v1/logs?projectId=default-project&serviceId=site-api&queryType=tail&lineLimit=10

After the connection opens, send this initialization message. Replace <API_TOKEN> with your token:

{
  "type": "init",
  "data": {
    "auth": {
      "type": "apiToken",
      "apiToken": "<API_TOKEN>"
    }
  }
}

A successful initialization returns an init message with data.auth set to successful. Log batches arrive in log-lines messages.

Tail requests default to 10 initial lines. Use startTime, duration, and text filters to limit the stream. endTime and direction apply to range requests only. An ordinary HTTP request with queryType=tail does not open a stream.

Resource-specific filters

For supported runtime targets, use podName or containerName to select instances. Repeat containerName to select several instances. podName takes precedence over containerName.

type filters log categories where supported. It does not replace logSource.

For build logs, OpenTofu jobs, and container scans, omit containerName or use all. These targets do not accept podName.

The Northflank Javascript client allows you to call methods, based on the resource and log type, which return a LogsClient object.

Methods

apiClient.get.service.buildLogTail({ parameters, options });
apiClient.get.service.logTail({ parameters, options });
apiClient.get.job.logTail({ parameters, options });
apiClient.get.job.buildLogTail({ parameters, options })
apiClient.get.addon.logTail({ parameters, options });

The logTail methods return logs from running deployment containers, and the buildLogTail methods return logs from build containers.

The method calls take two arguments, the required object parameters and an optional options object. The parameters object specifies the project and resource to return logs from, and the options object contains filters to return logs from specific containers or builds, time ranges, or log lines that match, or do not match, certain strings or regex patterns.

To fetch logs without streaming, use get.logs or get.buildLogs. For services, see get logs and get build logs.

LogsClient

Each method call returns a LogsClient, configured with the given parameters and options, which emits the events logs-received, error, open, and close, and exposes the async start and stop methods.

Logs are returned by the LogsClient batched in arrays. A batch is sent if it reaches 32.7kb or every second, whichever limit is reached first. Each log line in the array is an object which contains the following properties:

Example

Below is an example implementation to stream logs from a service, after initialising the JavaScript client. This example will return the most recent 5 log lines from a service, and wait until a log line containing END LOGS is received to close the connection.

const parameters = { projectId: "<my-project>", serviceId: "<deployment-service>" };
const options = { lineLimit: 5 };

// Create a new LogsClient object with the given parameters and options.
const logsClient = await apiClient.get.service.logTail({ parameters, options });

// On the 'logs-received' event process the log
logsClient.on('logs-received', async (logLines) => {
    // Print out each log line's details from the batch of logs received
    logLines.forEach((l) => {
        console.log(`Received log "${l.log}" from ${l.containerId} at ${l.ts.toTimeString()}`)
    });

    // Monitor for a specific string using regex and close the connection once received.
    const endLine = logLines.find(l => l.log.match(/END LOGS/));
    if (endLine) {
        await logsClient.stop();
    };
});

// Handle the error, open, and close events.
logsClient.on('error', (error) => console.log('error', error));

logsClient.on('open', () => console.log('open'));

logsClient.on('close', () => console.log('close'));

// Start the LogsClient to open a connection and begin receiving logs.
await logsClient.start();

The Northflank CLI can be used to open a log tailing session which prints logs continuously. You can add the -f or --tail flag to the standard get logs command for a resource to live tail logs.

You can open a session to live tail a deployment's logs using the following command:

northflank get service|job|addon logs -f

Similarly, you can live tail build logs using the following command:

northflank get service|job build-logs -f

The CLI command has the following options:

When you run the command with the -l or --lineLimit option, the most recent specified number of log lines from the given --startTime will be returned before live tailing begins.

Formatting

You can use the -o or --output flag to specify the format of the logs received in the console.

The format and json options will return logs in batches of JSON objects in an array. The format option returns printed JSON with highlights, while json returns pure JSON.

[ { containerId: 'deployment-service-734b0d6fe-x97pq',
log: '2023-04-13T08:18:13.944190216Z stdout F TEST',
ts: 2023-04-13T08:18:13.950Z },
{ containerId: 'deployment-service-734b0d6fe-x97pq',
log: '2023-04-13T08:18:14.570410388Z stdout F TEST',
ts: 2023-04-13T08:18:14.701Z },
{ containerId: 'deployment-service-734b0d6fe-x97pq',
log: '2023-04-13T08:18:15.039314832Z stdout F TEST',
ts: 2023-04-13T08:18:15.202Z }, ]
[ { containerId: 'deployment-service-734b0d6fe-x97pq',
    log: '2023-04-13T08:22:07.056818563Z stdout F TEST',
    ts: 2023-04-13T08:22:07.217Z } ]

The yaml option returns logs as plain YAML:

- containerId: deployment-service-734b0d6fe-x97pq
  log: '2023-04-13T08:24:57.405065103Z stdout F TEST'
  ts: 2023-04-13T08:24:57.644Z
- containerId: deployment-service-734b0d6fe-x97pq
  log: '2023-04-13T08:24:58.08643362Z stdout F TEST'
  ts: 2023-04-13T08:24:58.145Z
- containerId: deployment-service-734b0d6fe-x97pq
  log: '2023-04-13T08:24:58.852389498Z stdout F TEST'
  ts: 2023-04-13T08:24:58.895Z