Log tailing

You can stream live logs from your services, jobs, and addons using the log tailing feature in the Northflank JavaScript client and the Northflank CLI. Both methods use WebSockets to stream the requested logs from your resources.

You can receive logs from containers for deployed services and addons, or build logs from builds. Much like viewing logs in the Northflank application, you can live tail all logs or filter them based on various parameters.

To fetch logs without live tailing, you can use the standard get.logs and get.buildLogs methods, for example get logs and get build logs for services.

When you start a live log tailing session it will by default return the 10 most recent log lines from the previous 30 minutes. You can change these parameters to include more log lines from the previous period, or retrieve logs from an earlier start time. After retrieving the previous logs, the session will then begin returning log lines live, as they are generated.

The Northflank Javascript client allows you to call methods, based on the resource and log type, which return a LogsClient object.

Methods

apiClient.get.service.buildLogTail({ parameters, options });
apiClient.get.service.logTail({ parameters, options });
apiClient.get.job.logTail({ parameters, options });
apiClient.get.job.buildLogTail({ parameters, options })
apiClient.get.addon.logTail({ parameters, options });

The logTail methods return logs from running deployment containers, and the buildLogTail methods return logs from build containers.

The method calls take two arguments, the required object parameters and an optional options object. The parameters object specifies the project and resource to return logs from, and the options object contains filters to return logs from specific containers or builds, time ranges, or log lines that match, or do not match, certain strings or regex patterns.

LogsClient

Each method call returns a LogsClient, configured with the given parameters and options, which emits the events logs-received, error, open, and close, and exposes the async start and stop methods.

Logs are returned by the LogsClient batched in arrays. A batch is sent if it reaches 32.7kb or every second, whichever limit is reached first. Each log line in the array is an object which contains the following properties:

Example

Below is an example implementation to stream logs from a service, after initialising the JavaScript client. This example will return the most recent 5 log lines from a service, and wait until a log line containing END LOGS is received to close the connection.

const parameters = { projectId: "<my-project>", serviceId: "<deployment-service>" };
const options = { lineLimit: 5 };

// Create a new LogsClient object with the given parameters and options.
const logsClient = await apiClient.get.service.logTail({ parameters, options });

// On the 'logs-received' event process the log
logsClient.on('logs-received', async (logLines) => {
    // Print out each log line's details from the batch of logs received
    logLines.forEach((l) => {
        console.log(`Received log "${l.log}" from ${l.containerId} at ${l.ts.toTimeString()}`)
    });

    // Monitor for a specific string using regex and close the connection once received.
    const endLine = logLines.find(l => l.log.match(/END LOGS/));
    if (endLine) {
        await logsClient.stop();
    };
});

// Handle the error, open, and close events.
logsClient.on('error', (error) => console.log('error', error));

logsClient.on('open', () => console.log('open'));

logsClient.on('close', () => console.log('close'));

// Start the LogsClient to open a connection and begin receiving logs.
await logsClient.start();

The Northflank CLI can be used to open a log tailing session which prints logs continuously. You can add the -f or --tail flag to the standard get logs command for a resource to live tail logs.

You can open a session to live tail a deployment's logs using the following command:

northflank get service|job|addon logs -f

Similarly, you can live tail build logs using the following command:

northflank get service|job build-logs -f

The CLI command has the following options:

When you run the command with the -l or --lineLimit option, the most recent specified number of log lines from the given --startTime will be returned before live tailing begins.

Formatting

You can use the -o or --output flag to specify the format of the logs received in the console.

The format and json options will return logs in batches of JSON objects in an array. The format option returns printed JSON with highlights, while json returns pure JSON.

[ { containerId: 'deployment-service-734b0d6fe-x97pq',
log: '2023-04-13T08:18:13.944190216Z stdout F TEST',
ts: 2023-04-13T08:18:13.950Z },
{ containerId: 'deployment-service-734b0d6fe-x97pq',
log: '2023-04-13T08:18:14.570410388Z stdout F TEST',
ts: 2023-04-13T08:18:14.701Z },
{ containerId: 'deployment-service-734b0d6fe-x97pq',
log: '2023-04-13T08:18:15.039314832Z stdout F TEST',
ts: 2023-04-13T08:18:15.202Z }, ]
[ { containerId: 'deployment-service-734b0d6fe-x97pq',
    log: '2023-04-13T08:22:07.056818563Z stdout F TEST',
    ts: 2023-04-13T08:22:07.217Z } ]

The yaml option returns logs as plain YAML:

- containerId: deployment-service-734b0d6fe-x97pq
  log: '2023-04-13T08:24:57.405065103Z stdout F TEST'
  ts: 2023-04-13T08:24:57.644Z
- containerId: deployment-service-734b0d6fe-x97pq
  log: '2023-04-13T08:24:58.08643362Z stdout F TEST'
  ts: 2023-04-13T08:24:58.145Z
- containerId: deployment-service-734b0d6fe-x97pq
  log: '2023-04-13T08:24:58.852389498Z stdout F TEST'
  ts: 2023-04-13T08:24:58.895Z