Send logs to a HTTP endpoint from Northflank

You can create a HTTP sink to POST either text or JSON encoded log lines over HTTP. You send logs to your own custom system, or a not-yet-supported log collection system.

Is your log aggregator platform not on Northflank yet? Let us know.

For the this guide, I created a simple HTTP Node.js server to output the text or JSON body of incoming requests, which you can view later in the guide.

Your endpoint will receive logs from Northflank in either text or JSON format, and you can secure your endpoint by only accepting requests with either basic authentication or a bearer token.

Text logs will be sent with the header Content-Type: text/plain and the body will consist of a string containing the log in the format:

<timestamp: [yyyy-MM-dd’T’HH:mm:ss.SSSSSSSSS’Z’]> <stream [stdout | stderr]> <log line>.

JSON logs will be sent with the header Content-Type: application/json and the body will consist of a JSON object. JSON logs are sent as an array of objects, with each object representing a single log entry:

{
    [
      {
        "entity":"<entity_id>",
        "host":"Northflank",
        "message":"<datetime with timezone> <log message>",
        "path":"/",
        "pod":"<service_name>-<pod_id>",
        "project":"<project_name>",
        "service":"<service_name>",
        "source":"Northflank",
        "source_type":"http",
        "timestamp":"<datetime with timezone>",
        "type":"<service|job|addon>"
      },
      {...},
      {...}
    ]
  }

Northflank support two types of HTTP authentication. For security your endpoint should require HTTPS, otherwise credentials and logs will be transmitted unencrypted.

Basic authentication: basic authentication requires a password, and optionally a username. This will be encoded using base64 before being sent in the header as Authorization: "Basic <encoded credentials>".

Bearer token: your bearer token will be encoded using base64 and sent in the header of the request as Authorization: "Bearer <encoded token>".

You can create a sink that doesn't require authentication, but this should only be used for testing purposes.

Your endpoint should return a 2xx response code.

You can look at our documentation for more information on what to expect from a HTTP sink.

If you want to create a test endpoint you can create a simple Node.js HTTP server. Copy the code below into a file and either create a deployment on Northflank, or run it locally. If you want to run it locally, you'll need to use a forwarding service such as ngrok. This endpoint does not require authentication.

const http = require('http');

const port = '4011';

const logRequestListener = function (req, res) {

    // listen for logs received as plain text or JSON and print them to console
    // then send a 200 (OK) response

    req.on('data', (data) => {
        const contentType = req.headers['content-type'];
        if (contentType === 'text/plain') {
            console.log(data.toString());
        } else if (contentType === 'application/json') {
            console.log(JSON.parse(data));
        }
    });

    res.writeHead(200, { 'Content-Type': 'text/plain' });
    res.end();
}

const logServer = http.createServer(logRequestListener);

logServer.listen(port);

console.log('Listening for incoming logs on port:', port);

Enter a name, supply your endpoint, and select your authentication strategy, Enter the username (optional) and password for basic authentication, or supply the secret for bearer token.

You can target specific projects to forward logs from, and only containers running in the selected projects will have their logs forwarded.

Add the log sink and Northflank will send a log line to your endpoint to verify the credentials are correct. This appears as a log line consisting of validating log sink credentials.

After receiving a successful response, Northflank will begin sending logs from your resources to your endpoint.

Have questions or feedback? Contact us. We also have dedicated support plans for business customers available.