Introducing Log Sinks

We are excited to release log sinks to general availability! Log sinks allow users to seamlessly integrate their preferred log aggregators and observability platforms with their Northflank builds and deployments.

Users can now send all their Northflank container logs to one or multiple providers for any project. This makes it possible to integrate with existing observability platform(s), build dashboards across multiple platforms and providers, and access the features of a preferred log service. It also improves the ability to analyse and visualise log metrics, perform searches, set up real-time alerts, and meet log auditing requirements for resources hosted on Northflank.

This new feature enables users to improve debugging with a comprehensive view of application performance across different services in their organisation's stack. In turn, this will make it easier to identify any issues that arise during builds or deployments. It will also provide a simpler way for teams to securely audit historical logs using their existing automation tools.

Ready to implement log sinks? You can view the complete documentation here.

Supported Integrations

• Datadog
• Loki
• Papertrail
• HTTP
• AWS S3
• Mezmo
• Logtail
• Honeycomb
• Logz.io

Features

• Log encoding - By default, logs will be sent as plain text. There’s also support for sending logs in a JSON format instead.
• Project restrictions - You can restrict logs from certain projects, giving you granular control over which logs reach your log sink.
• Custom label extraction - When you need to extract custom labels from a JSON message body, you can enable Custom Labels.
  • Supported for Logtail, Honeycomb, Logz.io and Datadog.
  • Your JSON messages will be parsed, and then key value pairs will be extracted and added as labels and message, or msg will be picked from the object and set as the message body that is then forwarded on to the observability provider.
  • For example: { message: 'hello', custom: 'field', status: 'my-status' }

To use the API, check out the documentation here. The following is an example of how to create a Datadog log sink via the API:

await apiClient.create.logSink({
 data: {
   'name': 'example-log-sink',
   'description': 'This is an example log sink.',
   'restricted': true,
   'useCustomLabels': true,
   'projects': ['default-project'],
   'sinkType': 'datadog_logs',
   'sinkData': {
     'default_api_key': 'abcdef12345678900000000000000000',
     'region': 'eu',
   },
 },
});

Additional new releases and updates are on the horizon for observability and alerting. To keep up with everything new on Northflank, follow us on LinkedIn or Twitter.