Platform Eng Day Keynote: Make Workloads, not Infrastructure

You’ve built a platform, are currently building one, or recently re-platformed. Everyone in the infrastructure and cloud native space is working on an internal developer platform (IDP).

That’s how Will Stewart opened his keynote at Platform Engineering Day, an event co-located with KubeCon NA 2024. This six minute talk explores how developers often find themselves bogged down by configuring infrastructure and deploying clusters rather than focusing on their primary goal—delivering valuable workloads.

Overview

1. Cloud infrastructure is getting more complex.
2. Why devs need better abstractions than Terraform and YAML.
3. Examples of good abstractions in popular tools.
4. Why is self-service developer experience (DX) difficult to achieve?
5. Microservices and data storage are the essentials for a workload.
6. Unifying abstractions for logs, Disaster Recovery (DR), autoscaling, preview environments, and releases management.

About the Talk

Will Stewart is the Co-Founder & CEO of Northflank. He’s been building infrastructure since he was a teenager hosting gameservers on Mesos. This talk is born from his experience building Northflank, an app platform for Kubernetes.

Will showcases some abstractions that organise and reduce complexity well.

Can cloud complexity keep increasing forever? When building with the best cloud-native solutions today, the answer seems to be “yes.” This talk is Will’s perspective on what needs to be done to organise and reduce that complexity for app developers.

App developers work at the level of Go, .NET, or Next.js. They are experts in those languages and frameworks. Platform teams often get app developers to self-manage infrastructure through solutions like Terraform or YAML. This is almost always the wrong approach, because it is so far from the level developers are accustomed to working at.

Will points out that most of us are starting at infrastructure and building forward towards the app. From that starting point tools like Terraform and YAML are the natural choice. There is another way to work, however. What if we started at the app and worked back towards infrastructure?

He uses the metaphor that we’re spending all of our time building a factory without knowing what we want to actually produce. The unruly and seemingly ever-growing complexity is too much to expose developers to when all they want to do is deploy their workload. That workload is what creates your business value. Everything else should be oriented around supporting that. We don’t ask line workers to understand the ins and outs of factory logistics, and we don’t need to ask app developers to do that on their platforms, either.

Will closes the talk by noting that organising and reducing complexity means abstracting away the minutia needed to run an actual workload. A unified abstraction needs to encapsulate details like logging, metrics, disaster recovery (DR), autoscaling, preview environments, and release management.

Encapsulating the complexity like this means developers can work at the level where they’re most effective. They don’t have to be distracted and slowed down by the sprawling demands of cloud knowledge.

What is Northflank

Northflank is a unified abstraction layer for managing the multitudes of moving parts in your cloud. It is both a framework for building IDPs and an example of a polished developer experience on Kubernetes. You can see for yourself by deploying with Northflank today.