How to self-host Kimi K3 on AWS

Kimi K3 is a large open-weight model designed for reasoning, coding, long-context tasks, and AI agents. For teams that want to run Kimi K3 themselves, AWS provides the infrastructure needed to keep the model and its supporting services inside an environment they control.

Self-hosting a model at this scale is different from deploying a typical application. You need substantial GPU capacity, distributed inference, networking, storage, and a platform for managing the workloads around the model. For enterprises, this means either building and operating that infrastructure themselves or using a platform that can provide the management layer within their own cloud environment.

This is a high-level guide to self-hosting Kimi K3 on AWS, with links to detailed documentation and hands-on guides for the implementation.

Using the Kimi API is the simplest way to access Kimi K3, but some enterprises need more control over where their AI workloads run. Running Kimi K3 inside your own AWS environment means the model can operate alongside your existing applications, databases, and private services instead of sending inference workloads to an external hosted environment.

Self-hosting can also make sense when you have existing AWS infrastructure or committed cloud spend, need to meet internal data and security requirements, or want greater control over the model serving environment. The tradeoff is that you take on the infrastructure required to run a model of Kimi K3's scale.

Kimi K3 is open-weight, but enterprises should review its license and commercial-use conditions before deploying it in production to ensure their intended use complies with the applicable terms.

Kimi K3 is a 2.8-trillion-parameter sparse Mixture-of-Experts model with a 1-million-token context window. Its model weights are around 1.56 TB in the current repository, making it a substantial distributed GPU workload rather than a typical single-GPU deployment. 

That means a production deployment needs more than GPU capacity. You also need high-performance networking between accelerators, persistent storage for model weights, an inference engine such as vLLM, and the infrastructure to deploy, monitor, secure, and operate the resulting service.

Northflank is a runtime platform for AI-native companies deploying and managing applications, infrastructure, and AI workloads. It can run on Northflank's managed cloud, inside your own cloud environment through Bring Your Own Cloud (BYOC), or as a forward-deployed control plane.

With BYOC, Northflank deploys and manages your workloads within your AWS account and VPC while you retain control of the underlying infrastructure. This means you can use your own AWS compute, networking, storage, and GPU capacity while managing your workloads through the Northflank platform.

Northflank provides the infrastructure and developer tooling needed to run AI workloads and the applications around them, including GPU compute, secure sandboxes, managed databases, persistent storage, preview environments, CI/CD, networking, observability, and enterprise security controls.

For Kimi K3, this gives enterprises a way to combine the control of self-hosting with a managed platform experience. Instead of building and operating the entire application platform around a distributed GPU workload yourself, Northflank provides the tooling for deploying, securing, and managing Kimi K3 alongside the rest of your application infrastructure.

Running Kimi K3 directly on AWS gives you control over the underlying infrastructure, but it also means managing the platform around the model. Northflank provides that platform layer while allowing the Kimi K3 workload to remain inside your AWS environment.

With BYOC, Kimi K3 runs inside your AWS account and VPC. You retain control over the AWS region, networking, compute, and data, while Northflank provides the platform for deploying and managing the workload.

You can also use your existing AWS infrastructure and committed cloud spend rather than creating a separate infrastructure relationship. Northflank does not add a markup to the underlying AWS compute costs.

Kimi K3 requires substantial GPU capacity and distributed infrastructure. Operating that directly means dealing with Kubernetes, GPU configuration, networking, deployments, scaling, and monitoring in addition to running the model itself.

Northflank provides a consistent platform for managing these workloads in your AWS environment. Your team can provision infrastructure, deploy services, manage workloads, and monitor deployments without having to build and maintain a separate internal platform around Kimi K3.

Kimi K3 is unlikely to be the only component of a production AI application. You may also need APIs, databases, background workers, queues, object storage, scheduled jobs, and other AI services.

Northflank lets these workloads run through the same platform as Kimi K3. This gives your team one place to deploy and operate the model and the application infrastructure it depends on, rather than managing each component through separate infrastructure tooling.

Self-hosting gives you control over the underlying AWS environment, but you still need controls around the applications and workloads running inside it.

Northflank provides enterprise capabilities including RBAC, SSO, secrets management, audit logging, and network controls. Combined with BYOC, this allows enterprises to keep Kimi K3 within their own cloud environment while applying consistent security and governance controls around the model and the applications that interact with it.

This is particularly important when Kimi K3 is used by AI agents that access internal data, call private services, or take actions using enterprise credentials.

Security starts with where the workload runs. With BYOC, the runtime environment and data remain within your own cloud boundary, giving your organization control over the AWS account, region, networking, and underlying infrastructure.

Northflank adds security controls at the platform layer. These include RBAC for controlling who can access and manage workloads, SSO for integrating with enterprise identity providers, secrets management for protecting credentials, and audit logs for tracking platform activity. Network configuration can also be managed within your cloud environment.

For AI workloads, this separation is useful because the model may interact with sensitive applications, databases, credentials, and internal APIs. Instead of treating Kimi K3 as an isolated model endpoint, you can apply the same security and infrastructure controls around it as you do around the rest of your production environment.

You do not need to build an AI infrastructure platform from scratch to self-host Kimi K3 on AWS.

The high-level deployment process is:

1. Create a Northflank account
2. Connect your AWS account to Northflank.
3. Create a BYOC cluster in the AWS region where you want to run Kimi K3.
4. Provision the required GPU infrastructure for your Kimi K3 deployment.
5. Deploy the Kimi K3 inference workload using a supported inference engine such as vLLM.
6. Connect your applications and AI agents to the Kimi K3 inference endpoint.
7. Manage the workload through Northflank alongside your other services and infrastructure.

Yes. Northflank BYOC provisions and manages the Kubernetes cluster inside your AWS account, allowing your Kimi K3 workloads and supporting infrastructure to run within your own cloud environment.

With BYOC, the Kimi K3 workload runs within your AWS environment. Northflank provides the platform layer for managing the workload, while your AWS account remains responsible for the underlying infrastructure and data.

Northflank BYOC provisions and manages the Kubernetes cluster in your cloud account, reducing the amount of cluster management your team needs to handle directly. You still control the underlying AWS environment and its configuration.

Yes, depending on your setup. Northflank supports BYOC for AWS and can also manage existing Kubernetes clusters through BYOK, allowing organizations to bring existing infrastructure into the platform.

Yes. Northflank is designed to run application services, databases, jobs, GPUs, and other workloads through the same platform, so Kimi K3 can be part of a broader AI application architecture rather than operating as an isolated model server.

Self-hosting Kimi K3 on AWS gives enterprises control over where the model runs, how it connects to internal infrastructure, and where inference data is processed. But the scale of Kimi K3 also makes the underlying infrastructure more complex than a typical AI application, particularly when you need distributed GPUs and production-grade networking and operations.

Northflank BYOC provides a middle ground between a fully managed AI service and building the entire platform yourself. Kimi K3 runs inside your AWS environment while Northflank provides the platform layer for deploying, securing, and managing the workload and the applications around it.

• Kimi K3: benchmarks, pricing, hardware requirements, and self-hosting
• Self-host vLLM in your own cloud account with Northflank BYOC
• What is BYOC in cloud computing?
• Bring your own cloud app hosting: how BYOC works and when to use it