Which platforms support both CPU and GPU workloads in your own VPC?

Production AI applications rarely run entirely on GPUs. A GPU may handle training or inference, while CPU services authenticate requests, prepare data, coordinate jobs, and process results.

If those components must run in your own cloud, the platform needs to do more than provide GPU instances. It must operate distinct CPU and GPU workloads inside infrastructure you control, while giving your teams a practical way to deploy, scale, connect, observe, and govern them.

Three representative options are Northflank, Anyscale, and Red Hat OpenShift AI Self-Managed. This is not an exhaustive market list, and each platform serves a different requirement.

• Northflank runs CPU and GPU services and jobs through the same UI, CLI, API, CI/CD, and monitoring stack. Self-serve BYOC places workloads in your cloud account, while BYOK supports eligible existing Kubernetes infrastructure.
• Anyscale runs heterogeneous CPU and GPU Ray clusters in a customer data plane. On Kubernetes, its operator, Kubernetes, and Ray divide responsibility for provisioning pods, placing them on nodes, and scheduling application work.
• Red Hat OpenShift AI Self-Managed 3.5 adds AI development, training, and model-serving capabilities to supported OpenShift environments with CPU and accelerator resources. A customer-operated OpenShift deployment keeps the execution infrastructure under customer control.

Running CPU and GPU workloads in your own VPC means that distinct workload types execute in a cloud account, VPC or VNet, or cluster infrastructure your organisation owns or operates.

Azure uses VNet terminology, while self-managed platforms may run on private cloud or bare metal. Here, VPC is shorthand for customer-controlled infrastructure.

Four distinctions prevent a misleading comparison:

1. A GPU machine is not automatically a mixed-compute platform. Its attached CPUs do not prove support for independently scaled CPU and GPU workloads.
2. Private connectivity is not customer-cloud execution. Peering or PrivateLink can reach compute that remains in a vendor account.
3. Kubernetes does not prove infrastructure ownership. A Kubernetes option qualifies only when its worker nodes and workload network run in customer-controlled infrastructure.
4. Cloud infrastructure is not the complete platform layer. Hyperscalers supply instances, but your team still needs deployment, scheduling, release, and governance systems.

Ask where the workload runtime, data, and control plane operate. “BYOC” has no universal architecture.

A shared platform reduces the operational seams between an AI model and the application that uses it.

Consider a document-processing system. CPU workers validate inputs and prepare batches, a GPU service runs inference, and more CPU workers apply business rules and store the results. These components need to operate even when the GPU service scales independently.

One platform can give these components shared networking, secrets, identity, observability, and releases while retaining separate resource and scaling policies.

If your requirement includes the wider production stack, Northflank runs GPU workloads beside standard services, jobs, databases, storage, builds, preview environments, and release workflows. That is a different scope from a GPU endpoint or training service alone. The comparison of runtime platforms for AI applications and production infrastructure explains why that surrounding infrastructure affects the decision.

Compare the complete workload and responsibility boundary, not simply the list of available GPUs.

1. Platform scope: Decide whether you need a general application platform, a distributed AI-compute platform, or an AI layer for an existing Kubernetes estate.
2. Execution boundary: Confirm which CPU and GPU workers, storage systems, build artifacts, logs, and network paths remain in your environment.
3. Control-plane boundary: Identify where orchestration metadata, user activity, metrics, and administrative services live and who can access them.
4. Mixed-compute scheduling: Check whether CPU and GPU pools scale independently, whether jobs and long-running services are supported, and how the scheduler handles scarce GPU capacity.
5. Infrastructure coverage: Match supported clouds, regions, Kubernetes distributions, GPU models, drivers, and quota requirements to infrastructure you can actually procure.
6. Operational ownership: Assign responsibility for clusters, node pools, drivers, upgrades, capacity, networking, observability, recovery, and incident response.

BYOC, self-hosting, and managed cloud distribute platform and infrastructure work differently even when each can place workloads in an approved region.

These three representative platforms take different approaches to mixed compute in customer-controlled infrastructure.

The right choice depends on whether your primary unit of operation is an application, a Ray cluster, or an OpenShift estate.

Northflank fits teams that want one managed application platform for GPU workloads and the CPU services around them.

GPUs are treated as another workload resource. Developers use the same UI, CLI, API, Git integration, CI/CD, logs, and metrics for CPU and GPU services or jobs. An inference API, CPU workers, scheduled jobs, databases, volumes, and object storage can share the platform workflow.

Infrastructure placement is flexible:

• Northflank Cloud provides the managed-infrastructure option.
• Self-serve Northflank BYOC provisions and manages Kubernetes in your cloud account while you retain the account, network, IAM-policy, regional, and infrastructure-billing boundaries.
• Northflank BYOK connects eligible Kubernetes that your team already provisions and operates, including supported private and on-premises infrastructure.

Northflank's native BYOC integrations cover AWS, Google Cloud, Microsoft Azure, Oracle Cloud, CoreWeave, and Nebius. BYOK can also connect eligible Kubernetes clusters with GPU nodes. Available GPU models and capacity depend on the provider, region, quota, and cluster configuration; see Northflank's GPU infrastructure catalogue for current options.

Customer-cloud placement still requires a data-flow review. With Northflank BYOC, workload runtime and workload data reside in your environment, while the hosted control plane handles orchestration metadata. Confirm other data categories through Northflank security details.

Anyscale fits AI and data applications built around Ray. Its compute configurations support independently scaling CPU and GPU worker groups for workflows such as CPU preprocessing followed by GPU training or inference.

Ray clusters and workload resources run in the customer data plane, while Anyscale hosts the control plane. On Kubernetes, the Anyscale operator requests pods, Kubernetes places them, and Ray schedules tasks and actors on the available nodes. Application data and logs stay in the customer environment, but system logs, metrics, runtime metadata, and user activity are retained in the control plane.

Customer data planes can use AWS and Google Cloud VM deployments or customer-controlled Kubernetes. The Azure Native Integration uses AKS and is currently in Public Preview. Anyscale is therefore best suited to Ray-native distributed compute rather than a general application platform spanning web services, databases, and release workflows.

Red Hat OpenShift AI Self-Managed 3.5 fits organisations that already operate OpenShift or require a Kubernetes and AI platform under their own administration.

OpenShift Container Platform supplies the base application environment. OpenShift AI adds workbenches, pipelines, distributed training, and model serving. Resource queues cover CPU, memory, and GPU capacity, with accelerator support varying by workload and release.

A customer-operated deployment can span public cloud, private infrastructure, virtualization, and bare metal, with your organisation responsible for the OpenShift cluster and OpenShift AI components. Red Hat also supports the software on OpenShift Dedicated, ROSA, and Azure Red Hat OpenShift, where infrastructure and operational responsibilities differ.

That control brings more platform work. GPU nodes require compatible hardware, drivers, Node Feature Discovery, a GPU Operator, and cluster configuration. Support varies by accelerator, OpenShift version, and workload type.

OpenShift AI is most compelling when the organisation already has OpenShift skills, governance, and operations.

Select the platform whose operating model matches the system your team is actually building.

• Use Northflank when CPU and GPU components belong to a wider production application and you want one platform to manage services, jobs, databases, storage, builds, networking, releases, and observability in your cloud.
• Use Anyscale when the application is built around Ray and needs distributed CPU preprocessing, GPU training or inference, and Ray-native autoscaling in a customer data plane.
• Use Red Hat OpenShift AI Self-Managed 3.5 when your organisation already operates OpenShift, needs its hybrid infrastructure coverage, or deliberately wants to own the Kubernetes and AI-platform lifecycle.

Before procurement, test a representative workflow rather than one GPU container. Deploy its CPU workers, GPU service, storage, and observability path. Scale each pool, remove a GPU node, test network policy, rotate a secret, trace a request, and attribute spend.

If customer-cloud deployment is the deciding constraint, compare the broader BYOC options available for cloud infrastructure. If application security is the driver, use the controls in this guide to deploy AI-built applications securely in your own cloud.

No. A GPU cloud can provide accelerators in provider infrastructure. Running in your VPC requires the execution plane to operate in an account, network, or cluster your organisation controls.

Yes. Kubernetes can target separate CPU and GPU node pools with resource requests and scheduling controls. Whether one cluster is appropriate depends on tenancy, failure isolation, scaling, drivers, and network policy.

Not necessarily. Workload runtime and data may stay in your account while control-plane metadata, telemetry, or build services follow another path. Map each category before treating BYOC as a residency control.

It depends on the platform, provider, and deployment model. Customer-cloud platforms can often use infrastructure billed through your cloud account, but the GPU type, region, quota, reservation, node-pool configuration, and platform support must align. Confirm the exact capacity path before committing spend.

• Best runtime platforms for AI applications and production infrastructure
• BYOC vs self-hosting vs managed cloud
• Best options for BYOC in cloud computing
• How to deploy AI-built applications securely in your own cloud
• How to run AI workloads on cloud GPUs