How to deploy an AI agent from sandbox to production
An AI agent uses a model to plan work and call tools with varying autonomy. A successful sandbox run proves it can complete a task, not that your enterprise has a secure, governed production service.
Production adds controls for shared ownership: scoped access, isolation, repeatable releases, data boundaries, audit evidence, cost limits, and incident response.
This guide explains how to deploy an AI agent from sandbox to production, covering requirements, secure runtime design, testing, release, and operations. It also examines when to build the platform internally and how Northflank runs the agent and supporting stack.
A practical path packages and tests the orchestrator, then promotes it through a controlled release.
- Define the agent's inputs, outputs, tools, identity, state, limits, and failure behavior before choosing production infrastructure.
- Keep sandbox isolation in the production architecture when the agent executes generated, user-supplied, or otherwise untrusted code.
- Version the code, prompts, tool schemas, policy, and model configuration that determine behavior.
- Test task quality, permissions, failure handling, security, and load against the full stack, not only the model response.
- Release progressively, monitor agent and infrastructure behavior together, and prepare separate recovery for code, data, memory, and external side effects.
- Decide whether to build and maintain these controls or adopt a platform that applies them across sandboxes and the production stack.
If your organization needs to move agents into production without integrating separate tools for sandboxing, Kubernetes, CI/CD, databases, secrets, and observability, Northflank provides one deployment and governance model for agent sandboxes and the surrounding application stack. Platform teams can give developers a self-service path while retaining control over infrastructure placement, access, releases, and audit evidence.
Run on Northflank Cloud or use self-serve BYOC when private networking, regional placement, existing cloud commitments, or workload data boundaries make customer-controlled infrastructure important. Northflank Skills lets supported coding agents operate the stack through the API and CLI within assigned permissions.
Get started with Northflank self-serve, or book an architecture review to map your agent runtime, security boundary, data flows, cloud placement, and production release requirements.
Moving an AI agent to production adds a controlled service boundary around variable outputs and execution paths. It does not mean removing the sandbox, an isolated environment for code and commands.
During development, one sandbox may contain the agent, tools, code, temporary files, and developer credentials.
The agent orchestrator authenticates callers, applies policy, invokes the model, and coordinates tools. When required by the threat model, it isolates generated code, shell commands, unfamiliar dependencies, or unsafe file-processing tools. Durable records should typically live elsewhere.
This gives platform and security teams control over scaling, quotas, network policy, credentials, and termination.
An agent production contract defines the interface, authority, resources, and failure behavior the deployed system must enforce. Define it before packaging so each requirement is testable.
| Decision | Define | Why it affects production |
|---|---|---|
| Interface | Request, event, and response schemas | Enables validation, versioning, and retries |
| Identity | Who invokes the agent and which identity it uses | Determines authorization and attribution |
| Tools | Allowed operations, targets, and parameter rules | Sets the agent's maximum authority |
| State | Durable records, memory, workspace, and retention | Reduces the risk of temporary state becoming a system of record |
| Limits | Time, steps, tokens, cost, concurrency, and resources | Bounds runaway tasks and capacity demand |
| Failure behavior | Retry, compensate, escalate, or stop | Reduces duplicate or partially completed actions |
| Service objectives | Quality, latency, availability, and cost targets | Creates a production-readiness gate |
Keep tool authorization outside model discretion. The model can propose an action, but deterministic controls in the application, policy engine, IAM layer, gateway, or tool server should validate the caller, operation, target, parameters, environment, and approval. Use task-specific credentials.
Set thresholds.
Package the agent as a reproducible, immutable deployment artifact: typically a container image that is not changed after testing. Keep environment-specific configuration and secrets outside it.
Version behavior-changing components, including prompts, policy, tool schemas, model and retrieval configuration, and migrations. Record relevant versions with each run.
Promote the same artifact across staging and production where practical. Separate components that scale or release independently.
If your team wants a direct path from Git to a running agent, a Northflank combined service builds and deploys from a linked branch. When the same tested artifact must move through staging and production, separate the build from deployment so each environment references the exact image that passed evaluation.
Test the agent system because a good response does not prove identity, tools, state, and failure handling work safely.
- Deterministic tests: Validate schemas, policy, tools, retries, idempotency, and state transitions.
- Task evaluations: Run representative and edge-case tasks against a versioned dataset. Score completion, correctness, groundedness, and escalation where applicable.
- Security tests: Exercise prompt injection, malicious tools, permission escalation, approval bypass, memory poisoning, exfiltration, and sandbox escape attempts.
- Resilience tests: Inject model timeouts, tool failures, partial writes, queue redelivery, and outages. Check retries for duplicate actions.
- Load tests: Measure latency, errors, queue depth, sandbox starts, model quotas, token use, and cost under expected demand.
Where practical, test with representative networks, secret scopes, queues, data, and isolation. A preview tests a change temporarily; staging is a persistent pre-production stage.
If reviewers need to evaluate more than the agent API, Northflank preview blueprints create an isolated copy of the supporting stack from Git, a webhook, or an API call. The preview can include services, workers, jobs, databases, and test URLs, giving each change a production-like environment without making it permanent.
A production AI agent architecture typically separates request handling, orchestration, risky execution, durable state, and operational control. This request path is a practical reference, not a universal standard.
Here, the entry point authenticates the caller, validates the request, applies limits, and assigns a correlation ID linking events from one run. Asynchronous work can return a durable run ID and enter a queue.
Use durable idempotency or deduplication for retried operations. Consequential side effects may also require conditional writes, transactions, an outbox, downstream idempotency, or reconciliation.
If the entry point needs to run continuously or scale with request volume, Northflank deployment services can run the API or worker with configurable ports, health checks, and horizontal autoscaling.
The orchestrator manages model calls, context, limits, and tool selection. A separate policy layer checks each action against the caller, agent version, resource, and environment.
Set budgets for time, steps, tokens, tool calls, and cost. Independently authorize high-impact actions.
For the infrastructure side of this boundary, Northflank can inject secrets into selected workloads, connect the orchestrator to tools over private networking, and scope platform API access through assigned roles. Your application still decides which agent action is permitted.
Run generated code, shell commands, untrusted repositories, and unsafe file-processing tools outside the orchestrator. Give the environment only the resources, credentials, and network access required.
If the production agent must run code it did not author or trust, Northflank Sandboxes give the orchestrator an SDK-controlled execution boundary. The service can create a sandbox, execute commands, attach storage when state is required, then pause or destroy the environment at the end of the task.
Business records commonly belong in databases, artifacts in object storage, and pending work in queues. Agent memory retained across steps or sessions should have an owner, retention period, access policy, and deletion path.
Export required artifacts or attach narrowly scoped storage instead of persisting an entire sandbox. This makes cleanup predictable.
When the agent needs durable state, Northflank supports stateful workloads alongside the service, including PostgreSQL, MySQL, MongoDB, Redis, RabbitMQ, MinIO object storage, and persistent volumes. This keeps databases, queues, and artifacts separate from disposable execution sessions.
Connect agent traces to infrastructure events with the run ID. Depending on impact, record identity, versions, policy decisions, tools, approvals, resources, state changes, latency, token use, and cost. Capture content only where policy permits.
For agents capable of consequential actions, operators need independent controls to terminate runs, revoke credentials, disable tools, block requests, restrict egress (outbound network traffic), and quarantine state.
Northflank supplies service logs and metrics, health checks, alerts, and platform audit logs for the infrastructure layer. Correlate these signals with application-level agent and tool traces using the same run identifier.
A common release strategy promotes a tested agent version through progressively more realistic and consequential environments. Adapt this path to the workload:
- Build one versioned artifact from a reviewed commit.
- Evaluate it in a full-stack preview, then deploy the same artifact to staging.
- Compare it with the current version, then promote through a bounded rollout.
- Expand while acceptance criteria hold; stop or restore the previous artifact when they fail.
If your team needs a repeatable route between stages, Northflank Environments can model development, staging, and production while workflows automate builds, deployments, backups, migrations, and artifact promotion. Readiness checks keep traffic away from a new service until it can accept work.
When a release fails, Northflank workflows can roll back to previous deployments. Teams can inspect current and previous workflow runs, review the status of individual nodes, and rerun a previous workflow with the same structure and arguments.
Operate the agent as both an AI system and a software service. Uptime can remain healthy while task quality falls.
Track task success, policy denials, errors, latency, queues, sandbox startup, token use, cost, and user impact. Segment by agent version, model, tool, task, and environment.
Add failures to evaluations; check fixes against other tasks.
When the agent is live, Northflank production operations expose logs, metrics, notifications, health checks, and horizontal autoscaling for its services. These infrastructure signals help operators diagnose availability and capacity while application evaluations measure whether the agent still performs its task correctly.
Building a production agent platform internally means owning the infrastructure that takes agents from isolated development into governed production: sandbox runtimes, Kubernetes, identity integrations, release automation, databases, observability, upgrades, and incident response. Northflank gives platform teams one control plane for these layers while preserving cloud and architecture choices.
See how Northflank capabilities map to the requirements in this deployment path:
- An approved route to production: Git-based builds, deployment services, jobs, and Northflank Sandboxes give developers one delivery path for the agent service, workers, and untrusted execution. Environments and workflows promote tested artifacts through development, staging, and production with health checks, migrations, and rollback.
- Production-like evidence before merge: Preview environments reproduce the agent and supporting stack so reviewers can evaluate the complete change.
- One operating model with infrastructure choice: Stateful workloads, private services, workers, and GPU workloads can run alongside the agent. Northflank Cloud provides managed infrastructure, while Northflank BYOC supports customer-controlled cloud resources, networking, regional placement, and workload data paths. Northflank Enterprise also supports forward-deployed control-plane options.
- Enterprise access and governance: Secret injection, private networking, SAML or OIDC SSO, MFA enforcement, RBAC, scoped API roles, and platform audit logs provide infrastructure controls and evidence that complement application-level agent traces.
- Compliance support: Northflank is SOC 2 Type 2 compliant and HIPAA compliant, with Business Associate Agreements supported under Enterprise contracts.
- Agent-native operations under assigned access: Northflank Skills gives supported coding agents guidance for operating services, jobs, databases, previews, sandboxes, GPUs, and releases through the API and CLI. Assigned credentials and permissions remain the access boundary.
Versaia runs its agent orchestration platform on Northflank for enterprise, healthcare, and public-sector customers. After migrating in under two weeks, Versaia increased voice-agent throughput from 4–5 to 15 concurrent calls per node and cut compute costs by about 60%.
Get started with Northflank self-serve, or book an architecture review to map your sandbox boundary, identity, data flows, cloud placement, and releases.
These answers distinguish sandboxes and production services.
Not usually as-is. Promote the tested artifact with production configuration, identity, data services, health checks, scaling, and observability. Northflank can run the production service and the ephemeral sandboxes it creates under the same deployment model.
Monitor task quality and policy behavior alongside availability, latency, errors, capacity, and cost. Northflank provides infrastructure logs, metrics, health checks, and alerts that you can correlate with application-level agent and tool traces.
Restore the previous artifact, prompts, policy, tool schemas, and model configuration. Northflank workflows can restore an earlier deployment, while database changes, memory, and external agent actions still require separate recovery.
