What's new in 0.8.0¶

Published on 2026-10-07: Python SDK rayito 0.8.0, TypeScript SDK rayito 0.8.0 and rayd 0.8.0.

0.8.0 brings a coding agent that runs inside the sandbox (sbx.agent, with OpenCode or deepagents), the pieces so that its first response arrives quickly (AgentTemplate with prefetch, pause between turns and PoolConfig.warmup), gateway presets to call the model without the agent being able to read the credential, and a Pricing page that separates what the VM costs from what the tokens cost. As always, everything that has a cost is off by default: without calling sbx.agent, without building an AgentTemplate and without passing warmup=, a sandbox makes no new call to AWS.

There is one breaking change: register_webhook / registerWebhook now rejects URLs to localhost and to non-public IPs (How to upgrade from 0.7.x).

At a glance¶

What's new	Python	TypeScript	Needs	Own cost	Guide
Agent in the sandbox	`sbx.agent.run()` / `.stream()` / `.prepare()`	`sbx.agent.run()` / `.stream()` / `.prepare()`	an image with the runtime (`AgentTemplate`) and a gateway to the model (`gateways=`)	$0 from the SDK; the model's tokens and the gateway	Agent in the sandbox
Gateway presets	`bedrock_gateway`, `anthropic_gateway`, `openai_compatible_gateway`	`bedrockGateway`, `anthropicGateway`, `openaiCompatibleGateway`	the model's secret in Secrets Manager	that of the secret gateway	The secret and the gateway
Agent image	`AgentTemplate(...).build()`, `rayito agent template build`	`new AgentTemplate({...}).build()`	the `templates` stack	≈ $0.057/week per image version	Agent templates
Pool warm-up	`PoolConfig(warmup=agent_pool_warmup(...))`	`new SandboxPool({ warmup: agentPoolWarmup(...) })`	a pool with `allow_internet_access=False`	≈ $0.64/slot/month (versus $0.60 for a base slot)	Pool
Pricing	—	—	—	—	Pricing

A coding agent inside the sandbox¶

sbx.agent runs a coding agent inside the MicroVM itself: it sees the same filesystem, the same commands and the same egress as the rest of the sandbox's code, and calls its model only through the secret gateway. The agent uses the model's credential, but can never read it.

PythonTypeScript

from rayito import AgentModel, AgentSpec, Sandbox, bedrock_gateway

MODEL_ID = "us.anthropic.claude-haiku-4-5-20251001-v1:0"

spec = AgentSpec(
    model=AgentModel(provider="bedrock", id=MODEL_ID, gateway="bedrock", region="us-east-1"),
    instructions="Reply in English and do not modify files outside /home/user.",
)

with Sandbox.create(
    template="rayito-agent",
    allow_internet_access=False,
    gateways={
        "bedrock": bedrock_gateway("bedrock-key", region="us-east-1", models=[MODEL_ID], rate_per_minute=60),
    },
) as sbx:
    result = sbx.agent.run("List the files in /home/user and summarize what is there.", spec=spec)
    print(result.text, result.steps, result.usage.total)

import { AgentModel, AgentSpec, Sandbox, bedrockGateway } from "rayito";

const MODEL_ID = "us.anthropic.claude-haiku-4-5-20251001-v1:0";

const spec = new AgentSpec({
  model: new AgentModel({ provider: "bedrock", id: MODEL_ID, gateway: "bedrock", region: "us-east-1" }),
});

await using sbx = await Sandbox.create({
  template: "rayito-agent",
  allowInternetAccess: false,
  gateways: {
    bedrock: bedrockGateway("bedrock-key", { region: "us-east-1", models: [MODEL_ID], ratePerMinute: 60 }),
  },
});
const result = await sbx.agent.run("List the files in /home/user and summarize what is there.", { spec });
console.log(result.text, result.steps, result.usage.total);

What it brings, the same in synchronous Python, asyncio and TypeScript:

Three calls. run() runs the agent to completion and returns an AgentResult (text, session_id, steps, usage, tool_calls) or raises AgentException / AgentError. stream() returns an AgentStream that never raises on an agent failure: the last event is Done or AgentFailed. prepare() launches the runtime's warm-up steps in the background and returns right away.
Two runtimes. OpenCode by default and deepagents with runtime="deepagents" or runtime=DeepAgents(entrypoint="pkg.mod:build") (TypeScript: runtime: "deepagents" or new DeepAgents({...})), with the same events, the same limits and the model only through the gateway (deepagents).
AgentSpec describes the run and is validated on construction, without calling AWS: AgentModel (Bedrock, Anthropic or an OpenAI-compatible API, always through an entry of gateways=), instructions, AgentPermissions (question, webfetch and websearch denied by default), SubAgent and MCP servers (McpLocal, McpRemote).
SDK limits with AgentLimits: 50 steps, 600 s, 16 MiB of output and 1,000,000 tokens by default. max_steps and max_total_tokens are enforced by the SDK itself over the events, not by the runtime.
Events: StepStarted, Text, TextDelta (deepagents), Reasoning, ToolCall, StepFinished (with usage), AgentFailed and Done. Sessions with session_id / sessionId to continue a conversation, and abort() or an AbortSignal to cut it off (stream, sessions, abort and limits).
Errors with a closed reason and a fixed message in Spanish: never the provider's text, the prompt or sandbox content (AgentException / AgentError).
Gateway presets per provider, with allow restricted to the bare minimum: bedrock_gateway only opens converse/converse-stream of the models you pass, never /model/*. All three accept rate_per_minute / ratePerMinute.
Optional telemetry: with tracer_provider= / tracerProvider, each run opens a rayito.agent.run span with gen_ai.* and token attributes; never the prompt, the response or the tool arguments (OpenTelemetry).

sbx.agent sends no RPC until the first run()/stream(), and in Python import rayito does not load the agent until you use one of its names.

Permissions are not a security boundary

The runtime executes whatever the model asks for within what is allowed, without asking. The real boundary is the MicroVM plus closed egress: Sandbox.create leaves it open by default, so pass allow_internet_access=False yourself. Threats and mitigations in Security.

Full guide: Agent in the sandbox.

Fast start¶

OpenCode's first exec on a freshly created VM is slow (it reads the binary from the restored VM's disk). For most cases no pool is needed: that cost is paid once per conversation. The guide has a decision tree, What do I use?, with four options that combine:

Option	What it does	When	Cost on top of the VM
A. `AgentTemplate` with prefetch	the image carries a daemon that, after each restore, waits for the guest to settle and preloads the binaries	whenever you build the image with `AgentTemplate` (it is the default)	≈ $0
B. `pause()` between turns	one VM per conversation, suspended between turns; `connect()` resumes it (≈ 3 s to the first token)	the general case: conversations under 8 h	≈ $0.005–0.006 per cycle + ≈ $0.0001/h paused
C. Pool with `warmup`	`agent_pool_warmup("opencode")` runs the binary in each slot before parking it (≈ 5 s to the first token from the take)	many new conversations whose first message has to be fast	≈ $0.64/idle slot/month
D. Pool with a resident `opencode serve`	`agent_pool_warmup("opencode", serve=True)`; `run` attaches with `--attach`	not recommended: it barely beats C	≈ $0.82/idle slot/month

The figures are measured on real AWS (AWS_API_NOTES.md Q146–Q154): Fast start.

Option D is removed in 0.9.0

serve=True, attach and the resident opencode serve disappear in 0.9.0 with no deprecation period. Use C (agent_pool_warmup("opencode")) or the normal start; see the note in What's new.

The agent image. AgentTemplate installs OpenCode and ripgrep pinned by sha256, the deepagents venv with --require-hashes (except --no-deepagents / runtimes: ["opencode"]), its runner and the rayito.agent-template/1 manifest, all owned by root:

PythonTypeScriptCLI

from rayito import AgentTemplate

AgentTemplate(
    name="rayito-agent",
    base="rayito-base-caps",
    runtimes=("opencode", "deepagents"),
    prefetch=True,
    memory_mib=2048,
).build(bucket="amzn-s3-demo-bucket")

import { AgentTemplate } from "rayito";

await new AgentTemplate({
  name: "rayito-agent",
  base: "rayito-base-caps",
  runtimes: ["opencode", "deepagents"],
  prefetch: true,
  memoryMib: 2048,
}).build({ bucket: "amzn-s3-demo-bucket" });

rayito agent template build --name rayito-agent --bucket amzn-s3-demo-bucket

Details and build cost: Agent templates.

The warmed pool. PoolConfig gains warmup (steps each slot runs before being parked; a failed step is a failed warm-up), allow_internet_access and network:

PythonTypeScript

from rayito import PoolConfig, agent_pool_warmup

config = PoolConfig(
    size=3,
    template="rayito-agent",
    allow_internet_access=False,
    warmup=agent_pool_warmup("opencode"),
)

import { SandboxPool, agentPoolWarmup } from "rayito";

await using pool = await new SandboxPool({
  size: 3,
  template: "rayito-agent",
  allowInternetAccess: false,
  warmup: agentPoolWarmup("opencode"),
}).start();

Details and measurements: Pool.

Cost and activation

None of this costs anything unless you ask for it. AgentTemplate only costs if you build the image (≈ 3.1 GB of snapshots, ≈ $0.057/week per version); warmup only if you pass it to a pool (≈ $0.64/slot/month, or ≈ $0.82 with serve=True); sbx.agent has no SDK cost of its own beyond the VM, the model's tokens and the gateway. Each symbol carries its "Cost and activation" block and its row in Optional features.

Pricing: the VM versus the tokens¶

The Pricing page gathers the cost of each MicroVMs component (launch, compute, snapshots, suspension), of the pool per slot and of an agent, separating what the VM costs from what the model's tokens cost. Each figure is marked as a list price (with its lookup date) or as measured on real AWS. For an agent, the model usually weighs more than the VM: Cost of an agent and which fast-start option to choose.

Other fixes and changes¶

Agent (all new in 0.8.0; fixed before the release after acceptance on real AWS):

abort() and the max_steps / max_total_tokens limits also stop what the agent's shell tool launched (a sleep or a server no longer survive), the runtime does not keep spending tokens in the background and the next run() does not find it busy.
An AgentLimits timeout ends with reason="timeout", not "runtime_error". The timeout still does not reach what the agent's shell launched (warning).
The prefetch daemon waits for the guest to go 1 s without I/O (at most 60 s) before reading: before, it competed with startup and create() took ≈ 8 s longer.
AgentTemplate installs the deepagents runner (and its sha256 in the manifest); without it, runtime="deepagents" failed on those images.
prepare() starts its steps with no deadline: with serve=True the resident server no longer dies when timeout_seconds / timeoutMs runs out.
With a resident opencode serve (option D), the SDK rereads each turn's messages from the server and no longer returns protocol_error or a Done without text.
deepagents: the file tools use an absolute path as is, like the shell.

The rest of the SDK:

S3 mounts: the MountException / MountError with code="not_allowed" explains that the bucket is missing from RAYITO_ALLOWED_MOUNT_BUCKETS and how to republish the image with it, without naming the bucket (the image's allowlist).
Python: import rayito is lightweight again. The agent and the lifecycle events' boto3 adapter are imported on the first access to their names (about 20 fewer entries in sys.modules).
rayito doctor knows the 0.8 series: it requires the rayd from the rayd-v0.8.0 tag for SDK 0.8 (Compatibility).

rayd 0.8.0: no changes in the agent. The AI agent runs on the RPCs that already existed, and the version goes up in step with the SDKs.

How to upgrade from 0.7.x¶

Upgrade the SDK.

PythonTypeScript

pip install -U "rayito[cli]"

pnpm add rayito@latest

Republish your image on rayd 0.8.0 (Images). The compatibility series moves to 0.8: rayito doctor requires the rayd from the rayd-v0.8.0 tag for SDK 0.8 and gives FAIL on the compatibility check with a 0.7 image.
To use the agent, build its image with AgentTemplate or rayito agent template build (Agent templates). Without the runtime in the image, sbx.agent fails with reason="runtime_missing".
Review the breaking change:

What	Before (up to 0.7.1)	Since 0.8.0
`register_webhook` / `registerWebhook` with a URL whose host is `localhost` (or `*.localhost`) or a non-public literal IP (loopback, private, link-local, metadata, multicast, reserved, unspecified or CGNAT, also as IPv6 with an IPv4 inside or in numeric forms such as `0x7f.1`)	it was stored, but the deliverer discarded it on every delivery: nothing was ever delivered	`InvalidArgumentException` / `InvalidArgumentError` before any call to AWS

Migration: register an https:// URL with a public name (for example https://hooks.example.com/rayito), behind your TLS termination. A DNS name is still not resolved at registration; if it points to a non-public address, the deliverer stops it on delivery (Responses and retries).

Outside that row, there are no incompatible changes in the public API from 0.7.x.

What 0.8.0 does not bring¶

Sandbox.create(domain=) and get_host()/expose() returning the custom domain's URL: they remain in development.
rayd killing the entire process tree when the agent's timeout expires, and the SDK warning when an agent runs with open egress: coming soon.

Full details, change by change: Changelog.