What's new in 0.10.0¶

Published on 2026-10-08: Python SDK rayito 0.10.0, TypeScript SDK rayito 0.10.0 and rayd 0.10.0.

0.10.0 brings three things for those who use the agent in the sandbox (sbx.agent):

More model providers. Up to 0.9 the agent talked to Bedrock, Anthropic or an OpenAI-compatible API that you configured by hand. Now there is a ready-made preset for OpenAI, Gemini, Azure OpenAI, OpenRouter, Groq, Mistral, DeepSeek, xAI and your own LiteLLM proxy.
The timeout and abort() really stop things. Before, a process the agent launched with its shell tool (a sleep, a server) could survive the timeout. Now rayd stops everything the run launched.
deepagents works outside Bedrock and counts its tokens.

There are no API-breaking changes. For point 2 and for deepagents you have to rebuild the image: see How to upgrade from 0.9.x.

At a glance¶

Change	Python	TypeScript	Needs a new image
Nine provider presets and three new `AgentModel.provider` values	`openai_gateway()`... `litellm_gateway()`; `"openai"`, `"google"`, `"azure"`	`openaiGateway()`... `litellmGateway()`; the same `provider`s	yes, the agent's (`AgentTemplate`)
The agent's timeout and `abort()` stop its whole process tree	`sbx.agent`, `commands.run(kill_tree=)`	`sbx.agent`, `commands.run({ killTree })`	yes (`rayd` 0.10.0)
deepagents with non-Bedrock models (`socksio` was missing)	`deepagents` runtime	`deepagents` runtime	yes, the agent's
deepagents counts tokens for OpenAI-compatible APIs	`result.usage`, `AgentLimits.max_total_tokens`	`result.usage`, `AgentLimits.maxTotalTokens`	yes, the agent's
`rayito doctor` knows the 0.10 series	CLI	—	—

More providers for the agent's model¶

The agent never sees the model's key. The key lives in Secrets Manager and rayd's secret gateway puts it in the header of each request. A preset is that gateway already configured for a provider: it knows which address to call, which header carries the key and which routes are allowed.

Which one do I choose?¶

Your situation	Choose	Why
You already work on AWS and want a single bill	Bedrock (`bedrock_gateway`)	the gateway also limits which models can be called; it is the most tested route
You already have an OpenAI key	OpenAI (`openai_gateway`)	native Responses API in OpenCode and deepagents
Your company uses Azure	Azure OpenAI (`azure_openai_gateway`)	the model is a deployment of your resource, with its own quota
You want Gemini	Gemini (`gemini_gateway`)	as with Bedrock, the gateway limits the models
You want to try many models with a single key	OpenRouter (`openrouter_gateway`)	a large catalog behind one API
You want speed or a low price	Groq, DeepSeek or Mistral	OpenAI-compatible APIs
You already have Grok	xAI (`xai_gateway`)	speaks the same Responses API as OpenAI
You already have a LiteLLM proxy, or want local models	LiteLLM (`litellm_gateway`)	virtual keys with a budget and any model the proxy serves

If in doubt, start with Bedrock or with LiteLLM in front of Bedrock: those are the routes that have been tested with a real model (see what was actually tested).

Set it up in three steps¶

Store the key in Secrets Manager, once, from your machine or your backend. For OpenAI-style APIs the value is the full header: Bearer <key>. For Gemini and Azure, the key as is.
Pass the preset in gateways= when creating the sandbox. The name you give it (here "openai") is the one the model will use.
Point the AgentModel at that gateway with the provider's provider and its model id, and run.

PythonPython (async)TypeScript

from rayito import AgentModel, AgentSpec, Sandbox, SecretStore, openai_gateway

# 1. The key, once
SecretStore().create("openai-key", "Bearer <OpenAI API key>")

# 2. The preset, named "openai"
gateways = {"openai": openai_gateway("openai-key", rate_per_minute=60)}

# 3. The model points at that gateway
spec = AgentSpec(model=AgentModel(provider="openai", id="gpt-5-mini", gateway="openai"))

with Sandbox.create(template="rayito-agent", allow_internet_access=False, gateways=gateways) as sbx:
    print(sbx.agent.run("Summarize the README.", spec=spec).text)

from rayito import AgentModel, AgentSpec, AsyncSandbox, openai_gateway


async def main() -> None:
    gateways = {"openai": openai_gateway("openai-key", rate_per_minute=60)}
    spec = AgentSpec(model=AgentModel(provider="openai", id="gpt-5-mini", gateway="openai"))
    async with await AsyncSandbox.create(
        template="rayito-agent", allow_internet_access=False, gateways=gateways
    ) as sbx:
        result = await sbx.agent.run("Summarize the README.", spec=spec)
        print(result.text)

import { AgentModel, AgentSpec, Sandbox, SecretStore, openaiGateway } from "rayito";

// 1. The key, once
await new SecretStore().create("openai-key", "Bearer <OpenAI API key>");

// 2. The preset, named "openai"
const gateways = { openai: openaiGateway("openai-key", { ratePerMinute: 60 }) };

// 3. The model points at that gateway
const spec = new AgentSpec({
  model: new AgentModel({ provider: "openai", id: "gpt-5-mini", gateway: "openai" }),
});

await using sbx = await Sandbox.create({ template: "rayito-agent", allowInternetAccess: false, gateways });
console.log((await sbx.agent.run("Summarize the README.", { spec })).text);

For another provider only the preset and the AgentModel change:

Provider	Secret value	Preset (Python)	`AgentModel`
OpenAI	`Bearer sk-…`	`openai_gateway(secret)`	`provider="openai"`
Gemini	the key as is	`gemini_gateway(secret, models=[…])`	`provider="google"`
Azure OpenAI	the key as is	`azure_openai_gateway(secret, resource="my-resource")`	`provider="azure"`, `id` = deployment name
OpenRouter	`Bearer …`	`openrouter_gateway(secret)`	`provider="openai-compatible"`, `base_path="/api/v1"`
Groq	`Bearer …`	`groq_gateway(secret)`	`provider="openai-compatible"`, `base_path="/openai/v1"`
Mistral	`Bearer …`	`mistral_gateway(secret)`	`provider="openai-compatible"`, `base_path="/v1"`
DeepSeek	`Bearer …`	`deepseek_gateway(secret)`	`provider="openai-compatible"`, no `base_path`
xAI	`Bearer xai-…`	`xai_gateway(secret)`	`provider="openai"`
LiteLLM	`Bearer <virtual key>`	`litellm_gateway(secret, upstream="https://…")`	`provider="openai-compatible"`, `base_path="/v1"`

In TypeScript the presets have the same names in camelCase (openaiGateway, geminiGateway, azureOpenaiGateway, openrouterGateway, groqGateway, mistralGateway, deepseekGateway, xaiGateway, litellmGateway) and the options go in an object ({ models }, { resource }, { upstream }). Each provider, with its full example in both languages and the link to its console, is in Agent providers.

Set the spending cap at the provider

On Bedrock and Gemini the model goes in the path and the gateway only lets through the models you list. On OpenAI-style APIs (OpenAI, Azure OpenAI, xAI, OpenRouter, Groq, Mistral, DeepSeek, LiteLLM) the model goes in the body, which the gateway does not read: any code in the sandbox can request any model the key allows. Put the cap at the provider (project budget, the key's credit limit, the LiteLLM virtual key's budget) and use rate_per_minute / ratePerMinute. Details in Provider-side limits.

Your own LiteLLM proxy¶

If you already have a LiteLLM proxy, the agent can use any model it serves (from any provider, or a local one such as vLLM or Ollama) with a single virtual key. Three conditions:

The proxy listens on HTTPS and the sandbox's VPC reaches it. Your laptop's localhost will not do: the preset rejects http://, localhost, loopback, 0.0.0.0 and link-local.
The virtual key has a budget (max_budget) and a model list. It is your only real spending cap.
The AgentModel's base_path is the same as the preset's (/v1 by default).

PythonTypeScript

from rayito import AgentModel, SecretStore, litellm_gateway

SecretStore().create("litellm-key", "Bearer <LiteLLM virtual key>")
gateway = litellm_gateway("litellm-key", upstream="https://llm.example.com")
model = AgentModel(provider="openai-compatible", id="<proxy model>", gateway="litellm", base_path="/v1")

import { AgentModel, SecretStore, litellmGateway } from "rayito";

await new SecretStore().create("litellm-key", "Bearer <LiteLLM virtual key>");
const gateway = litellmGateway("litellm-key", { upstream: "https://llm.example.com" });
const model = new AgentModel({ provider: "openai-compatible", id: "<proxy model>", gateway: "litellm", basePath: "/v1" });

What was actually tested¶

Route	How it was tested
Bedrock (`bedrock_gateway`), OpenCode and deepagents	with a real model, on real AWS (MicroVMs) in this release's acceptance, from Python and TypeScript
Bedrock's OpenAI-compatible API (`bedrock-runtime` `/openai/v1` and `bedrock-mantle`) through `openai_compatible_gateway`	with a real model, in the local environment, with egress closed: tools, session and token usage
`provider="openai"` (Responses API) against `bedrock-mantle`	with a real model, locally, one turn without tools (`bedrock-mantle` departs from OpenAI's API with tools)
LiteLLM (`litellm_gateway`) in front of Bedrock	with a real model, locally: tools, session and token usage
OpenAI, Gemini, Azure OpenAI, OpenRouter, Groq, Mistral, DeepSeek and xAI	only against a fake upstream locally: the nine presets × OpenCode and deepagents × Python and TypeScript complete a turn, the key arrives in its header, the marker never leaves and a route outside the list gets 403

Those eight routes have not been tested with a real provider key. If you have one, the smoke test (make local-providers-smoke) does a short turn against the real API for less than 0.01 USD per provider.

Providers that are not supported, and why¶

Rayito only supports API keys. Consumer subscriptions were reviewed and are not supported:

Option	Why
ChatGPT plan (OpenCode's OAuth)	it uses another application's `client_id` and endpoints that OpenAI asks not to use (models-and-inference); it would also put the refresh token within the agent's reach
ChatGPT plan with "Sign in with ChatGPT"	deferred: for a remotely hosted app OpenAI asks for an interest form (overview)
Claude Pro/Max	Anthropic's consumer terms do not allow third parties to route requests with those credentials; use an Anthropic or Bedrock key
GitHub Copilot	no terms for unattended fleets (acceptable use policy)
SuperGrok	no terms for automation; use the xAI API key

The full explanation is in Unsupported providers.

The agent's timeout and `abort()` stop everything it launched¶

OpenCode and deepagents run each command of their shell tool in a session of its own (setsid). Up to 0.9, the timeout only killed the runtime's process: if the agent had launched sleep 600 or a server, that process stayed alive, and the next run() found the agent busy (reason="busy").

Since 0.10.0, with a rayd 0.10.0 image, this is guaranteed:

When the timeout expires (timeout_seconds / timeoutMs), rayd asks the whole tree to stop (SIGTERM) and, if anything is left after 5 s, it freezes and kills it. The stream ends with reason="timeout" only when nothing is left alive.
abort(), an AbortSignal and the SDK's limits (max_steps, max_total_tokens) kill the entire tree without waiting.
Nothing else is touched: not other commands, not the terminal, not the run_code kernel. And if the run ends by itself, what it deliberately left running (a server) stays alive.

You do not have to change your code:

PythonTypeScript

from rayito import AgentLimits, AgentSpec, Sandbox


def run_with_deadline(sbx: Sandbox, spec: AgentSpec) -> None:
    result = sbx.agent.run(
        "Start the development server and test the API.",
        spec=spec,
        limits=AgentLimits(timeout_seconds=120),
    )
    print(result.text)

import { AgentLimits, type AgentSpec, type Sandbox } from "rayito";

export async function runWithDeadline(sbx: Sandbox, spec: AgentSpec): Promise<void> {
  const controller = new AbortController();
  setTimeout(() => controller.abort(), 60_000);
  const result = await sbx.agent.run("Start the development server and test the API.", {
    spec,
    limits: new AgentLimits({ timeoutMs: 120_000 }),
    signal: controller.signal,
  });
  console.log(result.text);
}

The same guarantee is available for any command with commands.run(..., kill_tree=True) / commands.run(cmd, { killTree: true }) (Commands). With an earlier image the field is ignored and the timeout only reaches the runtime's group. Details in What the timeout and abort() guarantee.

deepagents fixes¶

It works with non-Bedrock models. rayd exports ALL_PROXY=socks5h://… in the sandbox, and deepagents' HTTP client failed at startup (AgentFailed with ImportError) with OpenAI, xAI, Azure OpenAI, Gemini, Anthropic and OpenAI-compatible APIs. The agent image now ships socksio.
It counts tokens for OpenAI-compatible APIs. With provider="openai-compatible", each step arrived with 0 tokens: result.usage stayed at zero and max_total_tokens / maxTotalTokens never cut off. Now usage is requested in streaming, as OpenCode already did.

Both need you to rebuild the agent image with 0.10.0's AgentTemplate.

Other changes¶

rayito doctor knows the 0.10 series: it requires the rayd from the rayd-v0.10.0 tag for SDK 0.10 (Compatibility).
rayd 0.10.0: StartRequest.kill_tree (new, false by default) makes the process a child subreaper and carries the timeout and signals to its whole tree.

How to upgrade from 0.9.x¶

Upgrade the SDK.

PythonTypeScript

pip install -U "rayito[cli]"

pnpm add rayito@latest

Republish your image on rayd 0.10.0 (Images). rayito doctor requires the rayd from the rayd-v0.10.0 tag for SDK 0.10 and gives FAIL on the compatibility check with a 0.9 image.
If you use the agent, rebuild its image with AgentTemplate or rayito agent template build: the new one carries rayd 0.10.0, socksio and the fixed deepagents runner.

There are no incompatible changes in the public API from 0.9.x.

What's new in 0.10.0¶

At a glance¶

More providers for the agent's model¶

Which one do I choose?¶

Set it up in three steps¶

Your own LiteLLM proxy¶

What was actually tested¶

Providers that are not supported, and why¶

The agent's timeout and abort() stop everything it launched¶

deepagents fixes¶

Other changes¶

How to upgrade from 0.9.x¶

The agent's timeout and `abort()` stop everything it launched¶