Streaming

Turning it on

Set "stream": true. The response becomes text/event-stream. Any OpenAI SDK handles it for you.

1const stream = await client.chat.completions.create({2  model: "anthropic/claude-haiku-4-5-20251001-v1:0",3  messages: [{ role: "user", content: "Write a haiku." }],4  stream: true,5});67for await (const chunk of stream) {8  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");9}

What arrives

1: SOTHE PROCESSING23data: {"choices":[{"delta":{"role":"assistant","content":""},"finish_reason":null}]}45data: {"choices":[{"delta":{"content":"Hello"},"finish_reason":null}]}67data: {"choices":[{"delta":{},"finish_reason":"stop"}]}89data: [DONE]

Lines beginning with : are SSE comments. We send one every 15 seconds while the model is still thinking, so an intermediary does not decide the connection is idle and drop it. Every SSE client ignores them; a hand-rolled parser must too.

Tool calls stream as delta.tool_calls with an index, the same way OpenAI sends them, and reasoning arrives as delta.reasoning.

Usage in a stream

Add "stream_options": { "include_usage": true } and a final chunk arrives with empty choices and a populated usage, including cost.

Mid-stream failures

If generation fails after output has started, the last data: line carries an error object instead of a chunk. The HTTP status is already 200 by then, so check for that key rather than relying on the status.