Images, audio & video
Sending media with a message.
How media is sent
Media travels inline, base64-encoded, inside the message. A URL is always refused — fetching one on your behalf would turn our servers into a way to reach hosts you cannot reach yourself.
Check input_modalities in the model list first. A model that cannot read a kind of media rejects it outright rather than answering as if nothing was attached.
Encoding inflates a file by roughly 33%, and the whole request body is capped at 8 MB. In practice that means one ~5 MB file per request, alongside a long conversation.
Images
PNG, JPEG, GIF or WebP, up to 8 per request, each under 5 MB.
1{2 "model": "amazon/nova-2-lite-v1:0",3 "messages": [{4 "role": "user",5 "content": [6 { "type": "text", "text": "What is in this picture?" },7 { "type": "image_url", "image_url": { "url": "data:image/png;base64,iVBORw0..." } }8 ]9 }]10}Audio
OpenAI's input_audio part, up to 4 per request. data is bare base64 and format is one of wav, mp3, flac, ogg, opus, m4a, mp4, aac, webm, mka, mpga or pcm. A data URL works too, in which case the format is read from it.
1{2 "model": "mistral/voxtral-mini-3b-2507",3 "messages": [{4 "role": "user",5 "content": [6 { "type": "text", "text": "Transcribe this." },7 { "type": "input_audio", "input_audio": { "data": "UklGRi...", "format": "wav" } }8 ]9 }]10}Audio encoders pad to a fixed window, so a two-second clip and an eight-second clip can cost the same in prompt tokens. The reservation we place before the call is deliberately generous; the charge on your balance is always the usage the model reports.
Video
OpenAI has no content part for video, so Sothe mirrors image_url: a video_url part with a data URL, up to 2 per request, in mp4, webm, mov, mkv, mpeg, mpg, flv, wmv or 3gpp.
1{ "type": "video_url", "video_url": { "url": "data:video/mp4;base64,AAAAIG..." } }