- SDK
- Image Tool
SDK
Image Tool
Generate, edit, and reuse images in assistant conversations.
Set image_generation="auto" to let the assistant use the built-in generate_image tool. The conversation model (llm_provider / model_name) decides when to call it; the image model (image_model_provider / image_model_name) creates the image. No custom tool registration or tool-output submission is needed.
Start with the SDK setup. For generation without a conversation model, use Stateless Images instead.
Choose an input pattern
Describe the image or edit in content. Upload local files with files, or mention previously uploaded document IDs and their roles in the prompt.
Use natural language: users do not need to name the tool or its arguments. The assistant selects the tool and supplies the input IDs. When attaching multiple files, explain which image to edit and which to use as a reference.
| Workflow | Message input | Assistant tool arguments |
|---|---|---|
| Text → image | Prompt only | No input IDs |
| Image → image | One image or an existing image ID | input_image_document_id |
| Multiple images → image | Several images with their roles | input_reference_document_ids |
| Base image + references → image | Base image and supporting references | Both fields above |
| Reuse an image | Generated or uploaded image ID | Either field, depending on its role |
| Audio/video or mixed references → image | Stored reference IDs | input_reference_document_ids; conditional model support and empty config required |
The document-ID fields are arguments the assistant supplies to the tool, not SDK message parameters. Do not use stateless input_image / input_references here. IDs must identify stored media accessible to the current thread or its assistant; input_image_document_id must always identify an image.
Text to image
These examples use an existing client and thread_id. Pass the image-tool settings again on each generation or edit turn.
settings = {
"llm_provider": "openai",
"model_name": "gpt-4.1",
"image_generation": "auto",
"image_model_provider": "openrouter",
"image_model_name": "google/gemini-3.1-flash-image",
}
result = await client.add_message(
thread_id,
content="Draw a robot watering plants.",
stream=False,
**settings,
)
print(result.generated_media)
send_message() also accepts these settings and can create a thread automatically. Use add_message() for files uploads.
Image to image
Attach an image and describe the edit. The assistant receives its document ID with the attachment.
result = await client.add_message(
thread_id,
content="Change the ball to red.",
files=["photo.png"],
**settings,
)
Choose an image model supporting both image input and output. Its reference-count limits also apply.
Multiple image references
Attach several files and explain how to use each one. There is no separate composition mode.
result = await client.add_message(
thread_id,
content="Make a poster with the subject from subject.png and the colors from style.png.",
files=["subject.png", "style.png"],
**settings,
)
Base image plus references
Upload separately when you want explicit ID-to-role mapping.
base = await client.upload_document_to_thread(thread_id, "photo.png")
style = await client.upload_document_to_thread(thread_id, "style.png")
result = await client.add_message(
thread_id,
content=(f"Change the colors in image {base.document_id} to match image {style.document_id}. "
"Keep the subject and layout the same."),
**settings,
)
The base and reference list count together toward the model/endpoint’s limit, with duplicate IDs counted once. A base image is not a mask or a guarantee of pixel-preserving edits.
Reuse a generated image
Collect images from the previous response and reference their IDs in the same thread. Previously uploaded IDs work the same way; no re-upload is needed.
images = [
media
for message in result.messages
for media in (message.get("generated_media") or [])
if media.get("media_type") == "image"
]
if images:
result = await client.add_message(
thread_id,
content=f"Make the background a sunset in image {images[0]['document_id']}.",
**settings,
)
Persist both the thread ID and document ID for later edits. Include the ID when you need to identify a particular image. For an unambiguous follow-up in the same conversation, a prompt such as “Make the background a sunset” can refer to the previous image.
Audio/video and mixed references
Audio, video, and mixed image/audio/video references are conditional: select an OpenRouter image-tool model that accepts every supplied input modality and produces images. Support is not universal, and unsupported inputs are rejected.
Use stored IDs in input_reference_document_ids, with an optional image-only base. Keep image_config empty in both the message and the assistant’s tool call, including any provider settings. Setting image_config={} alone does not stop the assistant from choosing controls, so explicitly request no custom controls. For example:
Create a cover illustration inspired by image [image ID] and audio [audio ID], without custom generation settings.
Upload references with files or a document upload, not audio_file (speech-to-text input). Stateless Images accepts image references only.
TypeScript and HTTP equivalents
Use the same prompts and files for each pattern. The TypeScript example assumes an existing server-side Node.js client and threadId.
const result = await client.addMessage(threadId, {
content: "Change the ball to red.",
files: ["photo.png"],
llmProvider: "openai",
modelName: "gpt-4.1",
imageGeneration: "auto",
imageModelProvider: "openrouter",
imageModelName: "google/gemini-3.1-flash-image",
stream: false,
});
if ("messages" in result) {
const images = result.messages.flatMap(message => message.generatedMedia ?? []);
for (const image of images) console.log(image.document_id, image.url);
}
For multiple HTTP uploads, repeat files. Serialize image_config as JSON in multipart requests; without uploads, you can send a JSON body with an object-valued image_config. TypeScript also provides uploadDocumentToThread(threadId, "photo.png"), returning documentId for use in prompts.
Configuration and results
- Configuration: pass Python
image_configor TypeScriptimageConfig, with snake_case inner keys, such as{"resolution": "1K", "aspect_ratio": "16:9"}where supported. Use only controls supported by the selected model and endpoint. Caller-supplied fields are fixed; the assistant may choose omitted fields but cannot override fixed values or switch the image model. See the configuration reference. - Non-streaming: read
generated_mediaon Python/HTTP response messages ormessage.generatedMediain TypeScript. Python’sresult.generated_mediais a convenience for the latest message only; collect acrossresult.messagesfor all outputs. - Streaming: set
stream=True/stream: trueand consumemedia_generatedevents.event.mediacontainsdocument_id,media_type,mime_type,file_size_bytes, andurl. These are stored-image notifications, not partial-image rendering. Media objects and events retain snake_case keys in TypeScript. - Check outputs: the assistant may decide not to generate or may encounter a tool error. Inspect response messages, tool results, and stream errors if no media is returned; do not extract image URLs from assistant prose.