Image Tool

The built-in Image Tool lets an assistant generate or edit images in a conversation. You enable it for a message; the conversation model decides whether to call generate_image, and Backboard executes the tool internally. No client-side custom tool handler is required.

For direct generation without a conversation-model call, see stateless images. For runnable Python SDK 1.5.17 and TypeScript examples of each pattern, see the SDK image-tool guide.

​

How it works

1. Send a normal message to /threads/messages with image_generation: "auto".
2. Choose llm_provider / model_name for the conversation and image_model_provider / image_model_name for image generation.
3. Optionally upload media and provide caller-fixed image_config.
4. The assistant decides whether to call generate_image, supplies a prompt and accessible document IDs, and may fill unspecified supported controls.
5. Backboard validates inputs, calls the image provider, stores outputs, and returns image metadata alongside the conversation response.

These settings are per turn, not permanent assistant settings. Enabling auto does not force a generation. Pass the settings again for follow-up edits.

​

Message fields versus tool arguments

The assistant, not the SDK caller, passes these internal tool arguments:

Do not pass input_*_document_id as chat SDK keyword arguments or put paths/URLs in them. Upload with files or the document-upload methods, then ask the assistant to use the IDs. The prompt is an instruction, not a guaranteed tool invocation. Inspect tool calls and outputs for workflows requiring exact reference selection.

​

Supported input patterns

Backboard combines the base and reference list in order, base first, removing duplicate IDs. A base image is not a mask or a guarantee of pixel-preserving edits. The tool can only access stored media from the current thread (including message attachments) or its assistant, not arbitrary documents from another thread. Save the thread ID and output document IDs for reuse; do not use the output URL as a document ID.

Same-turn image uploads with generation enabled require an image model supporting both supports_vision and supports_image_output; otherwise the API can return 400 image_model_vision_required. Missing image-model selection can return 400 image_model_required. These image-model checks are distinct from the conversation model’s capabilities.

​

Model-specific configuration

For OpenRouter, Backboard discovers image model and endpoint capabilities before the conversation model’s first call and exposes supported controls in the tool schema. SDK types describe possible fields, not what every model accepts.

Use only the selected model’s advertised enums and ranges. For a model/endpoint advertising resolution: "1K" and aspect_ratio: "16:9", those can be fixed together. That does not imply support for another model’s quality, size, or background values. The SDK guide reads model IDs and model-dependent configuration from environment variables rather than prescribing a universal combination.

Additional rules:

• Explicit API fields are fixed; a conflicting tool value fails validation. Omitted fields can be selected by the assistant. An empty object does not lock all controls to defaults.
• background: "transparent" cannot be combined with an explicit output format other than png or webp.
• output_compression cannot be combined with explicit output_format: "png"; use a supported JPEG/WebP combination instead.
• A compatible endpoint must support all controls and the combined reference count. Model-level support alone does not guarantee an endpoint accepts the combination.
• Invalid caller configuration is rejected before generation; internal tool arguments are validated against the capability snapshot used to build the schema. The assistant is instructed to ask before substituting unsupported requirements, but provider success is not guaranteed.

provider supports only, order, and ignore lists of endpoint provider tags; sort (price, throughput, or latency); and boolean allow_fallbacks. Where advertised, provider.options maps provider slugs to allowlisted passthrough keys. Consult that provider’s documented value types/ranges: allowlisting a key does not validate all upstream semantics. Backboard restricts generation to compatible endpoints rather than silently dropping controls. This is image routing, separate from conversation-model routing.

These controls apply to streaming and non-streaming, including image edits. OpenRouter image generation uses the dedicated image API except for the conditional legacy reference path below. Other image providers retain their existing generation path, but the structured configuration schema is empty: do not send nonempty OpenRouter-specific controls to them.

​

Legacy audio/video references are conditional

Audio/video documents may be used as references only on OpenRouter’s legacy chat-image path when the model advertises the relevant input modalities and image output and its tool schema exposes references. Unsupported modalities fail rather than being silently omitted. Mixed image/audio/video references require support for every supplied modality; capability flags alone are not evidence that a combination has been tested end to end.

The merged caller and assistant image_config must be empty. Any nonempty configuration, including routing controls, is incompatible with audio/video references. Request no custom settings and inspect tool arguments: caller {} does not stop the assistant from choosing controls. Use input_reference_document_ids, not the image-only base argument or the speech-to-text audio_file input.

This exception does not apply to the dedicated configured image API or stateless images, whose references are images only. No specific currently available model is guaranteed here to support legacy audio/video-to-image generation.

​

Streaming and non-streaming results

Streaming can emit a media_generated event for each successfully stored artifact:

{
  "type": "media_generated",
  "run_id": "run_123",
  "media": {
    "document_id": "a5750392-06da-47d0-a540-6caa99a0ef91",
    "media_type": "image",
    "mime_type": "image/png",
    "file_size_bytes": 904100,
    "url": "https://..."
  }
}

This is a completed stored image notification, not native partial-image rendering. If storage fails, that artifact has no media_generated event; inspect tool outputs and errors. Event order can vary with the tool loop. A successful conversation response without image metadata may mean the assistant did not call the tool or generation failed.

Non-streaming image metadata appears in generated_media on response messages. Python exposes message dictionaries and a latest-message generated_media convenience property; TypeScript exposes message.generatedMedia. Media object keys and SSE events remain snake_case. Collect across messages for multi-output turns, and save document_id rather than parsing URLs from assistant prose.

​

Related pages

• Python and TypeScript Image Tool Examples
• Stateless Images
• Tool Calling
• Threads