- Core Concepts
- Send Message
Core Concepts
Send Message
How to send a message, stream the response, and understand every field in the request and response.
Overview
POST /threads/messages is the primary endpoint for interacting with Backboard. Send a user message, get the assistant’s response. You can start a new conversation (omit thread_id) or continue an existing one (pass thread_id).
Request
Endpoint
POST https://app.backboard.io/api/threads/messages
Headers
| Header | Value |
|---|---|
X-API-Key | Your API key |
Content-Type | application/json (or multipart/form-data for file uploads) |
Body parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
content | string | — | Text content of the user message |
thread_id | string | — | Omit to auto-create a new thread; pass to continue an existing one |
assistant_id | string | — | Pin the thread to a specific assistant (shares its memory, documents, config) |
system_prompt | string | — | Instructions for this turn. Falls back to the assistant’s stored description if omitted |
tools | array | — | Tool definitions for this turn (OpenAI-style function calling format) |
stream | boolean | false | Stream the response as SSE events |
llm_provider | string | "openai" | Provider (openai, anthropic, google, xai, openrouter, etc.) |
model_name | string | "gpt-4o" | Model name (e.g. gpt-4o, claude-sonnet-4-20250514) |
memory | string | "off" | Memory Lite: "Auto", "Readonly", or "off" |
memory_pro | string | null | Memory Pro (higher accuracy): "Auto" or "Readonly". Cannot use with memory |
memory_response_citation | boolean | false | Include memory citations in the reply |
web_search | string | "off" | "Auto" to enable real-time web search |
json_output | boolean | false | Request JSON object output. Ignored when tools, RAG, or web search are active |
thinking | object | null | Enable reasoning (e.g. {"effort": "high"} for OpenAI, {"budget_tokens": 5000} for Anthropic) |
openrouter | object | null | OpenRouter only: which upstream provider serves the request (e.g. {"sort": "price"}). See below |
send_to_llm | string | "true" | Set to "false" to store the message without generating a response |
metadata | string | null | Optional metadata as a JSON string |
files | binary[] | — | File attachments (multipart/form-data only) |
Supported attachment types: .pdf, .doc(x), .ppt(x), .xls(x), .txt, .csv, .md, .json(l), .xml, .py, .js, .ts, .jsx, .tsx, .html, .css, .cpp, .c, .h, .java, .go, .rs, .rb, .php, .sql, .png, .jpg, .jpeg, .webp, .gif, .bmp, .tiff, .tif
Non-streaming response
When stream is false (or omitted), you receive a single JSON response after the model finishes.
import requests
headers = {"X-API-Key": "YOUR_API_KEY", "Content-Type": "application/json"}
response = requests.post(
"https://app.backboard.io/api/threads/messages",
headers=headers,
json={
"content": "What is machine learning?",
"llm_provider": "openrouter",
"model_name": "moonshotai/kimi-k2.6",
},
)
result = response.json()
print("Reply:", result["content"])
print("Thread ID:", result["thread_id"])
print("Tokens:", result["total_tokens"])
Response object
| Field | Type | Description |
|---|---|---|
content | string | The assistant’s reply text |
message_id | uuid | Unique message identifier |
thread_id | uuid | Thread this message belongs to |
assistant_id | uuid | Assistant used for this response |
role | string | user, assistant, or tool |
status | string | COMPLETED, REQUIRES_ACTION, IN_PROGRESS, FAILED, CANCELLED |
tool_calls | array | Tool call requests when status is REQUIRES_ACTION |
run_id | string | Run identifier (used for legacy tool output submission) |
reasoning | string | The model’s reasoning trace (when thinking is enabled) |
model_provider | string | Provider used for this response |
model_name | string | Model used for this response |
input_tokens | integer | Tokens sent to the model |
output_tokens | integer | Tokens generated by the model |
total_tokens | integer | Sum of input + output tokens |
memory_operation_id | string | ID to poll memory save status (when memory is active) |
retrieved_memories | array | Memories retrieved for this response |
retrieved_files | array | Document filenames used as context |
retrieved_files_count | integer | Number of documents retrieved |
attachments | array | File attachments on this message |
context_usage | object | Context window utilization (see below) |
timestamp | datetime | When the message was created |
Context usage
Every response includes a context_usage object:
| Field | Description |
|---|---|
percent | Percentage of context used (0–100) |
used_tokens | Total tokens currently in context |
context_limit | Model’s maximum context window |
summary_tokens | Tokens used by conversation summary (if active) |
model | Model used for this calculation |
When percent approaches 100%, Backboard automatically summarizes older messages to free up context space.
Streaming response (SSE)
When stream=true, the response is a stream of Server-Sent Events. Each event is a JSON object on a data: line.
SSE event types
| Event type | When it fires | Key fields |
|---|---|---|
run_started | The run has begun | run_id, message_id, provider, model_name |
message_start | The assistant message is about to stream | message_id, run_id |
content_streaming | Each piece of the assistant’s text | content |
reasoning_streaming | Each piece of reasoning (when thinking is enabled) | content |
reasoning_ended | Reasoning phase complete, content about to begin | — |
tool_call_start | A tool call’s name is known, arguments still streaming | run_id, tool_call_id, name |
tool_call_ready | A tool call’s arguments have finished streaming | run_id, tool_call |
tool_submit_required | Model is requesting one or more tool calls | tool_calls, run_id, model_provider, model_name, usage fields |
run_ended | The run finished successfully | thread_id, assistant_id, status, usage fields |
run_failed | The run failed | run_id, error, error_type |
error | The request could not be processed | error |
Handle run_failed and error explicitly. A stream that ends without run_ended has failed, and a client that only waits for run_ended will hang.
This table covers the events most clients need. Others exist (for example media_generated), so always ignore event types you do not recognize. New event types are additive, and clients that skip unknown types keep working across API updates.
thread_id is present on run_ended and user_message, not on every event. If you need it elsewhere, carry it from the request or from the first event that includes it.
Early tool-call events
tool_call_start and tool_call_ready let you show a tool call in your UI — or begin executing it — before the model finishes its turn.
{"type": "tool_call_start", "run_id": "run_abc", "tool_call_id": "call_1", "name": "get_weather"}
{
"type": "tool_call_ready",
"run_id": "run_abc",
"tool_call": {
"id": "call_1",
"type": "function",
"function": {"name": "get_weather", "arguments": "{\"city\": \"Paris\"}"}
}
}
When the provider streams a tool-call id, tool_call_start arrives before tool_call_ready. Some providers do not reveal an id until the arguments are complete — in that case tool_call_start is skipped, only tool_call_ready fires, and its tool_call.id is null. Each event fires at most once per call.
These events are advisory. tool_submit_required remains the authoritative list of tool calls you must respond to. Always reconcile against it — a call announced early may not appear in the final list. Submit outputs based on tool_submit_required, never on tool_call_ready alone.
Tools that Backboard runs internally, such as document search, web search, and image generation, are never announced. You only see calls you will be asked to execute.
Usage and cost fields
run_ended and tool_submit_required both carry usage telemetry:
| Field | Type | Description |
|---|---|---|
input_tokens | integer | Fresh (non-cached) prompt tokens, billed at the model’s full input rate. Can be 0 when the entire prompt was served from cache. |
output_tokens | integer | Generated tokens |
total_tokens | integer | input_tokens + output_tokens |
cached_input_tokens | integer | Prompt tokens served from the provider’s prompt cache at a discounted rate |
cache_write_input_tokens | integer | Prompt tokens written to cache at a premium rate (Anthropic and Bedrock only — always 0 on providers with free automatic caching, such as OpenAI) |
cost_usd | float | null | Cost for the run in USD, or null when the model has no published price |
Example: handling SSE events
import json
import requests
headers = {"X-API-Key": "YOUR_API_KEY", "Content-Type": "application/json"}
response = requests.post(
"https://app.backboard.io/api/threads/messages",
headers=headers,
json={
"content": "Tell me a fun fact about space.",
"stream": True,
},
stream=True,
)
for line in response.iter_lines():
if not line:
continue
decoded = line.decode()
if not decoded.startswith("data: "):
continue
event = json.loads(decoded[6:])
event_type = event.get("type")
if event_type == "content_streaming":
print(event.get("content", ""), end="", flush=True)
elif event_type == "reasoning_streaming":
# Only present when thinking is enabled
print(f"[thinking] {event.get('content', '')}", end="", flush=True)
elif event_type == "reasoning_ended":
print("\n--- reasoning done ---")
elif event_type == "tool_submit_required":
# The model wants you to call tools — see Tool Calling
print(f"\nTool calls requested: {event.get('tool_calls')}")
elif event_type == "run_ended":
print(f"\nDone. Tokens: {event.get('total_tokens')}")
Choosing a model per message
Override the model on any message — you can switch freely within the same thread:
# Turn 1: use OpenAI
response_1 = requests.post(
"https://app.backboard.io/api/threads/messages",
headers=headers,
json={
"content": "Explain quantum computing",
"llm_provider": "openai",
"model_name": "gpt-4o",
},
).json()
# Turn 2: switch to Anthropic on the same thread
response_2 = requests.post(
"https://app.backboard.io/api/threads/messages",
headers=headers,
json={
"thread_id": response_1["thread_id"],
"content": "Now simplify that explanation",
"llm_provider": "anthropic",
"model_name": "claude-sonnet-4-20250514",
},
).json()
Defaults to openai / gpt-4o if llm_provider and model_name are not specified.
OpenRouter options
Everything works with just llm_provider and model_name. OpenRouter is the one
provider with an extra choice to make: it is a gateway, so it serves each model
from several upstream providers — Together, Fireworks, DeepInfra and others — and
each charges its own rate. By default it picks for you.
Add an optional openrouter object when you want a say in that pick:
response = requests.post(
"https://app.backboard.io/api/threads/messages",
headers=headers,
json={
"content": "Summarize this thread",
"llm_provider": "openrouter",
"model_name": "deepseek/deepseek-chat-v3.1",
"openrouter": {"sort": "price"}, # or pin one: {"providers": ["Together"]}
},
).json()
| Field | Type | Description |
|---|---|---|
providers | string[] | Upstream providers to try, in order, e.g. ["Together", "Fireworks"] |
allow_fallbacks | boolean | false pins the request to providers — it fails rather than moving to another upstream (and another price) |
sort | string | Rank upstreams by "price", "throughput", or "latency" |
ignore | string[] | Upstream providers to exclude |
max_price | object | Ceiling in USD per 1M tokens: {"prompt": 1.0, "completion": 2.0} |
allowed_models | string[] | Auto Router only — restrict by wildcard, e.g. ["anthropic/*"] |
excluded_models | string[] | Auto Router only — exclude by wildcard |
cost_tier | string | Auto Router only — "low", "medium", "high", "xhigh", "max" |
GET /billing/models/providers?model={model_id} lists every upstream serving a
model with its rates and uptime. Any upstream_provider value from there can go
straight into providers.
With the SDKs
from backboard import BackboardClient
async with BackboardClient(api_key="YOUR_API_KEY") as client:
response = await client.add_message(
thread_id=thread_id,
content="Summarize this thread",
llm_provider="openrouter",
model_name="moonshotai/kimi-k3",
openrouter={"providers": ["together"], "allow_fallbacks": False},
)
Pass the endpoint tag (baseten/fp8) rather than the display name when a
provider serves several endpoints — the tag names one endpoint, and therefore
one price.
Letting OpenRouter pick the model too
Set model_name to openrouter/auto and it chooses the model per prompt. You
pay the standard rate of whatever it picks — there is no router fee.
{
"llm_provider": "openrouter",
"model_name": "openrouter/auto",
"openrouter": {"cost_tier": "low"}
}
The response tells you what actually ran: resolved_model and
upstream_provider appear alongside cost_usd.
What you are billed
The exact amount the serving upstream charged for your request — not an estimate, and for tool-using turns the sum of every leg. The rates shown in the model library are indicative, useful for choosing; the bill comes from the response.
Prompt caches live on the upstream provider and are not shared between them, so
a request that moves to a different upstream starts cold. Backboard keeps each
thread pinned to the provider that served it, but note that sort and
max_price can move you between upstreams as prices change — pin with
providers when warm caches matter more than shaving the rate.
Message with file attachments
Use multipart form-data to attach files inline with a message:
import requests
response = requests.post(
"https://app.backboard.io/api/threads/messages",
headers={"X-API-Key": "YOUR_API_KEY"},
data={
"content": "Summarize this document",
"stream": "false",
},
files=[("files", open("report.pdf", "rb"))],
)
print(response.json()["content"])
Memory modes
| Mode | Saves? | Retrieves? | Cost | Description |
|---|---|---|---|---|
memory="off" | No | No | — | Only uses conversation history |
memory="Auto" | Yes | Yes | Low | Memory Lite — saves and retrieves memories |
memory="Readonly" | No | Yes | Low | Only retrieves saved memories |
memory_pro="Auto" | Yes | Yes | Higher | Memory Pro — higher accuracy retrieval |
memory_pro="Readonly" | No | Yes | Higher | Pro retrieval only |
memory and memory_pro cannot be used together in the same message.
JSON output mode
Set json_output=true to request a structured JSON response:
response = requests.post(
"https://app.backboard.io/api/threads/messages",
headers=headers,
json={
"content": "List 3 planets as JSON with name and diameter_km fields",
"json_output": True,
},
).json()
print(response["content"]) # Valid JSON string
json_output is automatically ignored when RAG (documents), web search, or custom tools are active on the message.
Not all models support JSON output. Use the Models API with supports_json_output=true to find compatible models.
Voice (TTS & STT)
You can add voice capabilities to any message. Send audio for speech to text, get audio back for text to speech, or both at once.
See Voice (TTS & STT) for file uploads, speech output, and the live STT pipeline. Use Realtime Audio for a model that listens and speaks directly.
Thinking (reasoning)
Enable the model to reason step-by-step before answering:
response = requests.post(
"https://app.backboard.io/api/threads/messages",
headers=headers,
json={
"content": "Why can't you stop on a one-way street if you're walking?",
"thinking": {"effort": "high"},
"llm_provider": "openai",
"model_name": "o3",
},
).json()
print("Reasoning:", response["reasoning"])
print("Answer:", response["content"])
For supported fields per provider and streaming examples, see Thinking.
Related
- Add Message
- Submit Tool Outputs
- First Message (SDK) — send your first message with the SDK
- Tool Calling — handle tool call requests
- Web Search
- Thinking — reasoning configuration per provider
- Voice
- Realtime Audio
- Models — available models and providers