Commercial SupportGuaranteed response times on Shiny bug fixes, straight from the maintainers. Plans from US$1,500 a year.See the plans

AI Conversation

The AI conversation bridge gives the page conversations with the app’s AI model. Replies stream as events, the AI can ask questions with choices, sessions are stored and resumable, and the assistant remembers things across them. A hands-free voice loop (wake word, push-to-talk, talking over a reply) is optional. The model, its key, its tools and its prompts stay in the app; the page never sees them.

Install and register

dotnet add package Shiny.AppDeviceBridge.AiConversation

builder.UseAppDeviceBridge(
    bridge => bridge.AddAiConversationBridge(
        ai => ai
            .SetChatClientProvider<MyChatClientProvider>()   // or register an IChatClient
            .AddSessionTitles(),
        o => o.Voice = true                                   // optional: the voice loop
    ),
    webApp => { /* … */ }
);

The first argument configures Shiny.AiConversation itself: the model, stores, tools, memory and interceptors. Without a provider it uses the IChatClient registered in the container. The in-memory stores are the default; register the SQLite stores from Shiny.AiConversation.Stores.SqliteDocDb to keep sessions across launches.

Client	C#	TypeScript
`IAiConversationBridge`	`AddAiConversationBridgeClient()`	`AiConversationBridge`

Details

Built on Shiny.AiConversation 3.0.

Topic	Details
Sessions	`POST sessions` starts one, `GET sessions/latest` carries on the most recent (`204` when there are none), `GET sessions` lists and searches them, and `DELETE sessions/{id}` deletes one with its messages and memory. A session never ends: it is compacted as its context fills.
Sending	`POST sessions/{id}/messages` takes `text`, or an `answer` to a question with choices, plus up to 4 `images` from file roots and per-turn `instructions`. It answers with the `turnId` straight away; with `"wait": true` it answers once the turn ends, with the reply. A new message supersedes a turn still running, and `DELETE sessions/{id}/turn` interrupts it.
Streaming	`ai.delta` carries each piece of the reply, `ai.message` each stored or changed message, and `ai.turn.ended` how the turn ended (`Completed`, `AwaitingUser`, `Interrupted`, `Superseded`, `Failed`, with the error). Every event names its session.
Open sessions	Events come from sessions the bridge has open: starting, reading or sending to one opens it. It keeps `MaxOpenSessions` (8) open and closes the least recently used idle one beyond that; using it again reopens it.
Questions	When the model asks something, `ai.questions` and the session’s `pendingQuestions` carry it. Answer a question with choices with `answer: { questionId, choiceIds }` (one id unless `allowMultiple`), and one without choices with text. A question no longer pending answers `409 no_such_question`.
History	`GET sessions/{id}/messages?before=&count=` pages backwards. Each message has its display `text`, what was actually `spokenText` when it was interrupted, and its `parts`: text, tool calls and results, answers and images. An image part’s `attachmentId` fetches it from `GET attachments?id=`.
Memory	`GET memory`, `PUT memory` and `DELETE memory` read, write and forget what the assistant remembers, globally, per profile or per session; `ai.memory` fires when the assistant changes it. Writes from the page are recorded as the user’s.
Voice	With `o.Voice = true`, `POST voice` attaches the voice loop to a session, with an optional `wakeWord`, interruption mode, quiet words, follow-up timeout and culture. It needs a listener on `ai.voice.state` to attach (`409 not_listening`) and detaches once the last one is gone, so a page that went away can’t leave the microphone waiting for a wake word. `voice/talk`, `voice/stop`, `voice/pause` and `voice/resume` drive it, and `ai.voice.transcript`, `ai.voice.speaking`, `ai.voice.interrupted` and `ai.voice.level` report it. A loop the app attached itself answers `409 voice_busy`.
Platforms	Sessions and memory work on every platform. The voice loop uses Shiny.Speech’s on-device recognizer and synthesizer on iOS, Mac Catalyst, macOS, Android and Windows; Linux has none, so register a cloud or Whisper `ISpeechToTextService` and `ITextToSpeechService` there. Without AI conversations registered, the bridge answers `501`; without the voice loop, its routes do.
Platform setup	For voice: Android needs `RECORD_AUDIO`. Apple platforms need `NSSpeechRecognitionUsageDescription` and `NSMicrophoneUsageDescription`, plus the `com.apple.security.device.audio-input` entitlement for sandboxed apps.

From the page

import { AiConversationBridge } from "@shinyorg/appdevicebridge";

const ai = new AiConversationBridge();
const session = (await ai.getLatestSession()) ?? (await ai.startSession({}));

await ai.onDelta(e => e.sessionId === session.id && (reply.textContent += e.text));
await ai.onQuestions(e => showChoices(e.questions));
await ai.onTurnEnded(e => e.error && showError(e.error));

await ai.send(session.id, { text: "Plan a dinner for four" });

// Or wait for the whole reply.
const { reply } = await ai.send(session.id, { text: "Make it vegetarian", wait: true });

// Answering a question with choices.
await ai.send(session.id, { answer: { questionId: "size", choiceIds: ["large"] } });

Requests

GET/_bridge/ai#

What the app registered.

C#IAiConversationBridge.GetStatusAsyncTSAiConversationBridge.getStatus

Response

200 AiStatus

AiStatus · 3 fields

`supported`	`boolean`	Whether the app registered AI conversations.
`voiceSupported`	`boolean`	Whether the app registered the voice loop.
`openSessions`	`number`	Sessions the bridge holds open, and raises events for.

GET/_bridge/ai/sessions#

Stored sessions, most recently active first, filtered by text in their messages, a tag or a profile.

C#IAiConversationBridge.GetSessionsAsyncTSAiConversationBridge.getSessions

Parameter	In	Type
`text`	query	`string \| null`	optional
`tag`	query	`string \| null`	optional
`profile`	query	`string \| null`	optional
`skip`	query	`number`	optional
`take`	query	`number`	optional

Response

200 AiSessionSummary[]

AiSessionSummary · 8 fields

`id`	`string`
`title`	`string \| null`
`started`	`string`
`lastActivity`	`string`
`messageCount`	`number`
`profile`	`string \| null`
`tags`	`string[]`
`usage`	`AiUsage`

POST/_bridge/ai/sessions#

Starts a session.

C#IAiConversationBridge.StartSessionAsyncTSAiConversationBridge.startSession

Body

application/json · AiStartSessionRequest

AiStartSessionRequest · 5 fields

`title`	`string \| null`
`profile`	`string \| null`	Selects the app's tools, memory and prompts for a kind of conversation.
`tags`	`string[] \| null`
`metadata`	`Record<string, string> \| null`
`excludeFromSearch`	`boolean`	Keep it out of conversation search and long-term memory.

Response

200 AiSession

AiSession · 9 fields

`id`	`string`
`title`	`string \| null`
`started`	`string`
`state`	`AiSessionState`
`pendingQuestions`	`AiQuestion[]`
`usage`	`AiUsage`
`context`	`AiContext`
`profile`	`string \| null`
`tags`	`string[]`

GET/_bridge/ai/sessions/latest#

The most recently active session — to carry on where the user left off — or null when there are none.

C#IAiConversationBridge.GetLatestSessionAsyncTSAiConversationBridge.getLatestSession

Response

200 AiSession · or 204 when there is none

AiSession · 9 fields

`id`	`string`
`title`	`string \| null`
`started`	`string`
`state`	`AiSessionState`
`pendingQuestions`	`AiQuestion[]`
`usage`	`AiUsage`
`context`	`AiContext`
`profile`	`string \| null`
`tags`	`string[]`

GET/_bridge/ai/sessions/{id}#

A session. Fails with 404 when there is none.

C#IAiConversationBridge.GetSessionAsyncTSAiConversationBridge.getSession

Parameter	In	Type
`id`	path	`string`	required

Response

200 AiSession

AiSession · 9 fields

`id`	`string`
`title`	`string \| null`
`started`	`string`
`state`	`AiSessionState`
`pendingQuestions`	`AiQuestion[]`
`usage`	`AiUsage`
`context`	`AiContext`
`profile`	`string \| null`
`tags`	`string[]`

Errors

404Not found

DELETE/_bridge/ai/sessions/{id}#

Deletes a session, its messages and its memory.

C#IAiConversationBridge.DeleteSessionAsyncTSAiConversationBridge.deleteSession

Parameter	In	Type
`id`	path	`string`	required

Response: 204 No content

PUT/_bridge/ai/sessions/{id}/title#

Renames a session.

C#IAiConversationBridge.SetTitleAsyncTSAiConversationBridge.setTitle

Parameter	In	Type
`id`	path	`string`	required

Body

application/json · AiTitleRequest

AiTitleRequest · 1 field

title string

Response

200 AiSession

AiSession · 9 fields

`id`	`string`
`title`	`string \| null`
`started`	`string`
`state`	`AiSessionState`
`pendingQuestions`	`AiQuestion[]`
`usage`	`AiUsage`
`context`	`AiContext`
`profile`	`string \| null`
`tags`	`string[]`

POST/_bridge/ai/sessions/{id}/messages#

Sends a message, replacing any turn still running. The reply streams through onDelta and ends with onTurnEnded; with wait, this answers with it too.

C#IAiConversationBridge.SendAsyncTSAiConversationBridge.send

Parameter	In	Type
`id`	path	`string`	required

Body

application/json · AiSendRequest

AiSendRequest · 5 fields

`text`	`string \| null`	At most 32,000 characters.
`answer`	`AiAnswer \| null`
`images`	`AiImageRef[] \| null`	At most 4, each a JPEG, PNG, WebP, HEIC or GIF of at most 10 MB.
`instructions`	`string[] \| null`	Extra direction for this turn only, not shown in the history.
`wait`	`boolean`	Answer once the turn ends, with its reply, rather than as soon as it starts.

Response

200 AiSendResult

AiSendResult · 5 fields

`turnId`	`string`
`outcome`	`AiTurnOutcome \| null`	Null when it is still running.
`reply`	`AiMessage \| null`
`questions`	`AiQuestion[] \| null`
`error`	`string \| null`

GET/_bridge/ai/sessions/{id}/messages#

The stored history, newest page first: pass the oldest sequence of a page as before for the one before it.

C#IAiConversationBridge.GetMessagesAsyncTSAiConversationBridge.getMessages

Parameter	In	Type
`id`	path	`string`	required
`before`	query	`number \| null`	optional
`count`	query	`number`	optional

Response

200 AiMessagePage

AiMessagePage · 2 fields

`messages`	`AiMessage[]`	Oldest first.
`hasMore`	`boolean`	Older messages remain; ask again with the first one's sequence.

DELETE/_bridge/ai/sessions/{id}/turn#

Interrupts the running turn, keeping what was written. Fails with 409 no_turn when none is running.

C#IAiConversationBridge.InterruptAsyncTSAiConversationBridge.interrupt

Parameter	In	Type
`id`	path	`string`	required

Response: 204 No content
Errors: 409Conflict

POST/_bridge/ai/sessions/{id}/compact#

Folds older messages into the session's summary now, rather than when the context fills.

C#IAiConversationBridge.CompactAsyncTSAiConversationBridge.compact

Parameter	In	Type
`id`	path	`string`	required

Response: 204 No content

GET/_bridge/ai/attachments#

An image attached to a message, by its part's attachmentId.

C#IAiConversationBridge.GetAttachmentAsyncTSAiConversationBridge.getAttachment

Parameter	In	Type
`id`	query	`string`	required

Response: 200 Binary content — Blob in TypeScript, Stream / byte[] in C#

GET/_bridge/ai/memory#

What the assistant remembers, optionally in one scope.

C#IAiConversationBridge.GetMemoryAsyncTSAiConversationBridge.getMemory

Parameter	In	Type
`scope`	query	`AiMemoryScope \| null`	optional
`scopeKey`	query	`string \| null`	optional

Response

200 AiMemoryDocument[]

AiMemoryDocument · 9 fields

`path`	`string`
`scope`	`AiMemoryScope`
`scopeKey`	`string \| null`	The session id or profile name; null for `Global`.
`description`	`string \| null`
`content`	`string`
`created`	`string`
`updated`	`string`
`updatedBy`	`AiMemoryAuthor`
`version`	`number`

PUT/_bridge/ai/memory#

Writes a memory document, as the user.

C#IAiConversationBridge.SaveMemoryAsyncTSAiConversationBridge.saveMemory

Body

application/json · AiMemorySaveRequest

AiMemorySaveRequest · 5 fields

`scope`	`AiMemoryScope`
`path`	`string`
`content`	`string`
`scopeKey`	`string \| null`	The session id or profile name; null for `Global`.
`description`	`string \| null`

Response

200 AiMemoryDocument

AiMemoryDocument · 9 fields

`path`	`string`
`scope`	`AiMemoryScope`
`scopeKey`	`string \| null`	The session id or profile name; null for `Global`.
`description`	`string \| null`
`content`	`string`
`created`	`string`
`updated`	`string`
`updatedBy`	`AiMemoryAuthor`
`version`	`number`

DELETE/_bridge/ai/memory#

Forgets one memory document.

C#IAiConversationBridge.DeleteMemoryAsyncTSAiConversationBridge.deleteMemory

Parameter	In	Type
`scope`	query	`AiMemoryScope`	required
`path`	query	`string`	required
`scopeKey`	query	`string \| null`	optional

Response: 204 No content

GET/_bridge/ai/voice#

Whether the voice loop is there, and where it is.

C#IAiConversationBridge.GetVoiceAsyncTSAiConversationBridge.getVoice

Response

200 AiVoiceStatus

AiVoiceStatus · 3 fields

`supported`	`boolean`	Whether the app registered the voice loop.
`state`	`AiVoiceState`
`sessionId`	`string \| null`	The session it is attached to.

POST/_bridge/ai/voice/access#

Requests microphone and speech recognition permission.

C#IAiConversationBridge.RequestVoiceAccessAsyncTSAiConversationBridge.requestVoiceAccess

Response

200 AiAccessResult

AiAccessResult · 1 field

access AccessState

POST/_bridge/ai/voice#

Attaches the voice loop to a session. Listen to onVoiceState first: without a listener it fails with 409 not_listening, and it detaches once the last listener is gone, so a page that went away cannot leave the microphone listening for a wake word.

C#IAiConversationBridge.AttachVoiceAsyncTSAiConversationBridge.attachVoice

Body

application/json · AiVoiceRequest

AiVoiceRequest · 7 fields

`sessionId`	`string`
`wakeWord`	`string \| null`	Listen for this before each conversation; without one, the loop starts on `pushToTalk`.
`interruption`	`AiInterruptionMode \| null`
`quietWords`	`string[] \| null`	What stops a reply under `QuietWords`; at most 32.
`followUpTimeoutSeconds`	`number \| null`	How long it listens for an answer to a question before needing the wake word again.
`acknowledgement`	`AiAcknowledgement \| null`
`culture`	`string \| null`	Recognition and speech, such as `en-US`.

Response

200 AiVoiceStatus

AiVoiceStatus · 3 fields

`supported`	`boolean`	Whether the app registered the voice loop.
`state`	`AiVoiceState`
`sessionId`	`string \| null`	The session it is attached to.

Errors

409Conflict

DELETE/_bridge/ai/voice#

Detaches the voice loop and releases the microphone.

C#IAiConversationBridge.DetachVoiceAsyncTSAiConversationBridge.detachVoice

Response: 204 No content

POST/_bridge/ai/voice/talk#

Listens now, without the wake word, interrupting any reply.

C#IAiConversationBridge.PushToTalkAsyncTSAiConversationBridge.pushToTalk

Response: 204 No content

POST/_bridge/ai/voice/stop#

Stops the reply being spoken, without listening.

C#IAiConversationBridge.StopSpeakingAsyncTSAiConversationBridge.stopSpeaking

Response: 204 No content

POST/_bridge/ai/voice/pause#

C#IAiConversationBridge.PauseVoiceAsyncTSAiConversationBridge.pauseVoice

Response: 204 No content

POST/_bridge/ai/voice/resume#

C#IAiConversationBridge.ResumeVoiceAsyncTSAiConversationBridge.resumeVoice

Response: 204 No content

Events

EVENTai.turn.started#

C#IAiConversationBridge.OnTurnStartedAsyncTSAiConversationBridge.onTurnStarted

Payload: AiTurnStarted

`sessionId`	`string`
`turnId`	`string`
`userMessage`	`AiMessage`

EVENTai.delta#

C#IAiConversationBridge.OnDeltaAsyncTSAiConversationBridge.onDelta

Payload: AiTextDelta

`sessionId`	`string`
`turnId`	`string`
`text`	`string`

EVENTai.tool#

C#IAiConversationBridge.OnToolCallAsyncTSAiConversationBridge.onToolCall

Payload: AiToolCall

`sessionId`	`string`
`turnId`	`string`
`callId`	`string`
`name`	`string`
`completed`	`boolean`
`isError`	`boolean`
`silent`	`boolean`	A tool the app marked as not worth showing.

EVENTai.questions#

C#IAiConversationBridge.OnQuestionsAsyncTSAiConversationBridge.onQuestions

Payload: AiQuestionsAsked

`sessionId`	`string`
`turnId`	`string`
`questions`	`AiQuestion[]`

EVENTai.turn.ended#

C#IAiConversationBridge.OnTurnEndedAsyncTSAiConversationBridge.onTurnEnded

Payload: AiTurnEnded

`sessionId`	`string`
`turnId`	`string`
`outcome`	`AiTurnOutcome`
`replyMessageId`	`string \| null`
`spokenText`	`string \| null`	What was heard before an interruption.
`error`	`string \| null`

EVENTai.message#

A message was stored or changed.

C#IAiConversationBridge.OnMessageAsyncTSAiConversationBridge.onMessage

Payload: AiMessageChanged

`sessionId`	`string`
`message`	`AiMessage`
`updated`	`boolean`	True when an existing message changed — finished streaming, interrupted, compacted.

EVENTai.state#

C#IAiConversationBridge.OnStateAsyncTSAiConversationBridge.onState

Payload: AiStateChanged

`sessionId`	`string`
`state`	`AiSessionState`

EVENTai.usage#

C#IAiConversationBridge.OnUsageAsyncTSAiConversationBridge.onUsage

Payload: AiUsageChanged

`sessionId`	`string`
`usage`	`AiUsage`
`context`	`AiContext`

EVENTai.title#

C#IAiConversationBridge.OnTitleAsyncTSAiConversationBridge.onTitle

Payload: AiTitleChanged

`sessionId`	`string`
`title`	`string \| null`

EVENTai.compacted#

C#IAiConversationBridge.OnCompactedAsyncTSAiConversationBridge.onCompacted

Payload: AiCompacted

`sessionId`	`string`
`compactedCount`	`number`
`wasHardTrim`	`boolean`	Messages were dropped without a summary, because summarizing failed or there was no room.

EVENTai.memory#

The assistant changed something it remembers.

C#IAiConversationBridge.OnMemoryAsyncTSAiConversationBridge.onMemory

Payload: AiMemoryChanged

`sessionId`	`string`
`document`	`AiMemoryDocument`
`change`	`AiMemoryChange`

EVENTai.error#

C#IAiConversationBridge.OnErrorAsyncTSAiConversationBridge.onError

Payload: AiSessionError

`sessionId`	`string`
`message`	`string`

EVENTai.voice.state#

C#IAiConversationBridge.OnVoiceStateAsyncTSAiConversationBridge.onVoiceState

Payload: AiVoiceStateChanged

state AiVoiceState

EVENTai.voice.transcript#

C#IAiConversationBridge.OnTranscriptAsyncTSAiConversationBridge.onTranscript

Payload: AiTranscript

`text`	`string`
`final`	`boolean`	False while the user is still speaking.

EVENTai.voice.speaking#

C#IAiConversationBridge.OnSpeakingAsyncTSAiConversationBridge.onSpeaking

Payload: AiSpeaking

text string

EVENTai.voice.interrupted#

C#IAiConversationBridge.OnInterruptedAsyncTSAiConversationBridge.onInterrupted

Payload: AiSpeechInterrupted

`reason`	`AiInterruptionReason`
`heardText`	`string \| null`
`spokenText`	`string \| null`

EVENTai.voice.level#

The microphone and speech levels about 20 times a second.

C#IAiConversationBridge.OnVoiceLevelAsyncTSAiConversationBridge.onVoiceLevel

Payload: AiVoiceLevel

`input`	`number`	The microphone, 0–1.
`output`	`number`	The speech, 0–1.

EVENTai.voice.error#

C#IAiConversationBridge.OnVoiceErrorAsyncTSAiConversationBridge.onVoiceError

Payload: AiVoiceError

message string

Types

JSON property names are camelCase and enums travel as their names. A ? marks a property the payload may omit;| null one that may be null.

AiStatusobject#

`supported`	`boolean`	Whether the app registered AI conversations.
`voiceSupported`	`boolean`	Whether the app registered the voice loop.
`openSessions`	`number`	Sessions the bridge holds open, and raises events for.

AiSessionSummaryobject#

A stored session, without loading it.

`id`	`string`
`title`	`string \| null`
`started`	`string`
`lastActivity`	`string`
`messageCount`	`number`
`profile`	`string \| null`
`tags`	`string[]`
`usage`	`AiUsage`

AiUsageobject#

Everything billed in a session.

`inputTokens`	`number`
`outputTokens`	`number`
`cachedInputTokens`	`number`
`reasoningTokens`	`number`
`totalTokens`	`number`
`requestCount`	`number`

AiStartSessionRequestobject#

A new session. Every field is optional.

`title?`	`string \| null`
`profile?`	`string \| null`	Selects the app's tools, memory and prompts for a kind of conversation.
`tags?`	`string[] \| null`
`metadata?`	`Record<string, string> \| null`
`excludeFromSearch?`	`boolean`	Keep it out of conversation search and long-term memory.

AiSessionobject#

A session as it is right now.

`id`	`string`
`title`	`string \| null`
`started`	`string`
`state`	`AiSessionState`
`pendingQuestions`	`AiQuestion[]`
`usage`	`AiUsage`
`context`	`AiContext`
`profile`	`string \| null`
`tags`	`string[]`

AiSessionStateenum#

"Idle""Thinking""Responding""AwaitingUser""Compacting"

AiQuestionobject#

Something the AI asked, with the answers to pick from when it gave some.

`id`	`string`
`text`	`string`
`choices`	`AiChoice[] \| null`
`allowMultiple`	`boolean`

AiChoiceobject#

`id`	`string`
`label`	`string`

AiContextobject#

How full the model's context window is.

`currentTokens`	`number`
`maxTokens`	`number \| null`
`percentUsed`	`number \| null`
`pressure`	`AiContextPressure \| null`
`isEstimate`	`boolean`	True when counted locally rather than reported by the model.

AiContextPressureenum#

"Normal""Warning""Critical"

AiTitleRequestobject#

title string

AiSendRequestobject#

A message to the AI: text, or an answer to a question with choices. A question without choices is answered with text.

`text?`	`string \| null`	At most 32,000 characters.
`answer?`	`AiAnswer \| null`
`images?`	`AiImageRef[] \| null`	At most 4, each a JPEG, PNG, WebP, HEIC or GIF of at most 10 MB.
`instructions?`	`string[] \| null`	Extra direction for this turn only, not shown in the history.
`wait?`	`boolean`	Answer once the turn ends, with its reply, rather than as soon as it starts.

AiAnswerobject#

An answer to one of pendingQuestions by its choices.

`questionId`	`string`
`choiceIds`	`string[]`

AiImageRefobject#

An image to send, from a file root — one the page wrote, a photo, a camera capture.

`file`	`BridgeFile`
`description?`	`string \| null`	What it shows, for models and history that cannot see it.
`retention?`	`AiImageRetention`

BridgeFileobject#

A file the page can reach through the files bridge: a named root and a path inside it. Bridges that take or return a file use this, so nothing reaches the page that /_bridge/files would not also serve.

`root`	`string`
`path`	`string`

AiImageRetentionenum#

What happens to an image after the turn that sent it.

"DropAfterTurn""KeepInContext""DontStore"

AiSendResultobject#

A turn that started, or with wait, one that ended.

`turnId`	`string`
`outcome`	`AiTurnOutcome \| null`	Null when it is still running.
`reply`	`AiMessage \| null`
`questions`	`AiQuestion[] \| null`
`error`	`string \| null`

AiTurnOutcomeenum#

How a turn ended.

"Completed""AwaitingUser""Interrupted""Superseded""Failed"

AiMessageobject#

`id`	`string`
`sessionId`	`string`
`sequence`	`number`
`timestamp`	`string`
`role`	`AiMessageRole`
`source`	`AiMessageSource`
`status`	`AiMessageStatus`
`text`	`string \| null`	What to show.
`spokenText`	`string \| null`	What was actually heard, when it was spoken and interrupted.
`parts`	`AiMessagePart[]`
`compacted`	`boolean`	Folded into a summary; no longer sent to the model.

AiMessageRoleenum#

"User""Assistant""Tool""System""Summary"

AiMessageSourceenum#

"Typed""Voice""ChoiceAnswer""Camera""Interceptor""Compaction""MemoryFlush""Model"

AiMessageStatusenum#

"Complete""Streaming""Interrupted""Failed"

AiMessagePartobject#

One part of a message. Which fields are set depends on kind.

`kind`	`AiPartKind`
`displayed`	`boolean`	Shown in the conversation.
`spoken`	`boolean`	Read aloud by the voice loop.
`text`	`string \| null`	The text; a tool call's arguments as JSON; a tool's result; an answer's labels.
`name`	`string \| null`
`callId`	`string \| null`
`isError`	`boolean`
`attachmentId`	`string \| null`	An image's id for `getAttachment`.
`mediaType`	`string \| null`
`questionId`	`string \| null`
`choiceIds`	`string[] \| null`

AiPartKindenum#

"Text""ToolCall""ToolResult""Answer""Image"

AiMessagePageobject#

`messages`	`AiMessage[]`	Oldest first.
`hasMore`	`boolean`	Older messages remain; ask again with the first one's sequence.

AiMemoryScopeenum#

Whose memory a document is: one session's, one profile's, or every session's.

"Session""Profile""Global"

AiMemoryDocumentobject#

Something the assistant remembers, as a markdown document at a path.

`path`	`string`
`scope`	`AiMemoryScope`
`scopeKey`	`string \| null`	The session id or profile name; null for `Global`.
`description`	`string \| null`
`content`	`string`
`created`	`string`
`updated`	`string`
`updatedBy`	`AiMemoryAuthor`
`version`	`number`

AiMemoryAuthorenum#

"Assistant""User""App"

AiMemorySaveRequestobject#

`scope`	`AiMemoryScope`
`path`	`string`
`content`	`string`
`scopeKey?`	`string \| null`	The session id or profile name; null for `Global`.
`description?`	`string \| null`

AiVoiceStatusobject#

`supported`	`boolean`	Whether the app registered the voice loop.
`state`	`AiVoiceState`
`sessionId`	`string \| null`	The session it is attached to.

AiVoiceStateenum#

"Off""WaitingForWakeWord""Listening""FollowUp""Thinking""Speaking""Paused"

AiAccessResultobject#

access AccessState

AccessStateenum#

Where a permission stands. Reported by every bridge that needs one; the names match Shiny's own.

"Unknown""NotSupported""NotSetup""Disabled""Restricted""Denied""Available"

AiVoiceRequestobject#

Attaches the voice loop to a session. Every field but the session is optional.

`sessionId`	`string`
`wakeWord?`	`string \| null`	Listen for this before each conversation; without one, the loop starts on `pushToTalk`.
`interruption?`	`AiInterruptionMode \| null`
`quietWords?`	`string[] \| null`	What stops a reply under `QuietWords`; at most 32.
`followUpTimeoutSeconds?`	`number \| null`	How long it listens for an answer to a question before needing the wake word again.
`acknowledgement?`	`AiAcknowledgement \| null`
`culture?`	`string \| null`	Recognition and speech, such as `en-US`.

AiInterruptionModeenum#

How the user can talk over a reply.

"Off""QuietWords""Full"

AiAcknowledgementenum#

What the voice loop does to acknowledge it heard the user.

"None""AudioBlip""LessWordy""Full"

AiTurnStartedobject#

`sessionId`	`string`
`turnId`	`string`
`userMessage`	`AiMessage`

AiTextDeltaobject#

The next piece of the reply as the model writes it.

`sessionId`	`string`
`turnId`	`string`
`text`	`string`

AiToolCallobject#

The model called one of the app's tools, or one finished.

`sessionId`	`string`
`turnId`	`string`
`callId`	`string`
`name`	`string`
`completed`	`boolean`
`isError`	`boolean`
`silent`	`boolean`	A tool the app marked as not worth showing.

AiQuestionsAskedobject#

`sessionId`	`string`
`turnId`	`string`
`questions`	`AiQuestion[]`

AiTurnEndedobject#

`sessionId`	`string`
`turnId`	`string`
`outcome`	`AiTurnOutcome`
`replyMessageId`	`string \| null`
`spokenText`	`string \| null`	What was heard before an interruption.
`error`	`string \| null`

AiMessageChangedobject#

`sessionId`	`string`
`message`	`AiMessage`
`updated`	`boolean`	True when an existing message changed — finished streaming, interrupted, compacted.

AiStateChangedobject#

`sessionId`	`string`
`state`	`AiSessionState`

AiUsageChangedobject#

`sessionId`	`string`
`usage`	`AiUsage`
`context`	`AiContext`

AiTitleChangedobject#

`sessionId`	`string`
`title`	`string \| null`

AiCompactedobject#

`sessionId`	`string`
`compactedCount`	`number`
`wasHardTrim`	`boolean`	Messages were dropped without a summary, because summarizing failed or there was no room.

AiMemoryChangedobject#

`sessionId`	`string`
`document`	`AiMemoryDocument`
`change`	`AiMemoryChange`

AiMemoryChangeenum#

"Created""Updated""Deleted""Renamed"

AiSessionErrorobject#

`sessionId`	`string`
`message`	`string`

AiVoiceStateChangedobject#

state AiVoiceState

AiTranscriptobject#

`text`	`string`
`final`	`boolean`	False while the user is still speaking.

AiSpeakingobject#

The sentence the voice loop is speaking now.

text string

AiSpeechInterruptedobject#

`reason`	`AiInterruptionReason`
`heardText`	`string \| null`
`spokenText`	`string \| null`

AiInterruptionReasonenum#

"QuietWord""Answer""NewSpeech""PushToTalk""Stop"

AiVoiceLevelobject#

`input`	`number`	The microphone, 0–1.
`output`	`number`	The speech, 0–1.

AiVoiceErrorobject#

message string

Security

These routes sit under AppDeviceBridgePolicies.Bridges like every other bridge: callers on this device by default. The page spends the app’s model, and every tool the app registers runs when the model decides to call it, on a prompt the page wrote. Register only the tools the web app may use, and treat a page that can reach this bridge as able to call them. See Security.