agents: document the asr config section and strict language

agents/build/configuration.mdx

@@ -34,7 +34,7 @@
 
 ## Voice
 
-The Voice panel selects the voice your agent speaks with (`voice_id`) and its speaking language (`speaking_language`, one of the [52 supported languages](/agents/build/voice-language#speaking-language)). It also holds the speech recognition model (`asr_model`), the multilingual recognition switch (`multilingual_asr`), and expressive delivery (`expressive`). See [Voice & language](/agents/build/voice-language) for all of them.
+The Voice panel selects the voice your agent speaks with (`voice_id`) and its speaking language (`speaking_language`, one of the [52 supported languages](/agents/build/voice-language#speaking-language)). It also holds expressive delivery (`expressive`) and the speech recognition settings, which the API keeps in their own `asr` section: the recognition model (`asr.model`) and the multilingual recognition switch (`asr.multilingual`). See [Voice & language](/agents/build/voice-language) for all of them.
 
 ## Conversation settings
 
@@ -59,7 +59,7 @@
   - `low` (**Hard to interrupt** in the Builder): The agent stops only for a firm, worded interjection, talking through background noise and short acknowledgements.
   - `balanced` (default): Interrupts on normal speech.
   - `high` (**Easy to interrupt**): The agent stops on the caller's first word.
 - `conversation.interruption_ignore_phrases` (up to 50 entries of 40 characters): Backchannels such as `"uh-huh"` or `"okay"` that never interrupt the agent when the caller says only them. Matching ignores case and punctuation. Send `[]` to clear the list.
 
 ### Call duration
 
@@ -77,7 +77,7 @@
 
 `conversation.timezone` is the default IANA timezone (like `Asia/Tokyo`) the agent uses for dates and times in conversation. Leave it empty for **automatic**: each session follows the caller's device or phone number, falling back to UTC. Set one when your agent serves a single region regardless of who calls. A per-session `timezone` on the [session request](/agents/build/time-timezone) overrides this. See [Time & timezone](/agents/build/time-timezone) for the full resolution order.
 
 ## Autosave and publishing
 
 There is no Save button. Each change is written to the agent's draft moments after you stop editing, and the **Saving… / Saved** indicator at the bottom-left of the page shows the current state. If a save fails, the Builder tells you and keeps your pending edits so nothing is lost.
 
@@ -106,6 +106,12 @@
     "voice_id": "802e3bc2b27e49c2995d23ef70e6ac89",
     "speaking_language": "en"
   },
+  "asr": {
+    "model": "deepgram:nova-3",
+    "multilingual": false,
+    "strict_language": false,
+    "keyterms": []
+  },
   "conversation": {
     "max_duration_seconds": 1800,
     "response_wait_ms": 550,
@@ -144,6 +150,7 @@
 |---|---|---|
 | `prompt` | System prompt and first-message settings | This page |
 | `voice` | Voice profile, speaking language, and expressive mode | [Voice & language](/agents/build/voice-language) |
+| `asr` | Speech recognition model, multilingual recognition, strict language, and recognition keyterms | [Voice & language](/agents/build/voice-language#speech-recognition-model) |
 | `conversation` | Call duration, turn-taking, interruption behavior, timezone, and storage | This page; storage in [Conversation history](/agents/monitor/conversation-history#what-gets-stored) |
 | `tools` | Attached webhook tools and system tool switches | [Tools](/agents/build/tools) |
 | `knowledge_base` | Attached knowledge sources | [Knowledge base](/agents/build/knowledge-base) |

agents/build/voice-language.mdx

@@ -106,13 +106,13 @@
 
 ## Speech recognition model
 
-**Speech recognition** picks the model that transcribes what callers say (`voice.asr_model` on the wire). Latency is the typical wait from the caller's last word to the transcript. The agent starts its reply after that.
+**Speech recognition** picks the model that transcribes what callers say (`asr.model` on the wire). Latency is the typical wait from the caller's last word to the transcript. The agent starts its reply after that.
 
 | Value | Model | Latency | Notes |
 |---|---|---|---|
 | `deepgram:nova-3` | Deepgram Nova-3 | ≈0.4 s | Default. Mid-call language switching covers English, Spanish, French, German, Hindi, Russian, Portuguese, Japanese, Italian, and Dutch. |
 | `elevenlabs:scribe_v2_realtime` | ElevenLabs Scribe v2 Realtime | ≈0.9 s | Keyterms longer than 20 characters are ignored. |
 | `elevenlabs:scribe_v2_medical` | ElevenLabs Scribe v2 Medical | ≈1.5 s | Tuned for clinical speech: medication names, anatomy, and pathology terms. Keyterms longer than 50 characters are ignored. |
 
 Choose it with the **Speech recognition** selector in the Builder's voice section, or via the API:
 
@@ -122,8 +122,8 @@
   --header "Authorization: Bearer $FISH_API_KEY" \
   --header "Content-Type: application/json" \
   --data '{
-    "voice": {
-      "asr_model": "elevenlabs:scribe_v2_realtime"
+    "asr": {
+      "model": "elevenlabs:scribe_v2_realtime"
     }
   }'
 ```
@@ -135,25 +135,52 @@
 
 **Multilingual recognition** lets the agent understand callers who switch to another language partway through a conversation. With it off, speech recognition listens only for the speaking language. That is more accurate when every caller speaks the same language, and it keeps short or accented phrases from being transcribed as another language.
 
-It is off by default. Toggle it with the **Multilingual recognition** switch in the Builder's voice section, or via the API (`voice.multilingual_asr`, default `false`):
+It is off by default. Toggle it with the **Multilingual recognition** switch in the Builder's voice section, or via the API (`asr.multilingual`, default `false`):
 
 <CodeGroup>
 ```bash API (curl)
 curl --request PATCH "https://api.fish.audio/v1/agent/agents/$AGENT_ID/config" \
   --header "Authorization: Bearer $FISH_API_KEY" \
   --header "Content-Type: application/json" \
   --data '{
-    "voice": {
-      "multilingual_asr": true
+    "asr": {
+      "multilingual": true
+    }
+  }'
+```
+</CodeGroup>
+
+<Note>
+  Which languages a caller can switch between depends on the [speech recognition model](#speech-recognition-model). With Deepgram Nova-3, a speaking language outside its switching set stays locked even with the setting on. With ElevenLabs Scribe v2, turning the setting off steers recognition toward the speaking language rather than locking it. To lock it, use [strict language](#strict-language).
+</Note>
+
+## Strict language
+
+**Strict language** keeps the agent from understanding any language other than its speaking language. When speech recognition detects that the caller spoke another language, the agent receives `[unintelligible speech]` instead of the transcript and answers as if it did not catch what was said. It applies whether multilingual recognition is on or off.
+
+It is off by default. Turn it on via the API (`asr.strict_language`, default `false`):
+
+<CodeGroup>
+```bash API (curl)
+curl --request PATCH "https://api.fish.audio/v1/agent/agents/$AGENT_ID/config" \
+  --header "Authorization: Bearer $FISH_API_KEY" \
+  --header "Content-Type: application/json" \
+  --data '{
+    "asr": {
+      "strict_language": true
     }
   }'
 ```
 </CodeGroup>
 
 <Note>
-  Which languages a caller can switch between depends on the [speech recognition model](#speech-recognition-model). With Deepgram Nova-3, a speaking language outside its switching set stays locked even with the setting on. With ElevenLabs Scribe v2, turning the setting off steers recognition toward the speaking language rather than locking it.
+  With the ElevenLabs Scribe v2 models, speech in another language reaches the agent as `[unintelligible speech]`. With Deepgram Nova-3, recognition runs a model for the speaking language only, so speech in another language is usually not transcribed at all and the agent keeps listening.
 </Note>
 
+<Warning>
+  Language is detected per utterance, so a short or heavily accented phrase in the speaking language can occasionally be detected as another language and replaced with `[unintelligible speech]` too. Only turn on strict language when you explicitly need to stop the agent from understanding other languages.
+</Warning>
+
 ## Speaking speed
 
 **Speaking speed** sets how fast the agent talks, as a multiplier from `0.5` (half speed) to `2.0` (double speed). The default is `1.0`. Many English-language agents sound more natural at a slightly faster rate, such as `1.2`.

api-reference/openapi.json

@@ -3613,7 +3613,10 @@
                       "$ref": "#/components/schemas/AgentPromptConfig"
                     },
                     "voice": {
-                      "$ref": "#/components/schemas/AgentVoiceConfig"
+                      "$ref": "#/components/schemas/AgentVoiceConfigView"
+                    },
+                    "asr": {
+                      "$ref": "#/components/schemas/AgentAsrConfig"
                     },
                     "conversation": {
                       "$ref": "#/components/schemas/AgentConversationConfig"
@@ -3647,6 +3650,7 @@
                     "config_hash",
                     "prompt",
                     "voice",
+                    "asr",
                     "conversation",
                     "tools",
                     "webhooks",
@@ -3779,7 +3783,7 @@
       },
       "patch": {
         "summary": "Update Draft Config",
-        "description": "Patch the draft configuration section by section; omitted sections keep\ntheir value. Changes only affect live sessions after the next publish.\n`prompt.system_prompt` is limited to 32000 tokens (422 beyond); keeping it\nunder 2000 tokens is recommended for latency and cost.\n`voice.voice_id` accepts any public voice model id.\n`voice.speaking_language` accepts any of the 52 supported ISO 639-1 codes\n(the same set the console offers, see the Voice & language docs); anything else is 422. `voice.expressive` (default `true`) enables richer\nexpressive delivery (emotion steering, laughter and sounds, pauses); off\nkeeps the standard delivery. `voice.keyterms` is a speech-recognition vocabulary of\nup to 50 plain terms (brand names, product terms, personal names), each at\nmost 100 characters with no commas or semicolons; `[]` clears it and 20-50\nfocused terms work best. `tool_ids` and\n`knowledge_source_ids` replace their attachment lists wholesale and every\nid must resolve, else 422. `llm.custom` points the agent at your own\nOpenAI-compatible endpoint; mutually exclusive with `llm.model`, cleared\nwith an explicit null.",
+        "description": "Patch the draft configuration section by section; omitted sections keep\ntheir value. Changes only affect live sessions after the next publish.\n`prompt.system_prompt` is limited to 32000 tokens (422 beyond); keeping it\nunder 2000 tokens is recommended for latency and cost.\n`voice.voice_id` accepts any public voice model id.\n`voice.speaking_language` accepts any of the 52 supported ISO 639-1 codes\n(the same set the console offers, see the Voice & language docs); anything else is 422. `voice.expressive` (default `true`) enables richer\nexpressive delivery (emotion steering, laughter and sounds, pauses); off\nkeeps the standard delivery. The `asr` section configures speech\nrecognition. `asr.model` picks the model: `deepgram:nova-3` (default),\n`elevenlabs:scribe_v2_realtime`, or `elevenlabs:scribe_v2_medical`.\n`asr.multilingual` (default `false`) lets recognition follow callers who\nswitch language mid-call, off tells the recognizer to expect\n`voice.speaking_language` (models other than `deepgram:nova-3` may still\ntranscribe clear speech in another language). `asr.strict_language`\n(default `false`) enforces `voice.speaking_language` whatever\n`asr.multilingual` says: speech recognized as another language reaches the\nagent as `[unintelligible speech]`. Language is detected per utterance, so\na short or heavily accented phrase in the speaking language can\noccasionally be detected as another language and replaced too. Only\nenable it when you explicitly need to stop the agent from understanding\nother languages.\n`asr.keyterms` is a recognition\nvocabulary of up to 50 plain terms (brand names, product terms, personal\nnames), each at most 100 characters with no commas or semicolons. `[]`\nclears it and 20-50 focused terms work best. `tool_ids` and\n`knowledge_source_ids` replace their attachment lists wholesale and every\nid must resolve, else 422. `llm.custom` points the agent at your own\nOpenAI-compatible endpoint; mutually exclusive with `llm.model`, cleared\nwith an explicit null.",
         "security": [
           {
             "BearerAuth": []
@@ -3829,7 +3833,10 @@
                       "$ref": "#/components/schemas/AgentPromptConfig"
                     },
                     "voice": {
-                      "$ref": "#/components/schemas/AgentVoiceConfig"
+                      "$ref": "#/components/schemas/AgentVoiceConfigView"
+                    },
+                    "asr": {
+                      "$ref": "#/components/schemas/AgentAsrConfig"
                     },
                     "conversation": {
                       "$ref": "#/components/schemas/AgentConversationConfig"
@@ -3863,6 +3870,7 @@
                     "config_hash",
                     "prompt",
                     "voice",
+                    "asr",
                     "conversation",
                     "tools",
                     "webhooks",
@@ -16022,6 +16030,71 @@
         "title": "PublicAgentAnalysisSummaryPatch",
         "type": "object"
       },
+      "PublicAgentAsrPatch": {
+        "additionalProperties": false,
+        "properties": {
+          "model": {
+            "anyOf": [
+              {
+                "enum": [
+                  "deepgram:nova-3",
+                  "elevenlabs:scribe_v2_realtime",
+                  "elevenlabs:scribe_v2_medical"
+                ],
+                "type": "string"
+              },
+              {
+                "type": "null"
+              }
+            ],
+            "default": null,
+            "title": "Model"
+          },
+          "multilingual": {
+            "anyOf": [
+              {
+                "type": "boolean"
+              },
+              {
+                "type": "null"
+              }
+            ],
+            "default": null,
+            "title": "Multilingual"
+          },
+          "strict_language": {
+            "anyOf": [
+              {
+                "type": "boolean"
+              },
+              {
+                "type": "null"
+              }
+            ],
+            "default": null,
+            "description": "Enforces voice.speaking_language whatever multilingual says: speech recognized as another language reaches the agent as [unintelligible speech]. Language is detected per utterance, so a short or heavily accented phrase in the speaking language can occasionally be detected as another language and replaced too. With deepgram:nova-3, speech in another language is usually not transcribed at all. Only enable it when you explicitly need to stop the agent from understanding other languages.",
+            "title": "Strict Language"
+          },
+          "keyterms": {
+            "anyOf": [
+              {
+                "items": {
+                  "type": "string"
+                },
+                "maxItems": 50,
+                "type": "array"
+              },
+              {
+                "type": "null"
+              }
+            ],
+            "default": null,
+            "title": "Keyterms"
+          }
+        },
+        "title": "PublicAgentAsrPatch",
+        "type": "object"
+      },
       "PublicAgentConfigPatchPayload": {
         "additionalProperties": false,
         "properties": {
@@ -16047,6 +16120,17 @@
             ],
             "default": null
           },
+          "asr": {
+            "anyOf": [
+              {
+                "$ref": "#/components/schemas/PublicAgentAsrPatch"
+              },
+              {
+                "type": "null"
+              }
+            ],
+            "default": null
+          },
           "conversation": {
             "anyOf": [
               {
@@ -16725,22 +16809,6 @@
             "default": null,
             "title": "Expressive"
           },
-          "keyterms": {
-            "anyOf": [
-              {
-                "items": {
-                  "type": "string"
-                },
-                "maxItems": 50,
-                "type": "array"
-              },
-              {
-                "type": "null"
-              }
-            ],
-            "default": null,
-            "title": "Keyterms"
-          },
           "speed": {
             "anyOf": [
               {
@@ -17158,6 +17226,41 @@
         "title": "AgentAnalysisSummaryConfig",
         "type": "object"
       },
+      "AgentAsrConfig": {
+        "properties": {
+          "model": {
+            "default": "deepgram:nova-3",
+            "enum": [
+              "deepgram:nova-3",
+              "elevenlabs:scribe_v2_realtime",
+              "elevenlabs:scribe_v2_medical"
+            ],
+            "title": "Model",
+            "type": "string"
+          },
+          "multilingual": {
+            "default": false,
+            "title": "Multilingual",
+            "type": "boolean"
+          },
+          "strict_language": {
+            "default": false,
+            "description": "Enforces voice.speaking_language whatever multilingual says: speech recognized as another language reaches the agent as [unintelligible speech]. Language is detected per utterance, so a short or heavily accented phrase in the speaking language can occasionally be detected as another language and replaced too. With deepgram:nova-3, speech in another language is usually not transcribed at all. Only enable it when you explicitly need to stop the agent from understanding other languages.",
+            "title": "Strict Language",
+            "type": "boolean"
+          },
+          "keyterms": {
+            "items": {
+              "type": "string"
+            },
+            "maxItems": 50,
+            "title": "Keyterms",
+            "type": "array"
+          }
+        },
+        "title": "AgentAsrConfig",
+        "type": "object"
+      },
       "AgentConversationConfig": {
         "properties": {
           "max_duration_seconds": {
@@ -17526,7 +17629,8 @@
         "title": "AgentTransferOnFailure",
         "type": "object"
       },
-      "AgentVoiceConfig": {
+      "AgentVoiceConfigView": {
+        "description": "Read shape of the voice section. The ASR fields that moved to the asr\nsection stay mirrored here, deprecated, so clients written before the split\nkeep reading them.",
         "properties": {
           "voice_id": {
             "default": "b347db033a6549378b48d00acb0d06cd",
@@ -17597,14 +17701,6 @@
             "title": "Expressive",
             "type": "boolean"
           },
-          "keyterms": {
-            "items": {
-              "type": "string"
-            },
-            "maxItems": 50,
-            "title": "Keyterms",
-            "type": "array"
-          },
           "speed": {
             "default": 1,
             "maximum": 2,
@@ -17613,7 +17709,7 @@
             "type": "number"
           }
         },
-        "title": "AgentVoiceConfig",
+        "title": "AgentVoiceConfigView",
         "type": "object"
       },
       "PublicAgentKnowledgeBaseConfig": {
@@ -17785,7 +17881,10 @@
             "$ref": "#/components/schemas/AgentPromptConfig"
           },
           "voice": {
-            "$ref": "#/components/schemas/AgentVoiceConfig"
+            "$ref": "#/components/schemas/AgentVoiceConfigView"
+          },
+          "asr": {
+            "$ref": "#/components/schemas/AgentAsrConfig"
           },
           "conversation": {
             "$ref": "#/components/schemas/AgentConversationConfig"
@@ -17819,6 +17918,7 @@
           "config_hash",
           "prompt",
           "voice",
+          "asr",
           "conversation",
           "tools",
           "webhooks",