> ## Documentation Index
> Fetch the complete documentation index at: https://koreai.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Speech Customization (Call Control Parameters)

[Back to Voice Gateway Advanced Configuration](https://koreai.mintlify.site/ai-for-service/channels.md#advanced-configuration)

This section provides in-depth information on customizing speech behavior using call control parameters.

## Introduction to Call Control Parameters

Call control parameters are general-purpose parameters that can modify a call's behavior, including ASR/STT & TTS configurations.

> [!NOTE]
>
> Automatic Speech Recognition (ASR) and Speech-to-Text (STT) are two terms that refer to the same technology. Both involve converting spoken language into written text by analyzing and interpreting audio input. The terms are used interchangeably, describing the same function—transforming speech into readable, actionable text.

There are two ways to define the Call Control Parameters - Node Level and Channel Level.

You can apply Call Control Parameters at either the Session or Node level, offering more flexibility in managing call behavior.

* **Session-Level Parameters**: Add the prefix `session.` to apply parameters throughout the session (for example, `session.ttsprovider`).
* **Node-Level Parameters**: Add the prefix node. to apply parameters only at a specific node (for example, `node.ttsprovider`).
* **Default Behavior**: The platform considers parameters without a prefix as session-level by default.
* Node-level parameters take precedence over session-level parameters. If no node-level parameters are defined, session-level properties apply.

## Node Level Call Control

The call control section is Available In [Entity Node](https://koreai.mintlify.site/ai-for-service/automation/dialogs/node-types/entity-node)/[Message Node](https://koreai.mintlify.site/ai-for-service/automation/dialogs/node-types/message-nodes#ivr-properties)/[Confirmation Node](https://koreai.mintlify.site/ai-for-service/automation/dialogs/node-types/confirmation-nodes#ivr-properties) > IVR Properties > Advanced Controls. [Learn more](https://koreai.mintlify.site/ai-for-service/automation/agent-flows.md#configuring-grammar).

![Node Level Call Control](https://mintcdn.com/koreai/VaEjvvsSenY-Yvtg/ai-for-service/channels/voice-gateway/images/node-level-call-control-parameters.png?fit=max\&auto=format\&n=VaEjvvsSenY-Yvtg\&q=85\&s=bf8923b243164891a1d07720362618eb)

## Channel Level Call Control

## Update/Modify Parameters

When updating language settings or modifying Automatic Speech Recognition (ASR) and Text-to-Speech (TTS) parameters in Call Control Parameters, users can specify the updated field along with a minimal set of required parameters.

For example, if a user has configured the Speech-to-Text (STT) provider and language in the call control parameters and wants to add existing values. Users only need to provide the additional sttLanguage parameter without redefining the previously set values.

This behavior applies to Session-Level Call Control Parameters.

**Example**

**Existing Parameters:**

```
{
  "sttProvider": "microsoft",
  "sttLanguage": "en-IN"
}
```

**Adding a New Language:**

```
{
  "sttLanguage": "en-ES"
}
```

In this scenario, the system retains the existing `sttProvider` and previously set `sttLanguage`, ensuring that only the new parameter adds without requiring users to re-enter unchanged values.

## Supported Speech Engines (ASR/TTS)

Voice Gateway supports several third-party service providers for ASR/TTS. [Learn more](https://koreai.mintlify.site/ai-for-service/channels/voice-gateway/third-party-asr-tts-support)

## Supported Call Control Parameters

### Provider Related Parameters

Speech-to-text (STT) and text-to-speech (TTS) services interface with the user using a selected language (for example, English US, English UK, or German).

TTS services also use a selected voice (for example, female or male) to respond.

* Use STT for recognizer:
  `sttProvider` → `google`, `microsoft`, `sarvam`.

* Use TTS for synthesizer:
  `ttsProvider` → `google`, `microsoft`, `aws`, `sarvam`, `openai`.

> [!NOTE]
>
> Apply the following parameters only when STT is set as Recognizer. Otherwise, the system uses default bot-level or Voice Gateway settings. These properties are applied at the session level.

**Examples**

```json theme={null}
{
  "sttProvider": "google",
  "sttLanguage": "en-IN",
  "ttsProvider": "google",
  "ttsLanguage": "en-IN",
  "voiceName": "en-IN-Wavenet-A"
}
```

| Parameter | Type | Description | Example Values |
| :- | :- | :- | :- |
| `sttLabel` | String | Identifies the ASR engine configured in Voice Gateway. | `my_azure-US` |
| `sttFallbackLabel` | String | Defines the fallback ASR label used when the primary engine fails. | `my_azure_Europe` |
| `sttFallbackProvider` | String | Specifies the fallback ASR provider. | `microsoft` |
| `sttFallbackLanguage` | String | Specifies the fallback ASR language. | `en-US` |
| `ttsLabel` | String | Identifies the TTS engine configured in Voice Gateway. | `my_azure-US` |
| `ttsFallbackLabel` | String | Defines the fallback TTS label used when the primary engine fails. | `my_azure_Europe` |
| `ttsFallbackProvider` | String | Specifies the fallback TTS provider. | `microsoft` |
| `ttsFallbackLanguage` | String | Specifies the fallback TTS language. | `en-US` |
| `ttsFallbackVoiceName` | String | Specifies the fallback voice used for TTS output. | `en-US-AmberNeural` |
| `enableTTSChunking` | Boolean | Splits sentences based on the following punctuations `full stop (.)` and `question mark (?)` | `true` |
| `preflightThreshold` | Number | Sets the confidence threshold for early speech detection before full turn completion. | `0.6` |
| `eotThreshold` | Number | Defines the confidence score required to trigger an End-of-Turn event. | `0.8` |
| `eagerEotThreshold` | Number | Sets the threshold for early End-of-Turn detection to start response generation sooner. | `0.5` |
| `eotTimeoutMs` | Number | Specifies the maximum silence duration before forcing an End-of-Turn event. | `3000` |
| `mipOptOut` | Boolean | Disables participation in model improvement programs using audio and transcripts. | `true` |
| `entityPrompt` | String | Provides contextual hints to improve recognition of domain-specific terms. | `"flight number, booking reference"` |
| `node.alternativeLanguages` | Array | Defines alternate languages, providers, and voices for dynamic switching during conversations. | `See example below` |
| `azureAudioLogging` | Boolean | Enables or disables audio logging for Azure Speech services. | `true` |
| `notifySttLatency` | Boolean | Enables capturing the STT Latency Values. | `true` |
| `noiseIsolationEnable` | Boolean | Turns on noise isolation preprocessing before speech recognition. Required when `turnDetectionEnable` is true in the current phase. | `true` |
| `noiseIsolationVendor` | String | Specifies the noise isolation vendor. Required when `noiseIsolationEnable` is true. | `true` |

Example for `node.alternativeLanguages`.

```
node.alternativeLanguages: [
  {
    "language": "es-ES",
    "voiceName": "es-ES-ArabellaMultilingualNeural"
  },
  {
    "language": "en-IN",
    "voiceName": "en-IN-PrabhatNeural",
    "ttsLabel": "microsoft2",
    "ttsProvider": "microsoft",
    "sttProvider": "microsoft2"
  }
]
```

### Use Cases and Benefits

| Use case | Parameter | Benefit |
| - | - | - |
| Reduce voice bot response latency | `enableTTSChunking` | Enables chunking of long bot responses so the system starts audio playback before processing the full text, improving time-to-first-audio and conversational flow. |
| Improve turn detection in voice conversations | `eotThreshold`, `eagerEotThreshold`, `eotTimeoutMs` | Controls how the system detects when a user finishes speaking, reducing interruptions, delays, and overlapping speech. |
| Enable faster AI response generation | `eagerEotThreshold` | Lets the system begin generating a response before the user fully stops speaking, improving responsiveness. |
| Improve speech recognition for domain terms | `entityPrompt` | Provides hints for domain-specific terms, such as order IDs or product names, to improve recognition accuracy. |
| Enable multi-language voice bots | `sttProvider`, `deepgramModel` | Uses Deepgram Flux to support callers who switch languages mid-call, handling recognition for all supported languages within a single uninterrupted ASR session. |
| Enable automatic mid-call language switching | `sttProvider`, `deepgramModel` | Uses Deepgram's flux-general-multi model to automatically detect and switch between supported languages during a call without restarting the ASR stream, supporting 10 languages. |
| Support Indian language voice automation | `sttProvider`, `ttsProvider`, `ttsLanguage`, `sttLanguage`, `voiceName` | Uses Sarvam ASR and TTS to build voice bots for Indian language speakers. Configure speech rate and variation through `ttsOptions`. |
| Privacy and compliance control | `mipOptOut` | Prevents the system from using customer audio data for model training. |
| Debugging and quality monitoring | `azureAudioLogging` | Enables logging to help analyze recognition errors, audio quality, and language detection issues during testing and troubleshooting. |
| Handle caller silence in Realtime LLM sessions. | `userNoInputTimeoutMS`, `botNoInputTimeoutMS`, `botNoInputSpeech`, `botNoInputRetries`, `botNoInputGiveUpTimeoutMS` | Plays a re-engagement nudge when the caller is silent after the LLM finishes its turn, and handles LLM response delays - preventing abandoned calls from staying connected indefinitely. |

## Cloned Voices

### Configure a Custom Cloned Voice (ElevenLabs Integration)

Voice Gateway supports custom voices cloned in ElevenLabs. To activate a cloned voice for an agent, share the ElevenLabs API credentials with the support team. Once integration is complete, enable the voice through call control parameters-no redeployment required.

Get the ElevenLabs API key for authentication from the ElevenLabs account under **Profile > API Key**.

Setup Steps:

1. Raise a support ticket with your credentials: Submit a [Support ticket](https://www.support.kore.ai), the team completes the integration on your account and shares the new label name.
2. Activate using call control parameters: On completion of the integration, enable the cloned voice in your Voice AI Agent by passing the appropriate call control parameters in your flow configuration. No code changes or redeployment needed.

Example call control parameters to use the new cloned voice:

```json theme={null}
"ttsProvider": "elevenlabs",
"ttsLanguage": "en-IN",
"voiceName": "Paste-voice-ID-here"
"ttsLabel": "new-label-name"
```

> [!NOTE]
>
> Treat the ElevenLabs API key as a password. Share it only through secure support tickets and don't include it in screenshots, logs, or public channels.

### Labels and Fallback Provider Related Parameters

Label-Assign/Create a label only if you need to create multiple speech services from the same vendor. Then, use the label in your application to specify which service to use.

**How to Configure Label**

1. Add a speech service inside the Speech tab.
2. Select a provider and add a label with a unique name.
3. Use the same label in the call control parameter.
4. At the same node where you use fallback call control parameters, you must also pass the primary Recognizer and Synthesizer.

Examples:

```json theme={null}
{
  "sttProvider": "google",
  "sttLanguage": "en-US",
  "sttLabel": "google-stt-2"
}
```

**STT**

```json theme={null}
{
  "sttProvider": "microsoft",
  "sttLabel": "my_azure-US",
  "sttLanguage": "en-US"
}
```

**TTS**

```json theme={null}
{
  "ttsProvider": "microsoft",
  "ttsLanguage": "en-US",
  "voiceName": "en-US-AmberNeural",
  "ttsLabel": "my_azure-US"
}
```

**Fallback**

```json theme={null}
{
  "sttProvider": "microsoft",
  "sttLabel": "my_azure-US",
  "sttLanguage": "en-US",
  "ttsProvider": "microsoft",
  "ttsLanguage": "en-US",
  "voiceName": "en-US-AmberNeural",
  "ttsLabel": "my_azure-US",
  "sttFallbackProvider": "microsoft",
  "sttFallbackLanguage": "en-US",
  "sttFallbackLabel": "my_azure_Europe",
  "ttsFallbackProvider": "microsoft",
  "ttsFallbackLanguage": "en-US",
  "ttsFallbackLabel": "my_azure_Europe",
  "ttsFallbackVoiceName": "en-US-AmberNeural"
}
```

> [!NOTE]
>
> * The node at which you use fallback call control parameters must also define the primary recognizer and synthesizer.
> * Best practice: use the same ASR engine in fallback with a different label or region.
> * If the current provider fails, Voice Gateway switches to the fallback provider.
> * Fallback properties are applied at the session level.

| Parameter | Type | Description | Example Values |
| :- | :- | :- | :- |
| `sttLabel` | String | Uniquely identifies the ASR engine in Voice Gateway. | `my_azure-US` |
| `sttFallbackLabel` | String | Fallback label for ASR; switch happens on error. Prefer same vendor, different region. | `my_azure_Europe` |
| `sttFallbackProvider` | String | Specifies fallback STT provider. | `microsoft` |
| `sttFallbackLanguage` | String | Specifies fallback STT language. | `en-US` |
| `ttsLabel` | String | Uniquely identifies the TTS engine in Voice Gateway. | `my_azure-US` |
| `ttsFallbackLabel` | String | Fallback label for TTS. | `my_azure_Europe` |
| `ttsFallbackProvider` | String | Specifies fallback TTS provider. | `microsoft` |
| `ttsFallbackLanguage` | String | Specifies fallback TTS language. | `en-US` |
| `ttsFallbackVoiceName` | String | Specifies fallback voice name for TTS. | `en-US-AmberNeural` |

### Continuous ASR Related Parameters

Continuous ASR (Automatic Speech Recognition) lets the speech-to-text engine to handle user inputs like phone numbers or customer IDs that may include pauses between utterances. This improves recognition accuracy for digit or character strings.

> [!NOTE]
>
> This feature is only available for Microsoft Azure.

Microsoft Azure offers a property called `AzureSegmentationSilenceTimeout` that performs the same function as Continuous ASR. Since Azure's ASR engine handles silence detection directly (rather than the Voice Gateway), it achieves higher accuracy when merging utterances. [Learn more](https://learn.microsoft.com/en-us/azure/ai-services/speech-service/how-to-recognize-speech?pivots=programming-language-csharp#change-how-silence-is-handled).

**Application Scope**:

Both `continuousASRTimeoutInMS` and `AzureSegmentationSilenceTimeout` apply at the session level. They remain active for the entire call and can be updated at individual nodes as needed.

| Parameter | Type | Scope | Description | Example |
| :- | :- | :- | :- | :- |
| `continuousASRTimeoutInMS` | Number (milliseconds) | All | Specifies the silence duration to wait after receiving a transcript before returning the result. If another transcript is received within this timeout, the input is merged and continued. | `5000` (5 seconds) |
| `continuousASRDigits` | Character (DTMF key) | All | Specifies a DTMF key that, if pressed, immediately ends the speech recognition process and returns the gathered input. | `&` |

### Barge-In Related Parameters

Barge-In lets the Voice Gateway to detect and respond when a user interrupts the bot by speaking or entering DTMF digits while the bot responds. This enables quicker interactions by preventing users from waiting for the bot to finish speaking.

> [!NOTE]
>
> Barge-In applies at the node level.

| Parameter | Type | Scope | Description | Example |
| :- | :- | :- | :- | :- |
| `listenDuringPrompt` | Boolean (`true` or `false`) | All | If set to `false`, the bot won't listen to user speech until it finishes playing the response. Defaults to `true`. | `true` |
| `bargeInMinWordCount` | Number | All | When barge-in is enabled, this defines the minimum number of words required to interrupt the bot's speech. Defaults to `1`. | `1` |
| `bargeInOnDTMF` | Boolean (`true` or `false`) | All | If `true`, DTMF input during bot speech interrupts playback, and the system starts collecting speech input. | `true` |
| `backchannelWords` | Array of strings | All | Defines words or phrases that don't interrupt the bot when spoken during audio playback. The audio continues playing and the system sends a `speechDetected` event instead of triggering barge-in. This setting applies only during playback - after playback completes, the system treats these words as normal speech. | uh-huh, okay, got it |

### Turn Detection Parameters

Turn detection determines when a caller has finished speaking by combining voice activity detection, acoustic turn probability, streaming ASR transcripts, text-based end-of-utterance scoring, and expected-answer hints. This model is ASR-provider agnostic.

When `turnDetectionEnable` is set to `true`, the platform automatically derives expected-answer configuration from the entity selected in the Entity node. The following parameters are fetched and sent automatically:

* `turnDetectionExpectedType`
* `turnDetectionExpectedLength`
* `turnDetectionExpectedMin`
* `turnDetectionExpectedMax`
* `turnDetectionExpectedOptions`
* `turnDetectionExpectedPattern`
* `turnDetectionOpenEnded`

| Parameter | Type | Description | Example |
| :- | :- | :- | :- |
| `turnDetectionEnable` | Boolean | Turns on intelligent turn detection for the speech gather. When turned on, the platform determines when the caller has finished speaking without requiring manual speech-segmentation or delay configuration. | `true` |
| `turnDetectionPrefetch` | Boolean | Turns on speculative transcript hints so the application can begin preparing a response before the turn is committed. The application must not play the response until it receives the final committed event. Default: `false`. | `true` |
| `turnDetectionModel` | String (enum) | Selects the acoustic turn-detection model. Supported value: `smart_turn` (default). | `"smart_turn"` |
| `turnDetectionEouThreshold` | Number | Controls the balance between response speed and patience. Lower values respond faster; higher values wait longer and reduce the chance of interrupting the caller. Values map to the nearest supported profile: 0.3, 0.4, 0.5, 0.6, or 0.7. Supported Range: 0.3 - 0.7. Default: 0.5. | `0.5` |
| `turnDetectionOpenEnded` | Boolean | Indicates that the question expects a narrative or multi-part answer. Applies longer holds and prevents aggressive fast commits at sentence boundaries. Use for questions such as "Please describe the issue." Default: `false`. | `true` |
| `turnDetectionExpectedType` | String (enum) | Identifies the expected shape of the caller's answer. Supported values: `digits`, `phone`, `date`, `time`, `email`, `spelling`, `choice`, `yesno`, `number`, `regex`. Auto-derived from the configured entity type when `turnDetectionEnable` is `true`. | `"digits"` |
| `turnDetectionExpectedLength` | Integer | Specifies the exact expected number of digits or spelled characters. Applicable to `digits` and `spelling`. The answer is held while below this length and considered complete when the required length is reached. | `6` |
| `turnDetectionExpectedMin` | Number or Integer | Specifies the minimum acceptable value or length. For `digits`, the minimum number of digits. For `number`, the minimum permitted numeric value. | `4` |
| `turnDetectionExpectedMax` | Number or Integer | Specifies the maximum acceptable value or length. For `digits`, the maximum number of digits. For `number`, the maximum permitted numeric value. Reaching the maximum lets the answer be considered complete. | `6` |
| `turnDetectionExpectedOptions` | Array of strings | Specifies permitted answers for the `choice` expected type. Matching is case and punctuation-insensitive, and the option may display within a longer short response. | `["billing", "technical support", "sales"]` |
| `turnDetectionExpectedPattern` | String | Defines a regular expression for structured answers that can't be represented by the standard expected types. Applicable only when `turnDetectionExpectedType` is `regex`. The complete normalized answer must match the pattern. | `"[A-Z]{3}[- ]?\\d{6}"` |

Turn detection requires noise isolation to be configured. The minimum required parameters are:

```json theme={null}
{
  "noiseIsolationEnable": true,
  "noiseIsolationVendor": "hecttor",
  "turnDetectionEnable": true,
  "turnDetectionEouThreshold": 0.5
}
```

### Timeout Related Parameters

These parameters control how long the Voice Gateway waits for user input (speech or DTMF).

> [!NOTE]
>
> All timeout-related parameters apply at the node level.

| Parameter | Type | Scope | Description | Example |
| :- | :- | :- | :- | :- |
| `userNoInputTimeoutMS` | Number (milliseconds) | All | Maximum wait time to receive user input after the bot finishes its turn. If set to 0, Voice Gateway waits indefinitely. Applies to both dialog-based bots and Realtime Speech-to-Speech (S2S) LLM sessions. | `"userNoInputTimeoutMS": 20000 → waits 20 seconds before timeout.` |
| `dtmfCollectInterDigitTimeoutMS` | Number (milliseconds) | All | Timeout between DTMF digit inputs. If the user doesn't enter another digit within this time, the gateway submits the collected digits to the bot. | `"dtmfCollectInterDigitTimeoutMS": 3000 → waits up to 3 seconds between key presses.` |
| `dtmfCollectSubmitDigit` | Number | All | Defines a special DTMF digit that submits all collected digits immediately to the bot, bypassing timeout and max digit wait. | `"dtmfCollectSubmitDigit": 9 → pressing 9 submits the digits entered so far.` |
| `dtmfCollectMaxDigits` | Number | All | Maximum number of DTMF digits to collect. If the user enters more than this number, only the first `maxDigits` are accepted. | `"dtmfCollectMaxDigits": 5 → only the first 5 digits are accepted.` |
| `dtmfCollectminDigits` | Number | All | Minimum number of DTMF digits expected. Defaults to `1`. | `"dtmfCollectminDigits": 3 → requires at least 3 digits before submission.` |
| `dtmfCollectnumDigits` | Number | All | Exact number of DTMF digits to collect. The bot waits until this number is reached before processing. | `"dtmfCollectnumDigits": 6 → system waits until exactly 6 digits are entered.` |

## Common ASR Parameters

| Parameter | Type | Supporting STT or TTS | Description | Example |
| :- | :- | :- | :- | :- |
| `alternativeLanguages` (Session Level) | Array of Objects | Google, Microsoft, Deepgram | Array of alternative languages the user might speak. Based on the utterance, the ASR chooses from the defined options. | `"alternativeLanguages": [ { "language": "de-DE", "voiceName": "de-DE-KatjaNeural" }, { "language": "fr-FR", "voiceName": "fr-FR-DeniseNeural" } ]` |
| `alternativeLanguages` (Node Level) | Array of Objects | Google, Microsoft, Deepgram | In addition to `voiceName` and `language`, this parameter supports `sttProvider`, `ttsProvider`, and `label`. If a label is configured initially, it updates automatically when the language changes. | `node.alternativeLanguages = [ { "language": "en-ES", "voiceName": "es-ES-ArabellaMultilingualNeural" }, { "language": "en-IN", "voiceName": "en-IN-PrabhatNeural", "ttsLabel": "microsoft2", "ttsProvider": "microsoft", "sttProvider": "microsoft2" } ]` |
| `sttMinConfidence` | Number (0.1-0.9) | All | If the transcript's confidence score is less than this threshold, the input is ignored and the timeout prompt is triggered. Ensures only high-confidence transcriptions are accepted. | `"sttMinConfidence": 0.5` |
| `hints` with phrase-level boost | Array of Objects | Google | Suggests specific phrases to the STT engine to improve accuracy. You can assign a boost value per phrase. Useful for distinguishing similar-sounding words. | `"hints": [ { "phrase": "benign", "boost": 50 }, { "phrase": "malignant", "boost": 10 }, { "phrase": "biopsy", "boost": 20 } ]` |
| `hints` with `hintsBoost` | Array + Number | Google, Microsoft | Instead of boosting each phrase individually, apply a single boost value to the entire array of hints. | `"hints": ["benign", "malignant", "biopsy"], "hintsBoost": 50` |
| `sttDisablePunctuation` | Boolean | Google, Microsoft | Controls punctuation in ASR output. `false` enables punctuation (default); `true` disables it. | `"sttDisablePunctuation": true` |
| `vadEnable` | Boolean | All | If `true`, the system delays connecting to cloud recognizer until voice activity is detected. | `"vadEnable": true` |
| `vadVendor` | String | All STT engines (when `vadEnable` is true) | Specifies the Voice Activity Detection (VAD) engine to be used for detecting speech before sending audio to the STT service. This parameter works only when `vadEnable` is set to `true`. | `"vadVendor": "silero"` |
| `vadVoiceMS` | Number (milliseconds) | All | Specifies how many milliseconds of detected speech are required before connecting to cloud recognizer. Only applies if `vadEnable` is `true`. | `"vadVoiceMS": 500` |
| `vadMode` | Number (0-3) | All | Determines the sensitivity of the voice activity detector. Lower values make it more sensitive. Only applies if `vadEnable` is `true`. | `"vadMode": 2` |

### Microsoft ASR

| Parameter | Type | Supporting STT or TTS | Description | Example or More Info |
| :- | :- | :- | :- | :- |
| `azureSpeechSegmentationSilenceTimeoutMs` | Number | — | Sets timeout between phrases. Unlike continuous ASR (handled by Voice Gateway), Azure handles segmentation, improving accuracy. | `"azureSpeechSegmentationSilenceTimeoutMs": 5000` |
| `sttEndpointID` | String | — | Specifies a custom Microsoft endpoint instead of the default regional endpoint. | `"sttEndpointID": "custom-endpoint-id-12345"` |
| `azurePostProcessing` | String | — | Enables final transcript enhancements such as text normalization, punctuation, or custom formatting. | `"azurePostProcessing": "enabled"` |
| `azureSpeechRecognitionMode` | String (Enum: `AtStart`, `Continuous`) | — | - `AtStart`: Starts transcription at first detected input and stops after the utterance ends. | |
| `Continuous`: Keeps transcribing without stopping. Ideal for long sessions. | `"azureSpeechRecognitionMode": "Continuous"` | | | |
| `profanityOption` | String (Enum: `masked`, `removed`, `raw`) | — | Controls how profane words appear in the transcript. Default: `raw`. | `"profanityOption": "masked"` |
| `initialSpeechTimeoutMs` | Number (milliseconds) | — | Sets how long the system waits for the user to start speaking before timing out. | `"initialSpeechTimeoutMs": 3000` |
| `requestSnr` | Boolean | — | If set to `true`, includes signal-to-noise ratio in the response. | `"requestSnr": true` |
| `outputFormat` | String (`simple`, `detailed`) | — | Sets the transcript detail level. Default: `simple`. | `"outputFormat": "detailed"` |
| `azureAudioLogging` | Boolean | — | Enables the logging of audio data for applications utilizing Azure's speech services. | — |

### Google ASR

| Parameter | Type | Supporting STT or TTS | Description | Example or Notes |
| :- | :- | :- | :- | :- |
| `sttProfanityFilter` | Boolean | — | Enables profanity filtering in transcripts. Default: `false`. | `"sttProfanityFilter": true` |
| `singleUtterance` | Boolean | — | If `true`, returns only a single utterance/transcript. | `"singleUtterance": true` |
| `sttModel` | String | — | Specifies the speech recognition model. Default: `phone_call`. | `"sttModel": "phone_call"` |
| `sttEnhancedModel` | Boolean | — | Enables the use of enhanced models for improved accuracy. | `"sttEnhancedModel": true` |
| `words` | Boolean | — | Enables word-level timestamps in the transcript. | `"words": true` |
| `diarization` | Boolean | — | Enables speaker diarization (that's, identifying speakers). | `"diarization": true` |
| `diarizationMinSpeakers` | Number | — | Sets the minimum number of speakers expected. | `"diarizationMinSpeakers": 2` |
| `diarizationMaxSpeakers` | Number | — | Sets the maximum number of speakers expected. | `"diarizationMaxSpeakers": 5` |
| `interactionType` | String | — | Specifies interaction type (for example, `discussion`, `presentation`, `phone_call`, `voicemail`, `professionally_produced`, `voice_search`, `voice_command`, `dictation`). | `"interactionType": "phone_call"` |
| `naicsCode` | Number | — | Sets a relevant industry NAICS code to tailor recognition. | `"naicsCode": 524210` |
| `googleServiceVersion` | String (`v1`, `v2`) | — | Specifies the Google ASR API version in use. | `"googleServiceVersion": "v2"` |
| `googleRecognizerId` | String | — | Identifies a specific speech recognition model. | `"googleRecognizerId": "projects/my-project/locations/global/recognizers/default"` |
| `googleSpeechStartTimeoutMs` | Number | — | Time (in ms) to wait before detecting speech initiation timeout. | `"googleSpeechStartTimeoutMs": 2000` |
| `googleSpeechEndTimeoutMs` | Number | — | Time (in ms) of silence to wait before detecting end of speech. | `"googleSpeechEndTimeoutMs": 1000` |
| `googleEnableVoiceActivityEvents` | Boolean | — | Enables start/stop speech detection events during recognition. | `"googleEnableVoiceActivityEvents": true` |
| `googleTranscriptNormalization` | Array | — | Normalizes transcripts (for example, punctuation, casing). | `"googleTranscriptNormalization": ["punctuation", "casing"]` |

### AWS ASR

| Parameter | Type | Supporting STT or TTS | Description | Example/Notes |
| :- | :- | :- | :- | :- |
| `awsAccessKey` | String | — | AWS access key used to authenticate requests. | `"awsAccessKey": "AKIAIOSFODNN7EXAMPLE"` |
| `awsSecretKey` | String | — | Secret key used with the access key for authentication. | `"awsSecretKey": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"` |
| `awsSecurityToken` | String | — | Temporary security token for requests using AWS STS. Optional. | `"awsSecurityToken": "FQoGZXIvYXdzEDQaDJ..."` |
| `awsRegion` | String | — | AWS region for service requests (for example, `us-west-2`, `eu-central-1`). | `"awsRegion": "us-west-2"` |
| `awsVocabularyFilterName` | String | — | Vocabulary filter name used to exclude specific words or phrases from transcription. | `"awsVocabularyFilterName": "BlockedWordsFilter"` |
| `awsVocabularyFilterMethod` | String (enum) | — | Method used to handle filtered words:  • `"remove"`-Removes the word from the transcript  • `"mask"`-Replaces the word with asterisks  • `"tag"`-Tags the word for reference | `"awsVocabularyFilterMethod": "mask"` |
| `awsLanguageModelName` | String | — | Custom language model name used for domain-specific transcription. | `"awsLanguageModelName": "FinanceDomainModel"` |
| `awsPiiEntityTypes` | Array | — | List of PII entity types to detect. | `"awsPiiEntityTypes": ["NAME", "EMAIL", "SSN"]` |
| `awsPiiIdentifyEntities` | Boolean | — | Indicates whether the system detects and highlights PII in the transcript. | `"awsPiiIdentifyEntities": true` |

### Deepgram ASR

| Parameter | Type | Supporting STT or TTS | Description | Example/Notes |
| :- | :- | :- | :- | :- |
| `deepgramApiKey` | String | — | Deepgram API key used for authentication. Overrides settings in the Voice Gateway portal. | `"dg12345abcd6789xyz" (API key provided by Deepgram for authentication)` |
| `deepgramTier` | String | — | Specifies the Deepgram tier: `enhanced` or `base`. | `enhanced (higher accuracy, higher cost) / base (faster, lower cost)` |
| `sttModel` | String | — | Deepgram model used for processing audio. Possible values: `general`, `meeting`, `phonecall`, `voicemail`, `finance`, `conversationalai`, `video`, `custom`. | `"nova-2-phonecall" (optimized for call audio)` |
| `deepgramCustomModel` | String | — | ID of the custom model. | `project-12345-model-abc (custom model ID for domain-specific vocabulary)` |
| `deepgramVersion` | String | — | Version of the Deepgram model to use. | `1.2 (use a specific model version for reproducibility)` |
| `deepgramPunctuate` | Boolean | — | Adds punctuation and capitalization to the transcript. | `true → Transcript: Hello, how are you? / false → Transcript: hello how are you` |
| `deepgramProfanityFilter` | Boolean | — | Removes profanity from the transcript. | `true → "**** this product" instead of profanity` |
| `deepgramRedact` | Enum (`String`) | — | Redacts sensitive data from transcript. Options: `pci`, `numbers`, `true`, `ssn`. | `"pci" → hides credit card numbers in transcript` |
| `deepgramDiarize` | Boolean | — | Assigns a speaker to each word in the transcript. | `true → Speaker 1: Hello / Speaker 2: Hi` |
| `deepgramDiarizeVersion` | String | — | Use `2021-07-14.0` to enable legacy diarization. | `"2021-07-14.0" (legacy diarization behavior enabled)` |
| `deepgramNer` | Boolean | — | Enables Named Entity Recognition. | `true → Tags entities like Microsoft (ORG), John (PERSON) in transcript` |
| `deepgramMultichannel` | Boolean | — | Transcribes each audio channel independently. | `true → Transcribes each audio channel (for example, agent vs. customer) separately` |
| `deepgramAlternatives` | Number | — | Specifies the number of alternative transcripts to return. | `3 → Returns 3 possible transcriptions ranked by confidence` |
| `deepgramNumerals` | Boolean | — | Converts written numbers. | `true → "twenty five" → "25"` |
| `deepgramSearch` | Array | — | An array of terms or phrases to search within the submitted audio. | `["refund", "cancellation"] → Flags all mentions of these words in transcript` |
| `deepgramReplace` | Array | — | An array of terms or phrases to search and replace within the submitted audio. | `[{"search": "um", "replace": ""}] → Removes filler words like um` |
| `deepgramKeywords` | Array | — | Keywords to boost or suppress in transcription for better context understanding. | `["premium", "loan"] → Boosts recognition accuracy for these terms` |
| `deepgramEndpointing` | Boolean / Number | — | Sets the silence duration (in ms) to consider a speaker has finished. Use `false` to disable. Default is 10 ms. Recommend ≤ 1000 ms. For longer pauses, use `utteranceEndMs`. | `500 → Ends utterance after 500 ms silence` |
| `deepgramVadTurnoff` | Number | — | Not documented. Placeholder for Voice Activity Detection (VAD) settings. | `2000 → Turns off VAD after 2 seconds of inactivity` |
| `deepgramTag` | String | — | Adds a tag to the request. Useful for tracking and usage reports. | `"BankingDemo-2025" → Helpful for tracking specific project/test runs` |
| `deepgramUtteranceEndMs` | Number | — | Duration (ms) of silence used to detect the end of an utterance in live audio. | `1000 → Marks utterance end after 1s silence in live calls` |
| `deepgramShortUtterance` | Boolean | — | Immediately returns transcript when `is_final` is set. Ideal for short commands. | `true → Immediately returns transcript for "Play" command without waiting for long silence` |
| `deepgramSmartFormatting` | Boolean | — | Enables Deepgram's Smart Formatting for improved transcript readability. Behavior depends on model/language combination. | `true → "one hundred dollars" → "$100"` |
| `deepgramFillerWords` | Boolean | — | Includes filler words such as `uh` or `um` in transcripts. | `true → Transcript includes uh, um instead of removing them` |
| `deepgramkeyterms` | String | — | Boosts recall accuracy for key terms or phrases. Improves Keyword Recall Rate (KRR) by up to 90%. | `["PIN number", "account ID"] → Special terms to highlight in transcription` |
| `deepgramEntityPrompt` | String | — | Provides custom instructions or hints to Deepgram's Named Entity Recognition (NER) system to improve detection and extraction of specific entities in transcripts. Useful for tailoring recognition to domain-specific terms (for example, product names, organizations). | `Extract all mentions of credit card numbers and bank names` |
| `eot_threshold ` | Float | — | Specifies the confidence required to trigger an `EndOfTurn` event. Default value is `0.7`. Supported range is `0.5-0.9`. Higher values improve turn-detection reliability but slightly increase response latency. | — |
| `eager_eot_threshold` | Float | — | Specifies the confidence required to trigger an `EagerEndOfTurn` event. The system doesn't set a default value. Supported range is `0.3-0.9`. This parameter enables early response generation. Lower values trigger earlier responses but increase the likelihood of false starts. | — |
| `eot_timeout_ms` | Integer | — | Specifies the maximum duration of silence, in milliseconds, before the system forces an `EndOfTurn` event regardless of confidence. Default value is `5000 ms`. Supported range is `500-10000 ms`. | — |
| `mipOptOutType` | Boolean | — | Specifies whether a user opts out of a specific feature or service. | — |
| `entityPromptType` | String | — | Specifies  the type of prompt used for entities, which may influence how information is presented or requested during interactions. | — |

### Deepgram Flux ASR

The Deepgram Flux model is a real-time speech-to-text model optimized for low-latency conversational voice AI.

It improves response time in voice interactions by detecting when a speaker finishes speaking and enabling the system to respond faster.

**Key Benefits**

* Smart turn detection-Identifies when the speaker finishes speaking.
* Supports automatic mid-call language detection.
* Ultra-low latency-\~260ms end-of-turn detection.
* Faster responses-Enable quicker AI agent replies.
* Clean transcripts-Turn-based conversation structure.
* Natural interruptions-Supports barge-in scenarios.
* High accuracy-Built on Deepgram Nova-3 transcription engine.

[Deepgram Documentation](https://developers.deepgram.com/docs/flux/quickstart)

**How to Configure Flux**

Customers can enable Flux using the following Call Control Parameters.

1. Select **Deepgram ASR** from the ASR provider list.
2. Configure Call Control Parameters

`sttProvider: deepgramflux`
`sttLanguage: en`
`deepgramEotThreshold: 0.8`
`deepgramEotTimeoutMs: 3000`
`deepgramModel: flux-general-multi`

**Parameter Details**

| Parameter | Range | Default | Description |
| :- | :- | :- | :- |
| `deepgramEotThreshold` | 0.5-0.9 | 0.7 | Confidence required to trigger EndOfTurn detection. Lower values detect turns faster. |
| `deepgramEotTimeoutMs` | 500-10000 ms | 5000 | Maximum silence before forcing EndOfTurn detection. |
| `deepgramModel` | - | `flux-general-multi` | The Deepgram model to use for speech recognition. Set `flux-general-multi` to enable multilingual recognition with automatic mid-call language switching across supported languages. |

**Recommended Configuration**

For most conversational AI use cases:

| Parameter | Recommended Value |
| :- | :- |
| `deepgramEotThreshold` | 0.7 |
| `deepgramEotTimeoutMs` | 3000 |

This provides a good balance between speed and reliability.

**Limitations**

* Flux doesn't support smart formatting for numbers.
* `eager_eot_threshold` isn't yet supported on the platform.

### Sarvam ASR

Sarvam ASR is a speech-to-text provider optimized for Indian languages. Configure it using call control parameters. For additional information on the supported languages, refer to the [list of supported languages](https://docs.sarvam.ai/api/api-guides-tutorials/speech-to-text/how-to/specify-language-codes).

| Parameter | Type | Supporting STT or TTS | Description | Example |
| :- | :- | :- | :- | :- |
| `sttProvider` | String | STT | Specifies Sarvam as the speech recognition provider. | "sarvam" |
| `sttLanguage` | String | STT | Sets the language for speech recognition. Must be one of the supported language codes. | "en-IN" |

Example Configuration

```json theme={null}
{
  "sttProvider": "sarvam",
  "sttLanguage": "en-IN"
}
```

## Common TTS Parameters

| Parameter | Type | Supporting STT or TTS | Description | Examples |
| :- | :- | :- | :- | :- |
| `disableTtsCache` | Boolean | ALL | Using cache for calling TTS engine if same statement or word found. | `"disableTtsCache": true` |
| `ttsEnhancedVoice` | String | AWS | Amazon Polly has four voice engines that convert input text into life-like speech. These include Generative, Long-form, Neural, and Standard. To use an Amazon Polly voice | `"ttsEnhancedVoice": "neural"` |
| `ttsGender` | String (Male, Female, Neutral) | Google | | `"ttsGender": "FEMALE"` |
| `ttsLoop` | Number/String | All | Use the `ttsLoop` parameter in Text-to-Speech (TTS) systems to control the repeated playback of a TTS-generated message. When `ttsLoop` is enabled, the specified TTS message plays multiple times in a loop. | `"ttsLoop": 2` |
| `earlyMedia` | Boolean | All | Use the `Early Media` parameter in TTS (Text-to-Speech) to control the playback of audio prompts or messages before a call connects. | `"earlyMedia": true` |
| `ttsOptions` | Object | Deepgram, ElevenLabs, Whisper | It's used to tune the TTS. | `"ttsOptions": {"stability": 0.7, "style": "conversational"}` |

## TTS Options in Voice Gateway

Voice Gateway now supports a `ttsOptions` parameter that lets bot developers to customize Text-to-Speech (TTS) messages by passing dynamic objects tailored to the specific TTS provider. Depending on the provider, use these options to fine-tune aspects like voice settings, speed, and other properties.

> [!NOTE]
>
> Each TTS provider has its own set of customizable parameters. For more detailed information on the parameters they support, refer to their official websites.

## Structure of `ttsOptions`

The `ttsOptions` object contains provider-specific settings in a key-value format. Following are examples of different TTS providers:

### ElevenLabs

* `optimize_streaming_latency`: Adjusts the latency during streaming.
* `voice_settings`: Includes various voice customization options like `stability`, `similarity_boost`, and `use_speaker_boost`. [Learn more](https://elevenlabs.io/docs/creative-platform/playground/text-to-speech#voice-settings).
* `speed`: Controls the speed of the generated speech. The default value is 1, and the allowable values are `>=0.7` and `<=1.2`. Values less than 1 slow down the speech, while values > 1 speed it up. [Learn more](https://elevenlabs.io/docs/conversational-ai/customization/voice/speed-control).

**Setting the ElevenLabs v3 Model via ttsOptions**

In addition to the voice customization parameters listed above, you can use `ttsOptions` to select a specific ElevenLabs model. To use the ElevenLabs v3 (`eleven_v3`) model-which is required for features such as [audio tags](https://koreai.mintlify.site/ai-for-service/channels/voice-gateway/configure-voice-gateway#audio-tags-elevenlabs-v3)-pass the `model_id` parameter inside `ttsOptions`.

**Parameter**

| Key | Value | Description |
| :- | :- | :- |
| `model_id` | `eleven_v3` | Selects the ElevenLabs v3 TTS model for the session. Required for audio tags support. |

**Example: Setting the model in a Script Node**

```JavaScript theme={null}

userSessionUtils.setCallControlParam("ttsOptions", {
    model_id: "eleven_v3",
});

```

Example: Combining model selection with other ElevenLabs options

You can pass `model_id` alongside other ElevenLabs-specific parameters in a single `ttsOptions` object:

```JavaScript theme={null}

userSessionUtils.setCallControlParam("ttsOptions", {
    model_id: "eleven_v3",
    optimize_streaming_latency: 2,
    voice_settings: {
        stability: 0.5,
        similarity_boost: 0.8,
    },
});

```

**Example: Setting at Channel level**

When configuring call control parameters at the channel level (for example, in SIP Trunk settings), set:

* Parameter Name: `ttsOptions`
* Value: `{"model_id": "eleven_v3"}`

### Deepgram

Apart from generic parameters like `ttsLanguage` and `voiceName`, which are common across most TTS engines, Deepgram offers a few additional parameters that enhance customization:

* `encoding` (string): You can specify the desired encoding format for the output audio file, such as `mp3` or `wav`.
* `model` (enum): Defines the AI model used for synthesizing the text into speech. The default model is `aura-asteria-en`, optimized for natural-sounding English voice output.
* `sample_rate` (string): This enables you to set the sample rate of the audio output, offering control over the quality and clarity of the sound produced.
* `Container`: The Container feature lets users to specify the desired file format wrapper for the output audio generated through text-to-speech synthesis.

**Aura-2 voice controls**

When the model is set to an Aura-2 voice (for example, `aura-2-thalia-en`), two additional `ttsOptions` parameters are available: `speed` and `pronunciations`.

| Parameter | Range | Default | Description |
| :- | :- | :- | :- |
| `speed` | 0.7 - 1.5 | 1.0 | Controls the speech rate. Values below 1.0 slow speech down; values above 1.0 speed it up. |
| `pronunciations` | — | — | Defines pronunciation overrides for specific words using IPA (International Phonetic Alphabet) notation. |

These settings appear only when an Aura-2 model is selected. Values outside the supported range, or malformed pronunciation entries, are rejected with a validation error before the request is sent. [Learn more](https://developers.deepgram.com/docs/tts-voice-controls).

Example configuration:

```json theme={null}
{
  "ttsProvider": "deepgram",
  "ttsOptions": {
    "model": "aura-2-thalia-en",
    "speed": 0.7
  }
}
```

These parameters provide additional flexibility for developers to fine-tune the audio output to meet their specific needs. Set all these parameters inside ttsOptions. [Learn more](https://developers.deepgram.com/docs/tts-rest).

### Google (Gemini 3.1 Flash TTS)

Gemini 3.1 Flash TTS is Google's latest expressive speech synthesis model. It generates natural, emotionally aware speech with context-sensitive prosody and multilingual support. Configure it through the existing Google TTS provider.

Parameters:

| Parameter | Type | Description | Example |
| :- | :- | :- | :- |
| `ttsProvider` | String | Specifies Google as the TTS provider. | "google" |
| `ttsLanguage` | String | Sets the language for speech synthesis. | "en-IN" |
| `voiceName` | String | Specifies the voice to use. | "kore" |
| `ttsOptions.model` | String | Specifies the Gemini 3.1 Flash TTS model. | "gemini-3.1-flash-tts-preview" |

```json theme={null}
{
  "ttsProvider": "google",
  "ttsLanguage": "en-IN",
  "voiceName": "kore",
  "ttsOptions": {
    "model": "gemini-3.1-flash-tts-preview"
  }
}
```

Capabilities:

| Capability | Description |
| :- | :- |
| Expressive speech | Generates speech that conveys emotion — empathetic, excited, calm, happy, or sad — for a more human-sounding voice. |
| Natural prosody | Adjusts pauses, pitch, stress, and speaking rhythm based on sentence structure and context. |
| Multilingual synthesis | Generates speech in multiple languages and can switch between languages within a supported session. |
| Style and tone control | Supports different speaking styles — friendly, professional, cheerful, or calm — where supported by the Google API. |

**Known limitations**

| Limitation | Detail |
| :- | :- |
| Voice identity consistency | Voice timbre, tone, and delivery may vary subtly across API calls even when the same voice is selected. Less suitable for long-form narration requiring a consistent voice identity. |
| Audio quality in long-form synthesis | Audio quality may degrade for speech generated beyond approximately 1-2 minutes. Better suited for shorter conversational responses. |

> [!NOTE]
>
> Gemini 3.1 Flash TTS doesn't support text input streaming.

### AWS

Apart from generic parameters like `ttsLanguage` and `voiceName`, which are common across most TTS engines, Aws offers a few additional parameters that enhance customization, like `ttsEnhanceVoice`, also known as an engine.

Amazon Polly has four voice engines that convert input text into lifelike speech. These include `standard`, `neural`, `generative`, and `long-form`.

`ttsEnhancedVoice = “neural”`

### OpenAI STT

OpenAI STT provides speech-to-text transcription through OpenAI's audio models. Configure it using call control parameters.

OpenAI STT isn't available as a selectable option in the UI. To configure it, contact the [Voice Gateway team](https://support.kore.ai). The team creates a label associated with your account, which you then use as the `ttsLabel` value in the call control parameters.

| Model | Notes |
| :- | :- |
| `gpt-live-transcribe` | Recommended. Supports keywords, language lists, delay control, and native turn detection. |
| `gpt-4o-transcribe` | Supports turn detection, prompt, and single language. |
| `gpt-4o-mini-transcribe` | Supports turn detection, prompt, and single language. |
| `gpt-transcribe` | Supports turn detection, prompt, keywords, language lists, and single language. |
| `gpt-realtime-whisper` | Supports delay and single language only. |
| `whisper-1` | Supports turn detection, prompt, and single language. |

Parameter Support by Model

Send only parameters supported by the selected model. Sending an unsupported parameter causes the session to fail.

| Parameter | `whisper-1` | `gpt-4o-transcribe` | `gpt-transcribe` | `gpt-live-transcribe` | `gpt-realtime-whisper` |
| :- | :- | :- | :- | :- | :- |
| `openaiTurnDetection` | ✓ | ✓ | ✓ | — | — |
| `openaiPrompt` | ✓ | ✓ | ✓ | ✓ | — |
| `openaiKeywords` | — | — | ✓ | ✓ | — |
| `openaiLanguages` | — | — | ✓ | ✓ | — |
| `openaiLanguage` | ✓ | ✓ | ✓ | ✓ | ✓ |
| `openaiDelay` | — | — | — | ✓ | ✓ |

| Parameter | Range | Default | Description |
| :- | :- | :- | :- |
| `openaiModel` | See supported models | — | Selects the OpenAI speech recognition model. |
| `openaiLanguage` | Language code (for example, `en`, `hi`, `fr`). | — | Specifies the single language the caller is expected to speak. |
| `openaiLanguages` | Array of language codes (for example, `["en", "ar"]`). | — | Lists the languages that may be spoken during the call. Useful for multilingual or code-switching scenarios. Don't use together with `openaiLanguage` when the selected model expects only one language field. |
| `openaiPrompt` | Text string | — | Provides context about the call — names, products, acronyms, or special vocabulary — to improve transcription accuracy. Not supported by `gpt-realtime-whisper`. |
| `openaiKeywords` | Array of strings | — | Lists important words OpenAI should recognize with priority, such as product names or account identifiers. Avoid line breaks and angle brackets inside a keyword. |
| `openaiTurnDetection` | `null`, `server_vad`, `semantic_vad`. | — | Configures how OpenAI detects when the caller starts or stops speaking. Set to `null` to use Voice Gateway's native turn detection instead. |
| `openaiDelay` | `minimal`, `low`, `medium`, `high`, `xhigh`. | — | Controls the balance between receiving transcripts quickly and giving OpenAI more time to improve accuracy. |
| `openaiInputAudioNoiseReduction` | `near_field`, `far_field`. | — | Reduces background noise before transcription. Use `near_field` for a close, clear microphone; `far_field` for audio captured farther away or in a noisier environment. |

Turn Detection Behavior

Models `gpt-live-transcribe` and `gpt-realtime-whisper` don't support server-side turn detection. For these models, Voice Gateway uses its native turn detection capabilities — `mod_vad_kore`, `mod_smart_turn`, or `mod_kore_turn_taking` — and sends `input_audio_buffer.commit` at the end of detected speech.
All other OpenAI STT models use server-side turn detection behavior as configured by `openaiTurnDetection`.

Example Configuration

```json theme={null}
{
  "sttProvider": "openai",
  "sttLanguage": "en-US",
  "ttsLabel": "your-label-here",
  "openaiModel": "gpt-4o-transcribe",
  "openaiTurnDetection": {
    "type": "server_vad",
    "threshold": 0.5,
    "prefix_padding_ms": 300,
    "silence_duration_ms": 500
  },
  "openaiPrompt": "Transcribe the caller accurately. Preserve names, numbers, and account identifiers.",
  "openaiLanguage": "en",
  "openaiKeywords": ["Kore", "OpenAI", "account number"],
  "openaiInputAudioNoiseReduction": {
    "type": "near_field"
  }
}
```

### Open AI (Whisper)

Apart from generic parameters like `ttsLanguage` and `voiceName`, which are common across most TTS engines, Whisper offers a few additional parameters that enhance customization, like a model.

For real-time applications, the standard tts-1 model provides the lowest latency but at a lower quality than the tts-1-hd model. Because of the audio generation process, tts-1 may produce more static in certain situations than tts-1-hd. Sometimes, the audio may not have noticeable differences depending on your listening device and the person.

```json theme={null}
ttsOptions = {
   model = "tts-1"
}
```

### Sarvam

Sarvam TTS supports Indian language voice synthesis. Use `ttsOptions` to configure speech rate and variation.

Example Configuration

```json theme={null}
{
  "ttsProvider": "sarvam",
  "ttsLanguage": "en-IN",
  "voiceName": "sumit",
  "ttsOptions": {
    "pace": 1,
    "temperature": 0.6
  }
}
```

`ttsOptions` parameters

| Parameter | Range | Default | Description |
| :- | :- | :- | :- |
| pace | 0.5 - 2.0 | 1 | Controls speech rate. Values below 1 slow speech down; values above 1 speed it up. |
| temperature | 0.01 - 1.0 | 0.6 | Controls variation in speech output. Lower values produce more consistent output; higher values introduce more variation. |

## TTS Limitations

The following table describes how Voice Gateway processes and sends responses to the TTS provider in streaming and non-streaming modes.

| Mode | Response source | How Voice Gateway processes the response | What Voice Gateway sends to the TTS provider | Limitation |
| :- | :- | :- | :- | :- |
| Non-streaming | Predefined text | Receives the complete text and splits it at punctuation boundaries, keeping each segment within the 1,000-character limit whenever possible. | Sends each segment separately to the TTS provider. | Supports long text through multiple TTS segments. |
| Non-streaming | LLM-generated text | Waits for the complete LLM response, then splits it at punctuation boundaries while respecting the 1,000-character limit. | Sends each segment separately to the TTS provider. | Supports long responses through multiple TTS segments. |
| Streaming | Predefined text | Receives the predefined text as a single stream chunk and forwards it without splitting it at punctuation boundaries. | Sends the complete chunk as one request. | Can't send the chunk for TTS generation if it exceeds 1,000 characters. |
| Streaming | LLM-generated text | Receives the LLM response incrementally, typically in chunks or tokens. | Sends each incoming chunk to the TTS provider as received. | Supports total responses longer than 1,000 characters, but each chunk must stay within the 1,000-character limit. |

## Common Use Cases for Call Control Parameters

### Continuous ASR

Continuous ASR (Automatic Speech Recognition) is a feature that lets tuning Speech-to-Text (STT) recognition for the collection of phone numbers, customer identifiers, and other strings of digits or characters, which, when spoken, are often spoken with pauses between utterances. Two parameters to enable it are:

| Parameter | Type | Supporting STT or TTS | Description | Example/Notes |
| :- | :- | :- | :- | :- |
| `continuousASRTimeoutInMS` | Number (milliseconds) | STT: Google, Microsoft TTS: Not required | Duration of silence (in milliseconds) after receiving a transcript before returning the result. If another transcript arrives within the timeout, the system combines transcripts and continues recognition. The system returns the final result after silence exceeds the timeout. | `5000` (5 seconds) |
| `continuousASRDigits` | Digit (for example, `*`, `%`, `&`, `#`) | STT: Google, Microsoft TTS: Not required | DTMF key that terminates the gather operation and returns the collected results immediately. | Example: `#` |

### Handling Bot Delay

Configure Voice Gateway to act when the bot takes time to respond to a message.

#### Add a delay between bot messages

When two message nodes, or a message node and an entity node, are placed sequentially within the same node, the bot plays the prompts in sequence. By default, the next prompt starts immediately after the previous prompt finishes.

To add a pause between prompts, configure a custom delay in the **Bot Response** node.

**Add a custom pause**

1. In the **Bot Response** node, select **Add (+)**.
2. Select the **SmartAssist Gateway** channel with **Custom JavaScript**.
3. Add the following function to the channel override template:

   ```text theme={null}
   print(voiceUtils.pauseAndPlay(time in seconds, message to play))
   ```

   For example:

   ```text theme={null}
   print(voiceUtils.pauseAndPlay(4, "Thanks for calling. This message will play after a 4-second delay."));
   ```

   This example adds a 4-second pause before the message plays. Specify the delay in seconds based on the required conversation flow.

#### Handle Bot Delay After User Input

**Configuring Bot No Input Timeout and Delays**

Bot No Input handles delays when the bot or a backend operation takes longer than expected to respond. It prevents dead air by playing configured speech or audio and provides a recovery action if the bot remains unresponsive.

**Turn on Bot No Input**

Turn on this feature in the [Start Flow](https://koreai.mintlify.site/ai-for-service/flows/create-and-manage-flows.md#create-a-start-flow) creation page to apply the configured settings.

When turned off, the default give-up timeout applies. If the bot doesn't respond within the default timeout, the call is disconnected.

![Edit Flow](https://mintcdn.com/koreai/w2XRp8Zjocsid9tc/ai-for-service/channels/voice-gateway/images/edit-flow.png?fit=max\&auto=format\&n=w2XRp8Zjocsid9tc\&q=85\&s=f64741b5056a10d6ee9a5b7fa59adbf0)

**Supported Parameters**

| Parameter | Type | Description |
| :- | :- | :- |
| `botNoInputTimeout` | Number | Wait time (in seconds) for a bot response before triggering a Bot No Input action. |
| `botNoInputSpeech` | String | One or more quoted sentences to play when the timeout is triggered. Sentences are separated by commas. |
| `botNoInputURL` | String | URL of the audio file to play when the timeout is triggered. |
| `botNoInputRetries` | Number | Number of times to perform the Bot No Input action. |
| `botNoInputGiveUpTimeout` | Number | Maximum time to wait before performing the configured give-up action. |
| `botNoInputPlaySelection` | String | Selects sequential or random playback for multiple speech messages. Currently, configure this parameter through call control parameters. |

**Interaction Types**

**Bot No Input Speech**

Enter one or more sentences in double quotation marks, separated by commas:

For example: `"Please wait while I check that for you.","I am still retrieving your information.","This is taking longer than expected."`

**Bot No Input URL**

Provide a valid audio URL. The audio file plays whenever the Bot No Input timeout is triggered.

**Speech selection**

The `botNoInputPlaySelection` parameter controls how multiple speech sentences are played.

```bash theme={null}
{
  "botNoInputPlaySelection": "sequential"
}
```

Supported values:

* `sequential` - Plays sentences in the configured order.
* `random` - Selects a sentence randomly for each retry.
* Parameter omitted - Uses random playback by default.

**Timer and playback behavior**

The Bot No Input timer starts when bot processing begins.

When the timer expires:

1. The configured speech or audio is played.
2. After playback completes, the Bot No Input timer starts again.
3. The next retry occurs after the configured timeout.

For example, with a 10-second timeout, if the first message starts at 10 seconds and takes 3 seconds to play, the next timer starts at 13 seconds. The next message starts at 23 seconds, not 28 seconds.

```
First message: 10 seconds
First message ends: 13 seconds
Second message: 23 seconds
```

If the bot response becomes available while a progressive message is playing:

* The progressive message stops immediately.
* The bot response starts playing.
* No further Bot No Input message is triggered for that response cycle.
* The complete progressive sentence appears in the transcription, even if playback was interrupted.

**Give-up actions**

When `botNoInputGiveUpTimeout` is reached, no further retries occur. The configured give-up action is performed.

**End the call**

The system plays the configured end-call message and disconnects the call.

**Transfer the call**

The system plays the configured transfer message and transfers the call to the configured SIP destination.

Transfer configuration may include:

* SIP transfer method
* SIP redirect URL
* SIP headers

Example: Sequential playback

```bash theme={null}
{
  "enabled": true,
  "botNoInputTimeout": 10,
  "botNoInputSpeech": "\"Please wait while I check that for you.\",\"I am still retrieving your information.\",\"This is taking longer than expected.\"",
  "botNoInputRetries": 3,
  "botNoInputGiveUpTimeout": 40,
  "botNoInputPlaySelection": "sequential",
  "giveUpAction": "endCall",
  "endCallMessage": "I am unable to retrieve the information right now. Please try again later."
}
```

The messages play sequentially after each configured timeout until the bot responds, the retries are exhausted, or the give-up timeout is reached.

#### Handle Delay Between Two Message Nodes

Voice Gateway can only handle delays when it sends a response to the bot and waits for the bot's reply. If a delay occurs, Voice Gateway can handle it. If a delay occurs between a Message node or Script node where the user hasn't spoken, Voice Gateway won't be aware of the delay, and the bot developer must handle it manually.

Place a Service Node between two Message nodes (delay observed between two Message nodes):
Manage this manually, as the gateway has received a command to play a message and isn't waiting for user input. The gateway won't initiate a delay timer and waits for the next bot message.

To handle this scenario:

* Play music before the API call in the Message node.
* Configure the Service node.
* Deliver the message after the Service node.

\[!NOTE] If you receive a response from the API and don’t want to play the full music, immediately abort the music and play the Message node prompt using the channel override utility function:
`print(voiceUtils.abortPrompt(“Dummy message“))` → (The message parameter is optional).

![Optional Message Parameter](https://mintcdn.com/koreai/VaEjvvsSenY-Yvtg/ai-for-service/channels/voice-gateway/images/optional-message-parameter.png?fit=max\&auto=format\&n=VaEjvvsSenY-Yvtg\&q=85\&s=5eac62ac8efedea45900300468143161)

### Barge-In Scenarios

The Barge-In feature controls Voice Gateway behavior in scenarios where the user starts speaking or dials DTMF digits while the bot is playing its response to the user. In other words, the user interrupts ("barges-in") the bot.

| Parameter | Type | Supporting STT/TTS | Description |
| :- | :- | :- | :- |
| `listenDuringPrompt` | Boolean (true or false) Similar to Barge-In | STT-Google and Microsoft TTS-Not Required | If `false`, the bot doesn't listen for user speech until the response finishes playing. Default: `true`. |
| `bargeInMinWordCount` | Number | STT-Google and Microsoft TTS-Not Required | If Barge-In is sensitive, the bot only interrupts playback after speaking the specified number of words. Default: 1. |
| `bargeInOnDTMF` | Boolean | STT-Google and Microsoft TTS-Not Required | Lets users to press a key to interrupt the audio playback. After pressing a key, the user can speak their input. |
| `dtmfCollectInterDigitTimeoutMS` | Number (milliseconds) | STT-Google and Microsoft TTS-Not Required | Time allowed between DTMF key presses before sending all digits to the bot. |
| `dtmfCollectSubmitDigit` | Number | STT-Google and Microsoft TTS-Not Required | Special digit that submits all collected DTMF input immediately, bypassing the timeout or max digit limit. |
| `dtmfCollectMaxDigits` | Number | STT-Google and Microsoft TTS-Not Required | Maximum number of DTMF digits to collect. Example: If set to `5` and input is `1234567`, only processes `12345`. |
| `dtmfCollectminDigits` | Number | STT-Google and Microsoft TTS-Not Required | Minimum number of DTMF digits to collect. Default: 1. |
| `dtmfCollectnumDigits` | Number | STT-Google and Microsoft TTS-Not Required | Exact number of DTMF digits to collect. |
| `input` | Array of strings Valid values: `['digits']`, `['speech']`, `['digits', 'speech']` | STT-Google and Microsoft TTS-Not Required | Specifies allowed input types.Default: `['digits']`. |

### Backchannel Words

Backchannel words are short acknowledgments - like uh-huh, okay, or got it - that callers say while listening without intending to interrupt. By default, any speech during playback triggers barge-in and stops the audio.

Configuring `backchannelWords` signals Voice Gateway to keep the audio playing when a caller speaks one of these words during playback, instead of treating it as a barge-in. When Voice Gateway matches an utterance to a configured word, it:

* Skips intent detection and dialog execution for that input.
* Leaves session context — including the last captured user input — unchanged.
* Excludes the utterance from NLP Insights and intent-training data.
* Doesn't count the utterance against no-input timeouts or retry limits.
* Still records the utterance in the conversation transcript, marked as a backchannel phrase.

**Configuration**

To set `backchannel` Words at the node-level:

* Each entry must be non-empty after trimming and have a maximum length of 100 characters.
* Lists support a maximum of 100 entries.
* Voice Gateway removes duplicate entries automatically after normalization.
* An empty array or an omitted parameter turns off the feature for that scope with no runtime impact.

**How it works**

1. Audio playback starts.
2. The caller speaks during playback.
3. Voice Gateway normalizes the detected phrase - converting to lowercase, trimming whitespace, and stripping punctuation - and checks it against the configured list.
4. If the phrase matches a configured backchannel word, audio continues playing and Voice Gateway sends a `speechDetected` event with backchannel metadata. Barge-in doesn't trigger.
5. If the phrase doesn't match, standard barge-in behavior applies.

After playback completes, the system treats these words as normal speech and doesn’t classify them as backchannels.

**Array semantics**

| Value | Behavior |
| :- | :- |
| Non-empty array | Replaces the active backchannel word list. |
| Empty array (`[]`) | Clears the list and turns off backchannel classification. |
| Parameter omitted | Keeps the current active list unchanged. |

**Example Configuration**

```json theme={null}
{
  "backchannelWords": ["uh-huh", "okay", "got it", "sure", "yes", "correct"]
}
```

> [!NOTE]
>
> Backchannel detection uses exact normalized phrase matching. Voice Gateway converts detected speech to lowercase, trims whitespace, and strips punctuation before comparing it against the list.

### Language Detection

In this setup, developers don't need to use DTMF or other methods to switch the bot's language. Instead, the bot automatically detects the language based on the user's utterance.

For example, if a user speaks in English, the conversation continues in English. If the user switches to Spanish, the language switches to Spanish. [Learn more](https://koreai.mintlify.site/ai-for-service/app-settings/language-management/managing-languages-for-multilingual-vas#adding-a-language-to-an-ai-agent).

**Configuration Steps**:

1. In Bot Builder (on the child bot), navigate to **Languages,** add a new language (for example, Spanish), and enable it.
2. Select English as the default language from the language dropdown menu. [Learn more](https://koreai.mintlify.site/ai-for-service/app-settings/language-management/managing-languages-for-multilingual-vas#adding-a-language-to-an-ai-agent).
3. Create a new dialog titled `Language Detection` (or choose a suitable name).
4. Inside this dialog, add an **entity node** to capture user intent input.
5. Set the entity precedence to **'Intent over Entity'** in the advanced controls.
6. Add the **AlternativeLanguage** call control parameter.
7. Switching languages mid-conversation isn't supported; doing so can cause the bot to lose context. Language detection should happen at the beginning of the conversation (for example, in the welcome task), with the switch based on the user's first utterance.
8. Opt for **'Intent over Entity'** to prioritize intent detection in the user's language.
9. Create another dialog with a specific intent (for example, "book flight") and add relevant entities (for example, selecting source and destination).
10. In the entity configuration, include the following call parameters:
    * **Name**: alternativeLanguages
    * **Value**: \[] (Leave it empty if no further language switching is needed).
11. Add utterances in the desired language and train the bot.
12. Change the language to Spanish in the bot language dropdown.
13. Open the intent and update utterances and intent details in Spanish.
14. Update entity details in Spanish as well.
15. Publish the app.

These steps make sure the bot can detect the user's language at the start and adjust the conversation flow accordingly.

### Click-to-Call

To configure ASR (Automatic Speech Recognition) and TTS (Text-to-Speech) in a ClickToCall flow, add the call control parameters in the [Script Task](https://koreai.mintlify.site/ai-for-service/flows/node-types/script-task) node at the start of the flow. Set the parameters must in the following format:

```
const ccParamList = [
    { key: 'ttsLanguage', val: 'ja-JP' },
    { key: 'ttsProvider', val: 'google' },
    { key: 'voiceName', val: 'ja-JP-Wavenet-B' },

    { key: 'sttLanguage', val: 'ja-JP'},
    { key: 'sttProvider', val: 'microsoft'}

];

ccParamList.forEach((ccParam) => {
    userSessionUtils.setCallControlParam(ccParam.key, ccParam.val);
})
```

| Parameter | Type | Description | Example |
| :- | :- | :- | :- |
| `ttsLanguage` | String | Specifies the language code for text-to-speech output. | ja-JP |
| `ttsProvider` | String | Defines the provider for text-to-speech services. | google |
| `voiceName` | String | Identifies the voice used for text-to-speech output. | ja-JP-Wavenet-B |
| `sttLanguage` | String | Specifies the language code for speech-to-text recognition. | ja-JP |
| `sttProvider` | String | Defines the provider for speech-to-text services. | Microsoft |

***

## Best Practices for Updating ASR or TTS

### When to update ASR or TTS

Every ASR or TTS update can trigger a connection or reconnection. During reconnection, messages may remain in a buffer until the system establishes the connection, which can delay or affect the call experience.

The system generally has sufficient time to configure ASR or TTS while establishing the call connection. Avoid unnecessary updates during the conversation.

Don't update ASR or TTS unless the change is required.

### Best practices

**1. Configure ASR or TTS in the Experience Flow**

Select the required ASR, TTS, and labels in the Experience Flow itself.

If additional parameters are required, configure them in the first node of the bot whenever possible. This practice avoids repeated configuration changes and unnecessary connection reinitialization.

**2. Pass required parameters when updating ASR or TTS**

When using custom labels, pass the required parameters whenever an ASR or TTS is updated:

* `ttsProvider`
* `sttProvider`
* `sttLabel`
* `ttsLabel`

If any of these parameters are missing, the system may create the connection using the default cloud vendor instead of the intended custom configuration. This gap can lead to unexpected behavior.

Example: When updating the `azureEndPointId` for ASR with a custom label, pass the corresponding `sttLabel` along with the other required parameters. Otherwise, the system may route the request to the default cloud vendor.

**3. Avoid unnecessary TTS updates for neural voices**

If the Neural TTS voice in use supports multiple languages, check whether the current voice supports the new conversation language. If the voice supports the language, the call control parameters, TTS language, and voice name don't require an update when the conversation language changes.

Example:

```text theme={null}
ttsLanguage = en-IN
voiceName = en-IN-Diya:DragonHDLatestNeural
```

If the current voice supports the new conversation language, don't update the call control parameters only to change the language. The Experience Flow uses the existing neural voice for the supported language without triggering another TTS update.

This voice supports both English and Hindi. Therefore, when the flow switches to Hindi and the current voice supports Hindi, the call control parameters, TTS language, and voice name don't require an update.

Example:

```text theme={null}
ttsLanguage = hi-IN
voiceName = hi-IN-Kalpana:DragonHDLatestNeural
```

If the current voice doesn't support the new conversation language, update the call control parameters with both the appropriate `ttsLanguage` and a `voiceName` that supports that language. Otherwise, the system may not apply the requested language correctly, and the update can trigger a reconnection.

**4. When changing TTS, keep the language configuration consistent**

When a TTS configuration change is necessary, verify that the language code in the TTS language parameter matches the language the voice name represents (except Neural voices from Azure).

Example:

```text theme={null}
ttsLanguage = en-AU
voiceName = en-AU-Natasha
```

The TTS language and voice language code must match whenever the selected voice requires language-specific configuration.

**5. Apply the same principle to ASR**

The same best practice applies to ASR. Avoid updating ASR parameters unless the change is required.

For example, when using a custom ASR label and updating the azureEndPointId, pass the custom label along with the required ASR parameters.

If the configuration omits the label, the system may create the connection using the default cloud vendor, which can lead to unexpected behavior.

**Recommendation**: Configure the required ASR or TTS settings at the beginning of the call and avoid updates during the conversation. When an update is required, pass the complete configuration to ensure the configuration uses the intended provider and settings.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.