> ## Documentation Index
> Fetch the complete documentation index at: https://koreai.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Natural Language Processing

[Back to NLP Topics](https://koreai.mintlify.site/ai-for-service/automation/natural-language/nlp-topics.md)

The Platform uses a multi-engine NLP approach to identify user intent, extract entities, and execute the correct task. Understanding NLP helps you train your VA for accurate intent detection and natural conversations.

***

## NLP Engines

| Engine | Description |
| - | - |
| **Fundamental Meaning (FM)** | Computational linguistics built on ChatScript. Analyzes word meaning, position, conjugation, capitalization, and sentence structure. |
| **Machine Learning (ML)** | Trains on example utterances; learns and generalizes to recognize similar inputs. |
| **Knowledge Graph (KG)** | Converts FAQ content into structured conversational responses. |
| **Traits Engine** | Multi-class classifier identifying characteristics in utterances to refine intent detection. |
| **Small Talk Engine** | Handles conversational pleasantries to make interactions feel natural. |
| **Ranking and Resolver (R\&R)** | Scores and ranks results from all engines to determine the winning intent. |

***

## Conversation Flow

1. **NLP Analysis** — User input passes through all NLP engines for intent detection and entity extraction.
2. **Task Execution** — The winning intent executes. The conversation engine maintains state (user details, previous intents, context).
   * **Preconditions** — If required conditions aren't met, the intent is rejected.
   * **Negative patterns** — Patterns that prevent incorrect intent matches.
   * **Event handling** — Welcome messages, sentiment, etc.
3. **Interruption Handling** — Manages mid-task intent switches and sentiment-triggered agent transfers.
4. **Response Generation** — A response is generated and rendered for the user's channel.

***

## NLP Preprocessing

Before intent detection, each utterance undergoes:

| Step | Description |
| - | - |
| **Tokenization** | Splits utterance into sentences, then words. Uses TreeBank Tokenizer for English. |
| **toLower()** | Converts to lowercase. Not applied to German (word meaning changes by case). ML and KG only. |
| **Stop word removal** | Removes low-signal words. Language-specific; disabled by default. ML and KG only. |
| **Stemming** | Cuts words to their stem (for example, "Working" → "work"). Output may not be a valid word. |
| **Lemmatization** | Converts to base form using a dictionary (for example, "housing" → "house"). |
| **N-grams** | Combines co-occurring words (for example, "New York City") for richer context. |

***

## Choosing an NLP Engine

| Scenario | Recommended Engine |
| - | - |
| Large corpus per intent | **ML** — flexible, auto-learns from examples. Corpus of 200-300 for distinct intents; 1000+ for similar intents. |
| Query/FAQ-type intents or document-based answers | **KG** — semantic matching for knowledge content. |
| Idiomatic, command-like sentences, or tolerance for false positives | **FM** — deterministic, rule-based. |

***

## NLP Configuration in the Platform

Go to **Automation > Natural Language**:

| Section | Purpose |
| - | - |
| **Training** | Add ML utterances, synonyms, concepts, and patterns. |
| **NLU Config** | Set confidence thresholds, engine tuning, and advanced settings. |

For engine-specific training and tuning, see:

* [Machine Learning Engine](https://koreai.mintlify.site/ai-for-service/automation/natural-language/training/machine-learning-engine)
* [Fundamental Meaning Engine](https://koreai.mintlify.site/ai-for-service/automation/natural-language/training/fundamental-meaning)
* [Knowledge Graph Training](https://koreai.mintlify.site/ai-for-service/automation/knowledge-ai/knowledge-graph-training)
* [Traits](https://koreai.mintlify.site/ai-for-service/automation/natural-language/training/traits)
* [Ranking and Resolver](https://koreai.mintlify.site/ai-for-service/automation/natural-language/training/ranking-and-resolver)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.