« Back to Glossary Index

A token is a fundamental unit of text that a large language model (LLM) uses to understand and generate language. It can be a whole word or a part of a word.