« Back to Glossary Index

Pretraining is the initial phase in training a foundation model, often conducted using unsupervised learning techniques. During this stage, the model learns general patterns and features from a large, diverse dataset without specific task-oriented labels. The result is a model with broad capabilities but without high accuracy for specialized tasks. After pretraining, the model is typically fine-tuned using a smaller, task-specific dataset to enhance its accuracy and adapt it to specific applications.