Inference
Definition
The process of running a trained AI model to generate an output or perform a task.
Put simply
Training creates the model. Inference is what happens whenever you actually use it.
What boards and leaders should understand
Inference becomes commercially significant as AI use scales because every interaction with a model consumes computing resources.
Different models, reasoning approaches, context sizes and workflows can have very different inference costs.
This is one reason AI economics increasingly involves more than the price of a software licence.
Questions to consider
- As AI usage grows, do we understand which workloads are driving inference cost and whether that cost is justified by the value created?
