Skip to content
Dictionary by pitch.dog
AI product & model operations · Working

Inference cost

Also called: serving cost

ELI5

What it costs each time the model generates an answer.

Used in conversation

“Inference cost makes the current free tier uneconomic.”

Definition

The cost of running a model to produce outputs, often affected by tokens, hardware, latency and volume.

Here “Inference cost” means: The cost of running a model to produce outputs, often affected by tokens, hardware, latency and volume.

Pitch context

You will see “Inference cost” in product strategy decks, roadmaps, experiments and architecture reviews when the discussion reaches ai product & model operations.

Why it matters: In ai product & model operations, the scope can change what data may be used, who may access it and which controls are required.

When the words have to travel

Need an investor or company deck?

Bring the business, the evidence or the half-formed ask. We help it travel beyond the spreadsheet.

See how pitch.dog can help
Sources & evidence · 2

Direct term-level sources and supporting source families.