Full definition

Time the model takes to generate a response after receiving the request.

Example in a business context

In customer support, latency >3 seconds increases conversation abandonment.