How do I troubleshoot request latency?

Separate connection time, time to first Token, and total response time before comparing model, context, and network.

Steps

1. Record DNS/TLS, first byte, first Token, and total duration.
2. Build a baseline with a short prompt and non-streaming request.
3. Compare models, regions, concurrency, and observation periods.

Limits and exceptions

Generation length, tool calls, upstream queues, and network variance affect latency; one sample is not an SLA.