Skip to main content
PProfileMatch
← Path

Large Language Models · Using LLMs via APIs: Latency, Cost Drivers and Choosing a Model

0 XP
  1. 1 current stop: learning
  2. 2 upcoming
  3. 3 upcoming
  4. 4 upcoming
  5. 5 upcoming

Finish after stop 5

Stop 1 of 5 · Traffic jam at the roadworks

Trick to keep in mind+3 XP

Output length is the slow lane: write less, stream sooner.

Generation produces one token after another, so total time grows with the number of tokens written. Asking for shorter output therefore saves the most, and streaming lets users start reading while the rest is still being produced. Small edits to the instructions barely change the waiting time, and retries make slow calls even longer.

Read it once, then a question tests whether the trick stuck.