AI 4UAnalyze my business

Plain-language AI glossary

Term 30TechniquesMeaning / context / connections

Techniques / Definition

Streaming

A method of receiving AI model output token-by-token in real time as it is generated, rather than waiting for the complete response.

Browse all termsUnderstand the idea
30of 75
01

MeaningThe one-sentence definition.

02

ContextHow the idea works in practice.

03

UsesWhere the concept becomes useful.

01 / Plain-language context

How Streaming works.

Streaming uses Server-Sent Events (SSE) to deliver tokens to the client as the model generates them. Instead of a 5-second wait followed by a wall of text, users see the response appear word by word, similar to watching someone type. This dramatically improves perceived latency and user experience.

All major AI APIs support streaming: OpenAI (stream: true in Responses API), Anthropic (stream: true in Messages API), and Google (streamGenerateContent endpoint). On the client side, you process the SSE stream and append each token to the UI. Most AI chat interfaces use streaming by default.

Streaming is essential for any user-facing AI feature. A 3-second time-to-first-token feels fast with streaming but painfully slow without it. Implementation considerations: you cannot parse structured JSON until the stream completes, error handling is different (errors may arrive mid-stream), and you need to handle connection drops gracefully. For batch/background tasks where no human is waiting, non-streaming is simpler.

02 / Practical uses

Where it helps.

  1. 01Chat interfaces and assistants
  2. 02Real-time content generation
  3. 03Code completion in IDEs
  4. 04Live document editing with AI
03 / Keep exploring
InferenceThe process of running a trained AI model to generate predictions or outputs from new inputs, as opposed to training the model.LatencyThe time delay between sending a request to an AI model and receiving the response, critical for real-time user-facing applications.Large Language Model (LLM)A neural network trained on massive text datasets that can generate, understand, and reason about human language.ChatbotAn AI application that conducts conversations with users through text or voice, handling questions, tasks, and interactions in natural language.