AI 4UAnalyze my business

Plain-language AI glossary

Term 35InfrastructureMeaning / context / connections

Infrastructure / Definition

GPU / TPU

Specialized processors designed for the parallel mathematical operations that AI models require for training and inference.

Browse all termsUnderstand the idea
35of 75
01

MeaningThe one-sentence definition.

02

ContextHow the idea works in practice.

03

UsesWhere the concept becomes useful.

01 / Plain-language context

How GPU / TPU works.

GPUs (Graphics Processing Units) and TPUs (Tensor Processing Units) are the hardware that makes modern AI possible. Unlike CPUs that excel at sequential tasks, GPUs have thousands of cores optimized for the matrix multiplications that neural networks use. NVIDIA dominates the AI GPU market with its A100 and H100 chips. Google's TPUs are custom AI accelerators available through Google Cloud.

For builders using AI APIs, you never interact with GPUs directly since the provider manages the hardware. GPU knowledge matters when: self-hosting models (you need to choose the right GPU), estimating costs (GPU time is the main cost driver), and understanding why AI APIs are priced the way they are.

Cloud GPU pricing varies widely. An NVIDIA H100 on AWS costs roughly $30-40/hour. This is why API providers charge per token: they are amortizing GPU costs across millions of requests. For self-hosting, the breakeven point depends on your volume. As a rough guide, if you spend over $5,000/month on API calls for a single model, it may be worth exploring self-hosting on dedicated GPUs.

02 / Practical uses

Where it helps.

  1. 01Understanding AI infrastructure costs
  2. 02Self-hosting model deployment
  3. 03Training custom models
  4. 04Capacity planning for AI applications
03 / Keep exploring
InferenceThe process of running a trained AI model to generate predictions or outputs from new inputs, as opposed to training the model.Model ServingThe infrastructure and process of hosting a trained AI model and exposing it as an API endpoint for real-time or batch inference.QuantizationA technique that reduces AI model size and memory requirements by using lower-precision numbers to represent model weights, trading a small accuracy loss for major efficiency gains.Large Language Model (LLM)A neural network trained on massive text datasets that can generate, understand, and reason about human language.