GPU (Graphics Processing Unit)
A GPU is a processor designed for parallel computation — originally for rendering graphics, now the dominant hardware for training and running AI models. Without GPUs, modern AI as we know it would not exist at the scale it does today.
What is a GPU?
A GPU (Graphics Processing Unit) is a specialised processor that can execute thousands of small calculations simultaneously. This parallelism, originally designed to render pixels in video games, turns out to be exactly what neural network training requires: thousands of matrix multiplications happening in parallel, over and over, across millions of training examples.
A standard CPU (Central Processing Unit) has a handful of powerful cores optimised for sequential tasks. A GPU has thousands of smaller cores designed for parallel tasks. For AI workloads, a high-end GPU can be 10 to 100 times faster than a CPU for the same operation.
GPUs in AI Training and Inference
GPUs are used at two stages of the AI lifecycle:
Training: Building a model from scratch or fine-tuning an existing one requires running forward and backward passes through a neural network billions of times. This is computationally intensive and essentially impossible at scale without GPU clusters. Training a large foundation model requires thousands of GPUs running for weeks or months.
Inference: Running a trained model on new inputs — processing a document, answering a question, generating a response — also benefits from GPU acceleration, especially at high volumes. Cloud AI APIs run inference on GPU infrastructure and expose it via API calls.
GPUs in Operations
Most operational users of AI do not manage GPUs directly. When you call an AI API to process an invoice or classify an exception, the GPU compute is abstracted into a per-token cost. Where GPU decisions matter operationally: choosing between a cloud API (pay-per-use, no infrastructure) versus self-hosted models (upfront GPU cost, lower marginal cost at high volume). For most midsize manufacturers and wholesalers, cloud APIs are the right starting point. The economics of on-premise GPU infrastructure only change at very high processing volumes.