Look a little closer · Lesson 14 of 18
How a GPU groups its work
A GPU groups small pieces of work together, then runs those groups on parts of the chip called SMs.
How it works, step by step
Give each piece its work
Each piece runs a copy of the same program. One copy is called a thread.
128 threads, each with its own work.
Make groups of 32
Group threads 0 to 31, 32 to 63, 64 to 95, and 96 to 127. Each group is a warp.
128 ÷ 32 = 4 warps in this block.
Run the groups
An SM is a part of the GPU. It picks groups that are ready and runs their instructions.
The work moves forward as the needed data becomes ready.
In plain language
A thread is one copy of a program doing its own work, and a warp groups 32 threads. Parts of the GPU called streaming multiprocessors, or SMs, run these groups.
A way to picture it
Picture tasks sorted into folders of 32, with each work area handling several folders. The folders describe how work is grouped, not the physical layout of the chip.
A worked example
A block is a group of threads given to the GPU together. A block of 128 threads contains 128 ÷ 32 = 4 warps. That does not reserve 128 physical calculation units.
Keep in mind
A thread is a running program copy, not a physical GPU core. Threads in a warp may take different paths, so some can sit out an instruction. Four warps do not promise four instructions finish at once.
What these words mean
- Thread
- One running copy of a program, not a physical GPU core.
- Warp
- A group of 32 CUDA threads. This is a work grouping, not a row of 32 physical cores.
- SM
- Short for streaming multiprocessor: a part of the GPU that runs groups of threads and holds their nearby working data.
- CUDA
- NVIDIA's platform for writing and running programs that use its GPUs.
- Different paths
- Threads in one warp can make different choices, so some may sit out an instruction while others run it.
How many groups of 32 fit into 128 threads?
4. 128 ÷ 32 = 4 warps. This counts groups of work, not physical cores or instructions finished in one tick of the chip's clock.
Where this comes from
This explanation is checked against primary documentation. The small arithmetic examples are ours and are not hardware measurements or vendor benchmarks.