Start here · Lesson 2 of 18
Where the numbers are kept
Memory stores numbers; how much it holds and how quickly it moves them are two different things.
How it works, step by step
Count the storage
A byte holds 8 bits. Multiply 16 by 16, then divide by 8 to count the bytes.
16 × 16 ÷ 8 = 32 bytes for the numbers.
Find where it is kept
These machines keep the numbers in different places. Spark lets its CPU and GPU share the same memory.
Different memory types, but the numbers still take 32 bytes.
Bring numbers to the work
The GPU brings the needed numbers to its calculation units. Nearby storage can keep copies for reuse.
The calculation has the numbers it needs.
In plain language
Memory holds the numbers a computer is working with. Capacity means how much it holds, while bandwidth means how much data can move through a connection each second.
A way to picture it
Capacity is the size of a pantry. Bandwidth is how quickly you can bring ingredients through its doorway. A bigger pantry does not guarantee a wider doorway.
A worked example
A byte contains 8 bits. Storing 16 numbers with 16 bits each needs 16 × 16 ÷ 8 = 32 bytes. The same numbers still need 32 bytes in HBM, GDDR or LPDDR, which are different kinds of memory.
Keep in mind
HBM sits beside a GPU chip on its package; GDDR is commonly placed on a graphics card. Spark shares LPDDR system memory between CPU and GPU. CUDA shared memory is small storage inside an SM, not the same as that shared system memory. These storage locations do not all take the same time to access.
What these words mean
- Bit and byte
- A bit is one 0 or 1 in computer data. A byte holds 8 bits.
- Capacity
- How much data memory can hold.
- Bandwidth
- How much data can move through a particular connection each second.
- HBM / GDDR / LPDDR
- Different memory types. HBM sits beside the GPU chip on its package; GDDR commonly sits on a graphics card; Spark uses shared LPDDR system memory.
- Cache
- Storage that keeps copies close to where they are needed. Reusing a copy can avoid fetching the data again from farther away.
- CUDA shared memory
- Small storage inside an SM, which is a part of the GPU. It is not the same as CPU/GPU shared system RAM.
Does more room in memory always mean a faster answer?
No, moving the data and doing the work also take time. More memory means more room for data. The answer still depends on how fast data moves and how the computer and software do the work.
Where this comes from
This explanation is checked against primary documentation. The small arithmetic examples are ours and are not hardware measurements or vendor benchmarks.