Reference for learners

Glossary

host

The entire system the GPU is built into, including CPU, main memory, network interface card, etc.

device

The GPU, including its device memory. Often used interchangeably with “GPU”.

host memory

Main memory of the host (CPU RAM). Also called system or main memory.

device memory

Main memory of the device (GPU VRAM). Also called global memory.

host code

Code that executes on the host (CPU).

device code

Code that executes on the device (GPU).

streaming multiprocessor

A subdivision of the GPU, comparable to a CPU core (sometimes called a “GPU core”). Each streaming multiprocessor can execute one or more thread blocks simultaneously. Abbreviated SM.

kernel

A function marked with __global__ (C/C++) or attributes(global) (Fortran) that runs on the device and is usually launched from the host. A kernel cannot have a return value.

thread

A single execution instance of a kernel. Variables declared in a kernel are private to each thread and stored in registers by default.

thread block

A group of threads that are assigned to the same streaming multiprocessor. Within a block, threads are grouped into warps of 32.

grid

The complete set of thread blocks launched by a kernel. Its dimensions are given by the execution configuration.

warp

A group of 32 threads within a thread block that execute in lock-step.

SIMT

Single Instruction Multiple Thread — the CUDA execution model in which all threads execute (almost) the same instructions but on different data.

execution configuration

The <<<gridDim, blockDim>>> arguments of a kernel launch, specifying the dimensions of the grid and of each thread block.

Further reading

Resources for going deeper, grouped by episode. All links were checked on 2026-09-30.

Software setup

Episode 1: CUDA Programming Model

Episode 2: Kernel Rules and Error Handling

Textbooks

  • Kirk, D. B., Hwu, W. W., El Hajj, I.: Programming Massively Parallel Processors: A Hands-on Approach, 4th edition, Morgan Kaufmann, 2022 — the standard textbook; chapters 2–4 cover the data-parallel programming model, the thread hierarchy and the GPU architecture

  • Cheng, J., Grossman, M., McKercher, T.: Professional CUDA C Programming, Wiley, 2014 — a practical treatment with many measured examples; chapter 2 covers the programming model including error handling, chapter 3 the execution model, chapter 4 global memory

  • Wilt, N.: The CUDA Handbook: A Comprehensive Guide to GPU Programming, Addison-Wesley, 2013 — the author maintains a free online edition and the sample code on the linked site; its chapters on the execution model, kernels and memory go far beyond this module

  • Ruetsch, G., Fatica, M.: CUDA Fortran for Scientists and Engineers, 2nd edition, Morgan Kaufmann, 2024 — the reference for the CUDA Fortran tabs, including data transfers and memory types

Reading materials for further learning

The external sources cited throughout this module are collected here. Each is defined once in references.bib and cited with {cite} where relevant in the episodes.

[1]

NVIDIA Corporation. CUDA C++ Programming Guide — Programming Model. URL: https://docs.nvidia.com/cuda/cuda-programming-guide/01-introduction/programming-model.html (visited on 2026-05-28).

[2]

NVIDIA Corporation. CUDA C++ Programming Guide — Hardware Implementation. URL: https://docs.nvidia.com/cuda/cuda-programming-guide/03-advanced/advanced-kernel-programming.html#hardware-implementation (visited on 2026-05-28).

[3]

NVIDIA Corporation. CUDA C++ Programming Guide — CUDA C++ Mathematical Standard Library Functions. URL: https://docs.nvidia.com/cuda/cuda-programming-guide/05-appendices/mathematical-functions.html#cuda-c-mathematical-standard-library-functions (visited on 2026-09-30).

[4]

NVIDIA Corporation. CUDA C++ Programming Guide — Function Execution Space Specifiers. URL: https://docs.nvidia.com/cuda/cuda-programming-guide/05-appendices/cpp-language-extensions.html#execution-space-specifiers (visited on 2026-05-28).

[5]

NVIDIA Corporation. CUDA Fortran Programming Guide — Subroutine / Function Qualifiers. URL: https://docs.nvidia.com/hpc-sdk/compilers/cuda-fortran-prog-guide/index.html#subroutine-function-qualifiers (visited on 2026-09-30).

[6]

NVIDIA Corporation. CUDA C++ Programming Guide — Error Checking. URL: https://docs.nvidia.com/cuda/cuda-programming-guide/02-basics/intro-to-cuda-cpp.html#error-checking-in-cuda (visited on 2026-05-28).