Reference for learners¶
Glossary¶
- host¶
The entire system the GPU is built into, including CPU, main memory, network interface card, etc.
- device¶
The GPU, including its device memory. Often used interchangeably with “GPU”.
- host memory¶
Main memory of the host (CPU RAM). Also called system or main memory.
- device memory¶
Main memory of the device (GPU VRAM). Also called global memory.
- host code¶
Code that executes on the host (CPU).
- device code¶
Code that executes on the device (GPU).
- streaming multiprocessor¶
A subdivision of the GPU, comparable to a CPU core (sometimes called a “GPU core”). Each streaming multiprocessor can execute one or more thread blocks simultaneously. Abbreviated SM.
- kernel¶
A function marked with
__global__(C/C++) orattributes(global)(Fortran) that runs on the device and is usually launched from the host. A kernel cannot have a return value.- thread¶
A single execution instance of a kernel. Variables declared in a kernel are private to each thread and stored in registers by default.
- thread block¶
A group of threads that are assigned to the same streaming multiprocessor. Within a block, threads are grouped into warps of 32.
- grid¶
The complete set of thread blocks launched by a kernel. Its dimensions are given by the execution configuration.
- warp¶
A group of 32 threads within a thread block that execute in lock-step.
- SIMT¶
Single Instruction Multiple Thread — the CUDA execution model in which all threads execute (almost) the same instructions but on different data.
- execution configuration¶
The
<<<gridDim, blockDim>>>arguments of a kernel launch, specifying the dimensions of the grid and of each thread block.
Further reading¶
Resources for going deeper, grouped by episode. All links were checked on 2026-09-30.
Software setup¶
NVIDIA CUDA Compiler Driver (
nvcc) documentation — all command-line options ofnvcc, including the optimisation, debugging and-lineinfoflags used in the setup episodeCUDA Installation Guide for Linux — installing the CUDA Toolkit on your own machine
NVIDIA HPC SDK documentation — the compiler suite that provides
nvfortranfor CUDA FortranCUDA GPU Compute Capability — look up the compute capability of your GPU (see the hardware and software requirements note)
NVIDIA cuda-samples — small, complete example programs for most CUDA features, useful to check that a fresh installation compiles and runs CUDA code, and as reference code for specific features
Episode 1: CUDA Programming Model¶
CUDA C++ Programming Guide — Programming Model — heterogeneous systems, kernels and the thread hierarchy (grid of thread blocks) as introduced in this episode
CUDA Fortran Programming Guide — the reference for the Fortran side of every example (kernels,
dim3, built-in variables, themanagedanddeviceattributes)An Even Easier Introduction to CUDA (NVIDIA Technical Blog) — a first CUDA program, from one thread to many blocks, with the same vector-addition example
CUDA Pro Tip: Write Flexible Kernels with Grid-Stride Loops (NVIDIA Technical Blog) — the grid-stride loop pattern from the last step of the vector-addition example
An Easy Introduction to CUDA Fortran (NVIDIA Technical Blog) — the same first steps in CUDA Fortran
CUDA C++ Best Practices Guide — Thread and Block Heuristics — guidance on choosing block sizes (multiples of the warp size) and grid sizes
Episode 2: Kernel Rules and Error Handling¶
CUDA C++ Programming Guide — C/C++ Language Extensions — function execution space specifiers, built-in variables and the execution configuration in full detail
CUDA C++ Programming Guide — Floating-Point Computation — the mathematical functions available in device code and their accuracy
CUDA Fortran Programming Guide — Kernels and device subprograms — the Fortran rules for
attributes(global),attributes(device)andattributes(host, device)subprogramsCUDA C++ Programming Guide — Error checking — the error state,
cudaGetLastErrorand asynchronous errorsCUDA Runtime API — Error Handling and Device Management — reference pages for
cudaGetErrorString,cudaGetLastError,cudaPeekAtLastErrorandcudaDeviceSynchronizeCompute Sanitizer — NVIDIA’s memory checker for finding the invalid memory accesses behind an “unspecified launch failure”
Textbooks¶
Kirk, D. B., Hwu, W. W., El Hajj, I.: Programming Massively Parallel Processors: A Hands-on Approach, 4th edition, Morgan Kaufmann, 2022 — the standard textbook; chapters 2–4 cover the data-parallel programming model, the thread hierarchy and the GPU architecture
Cheng, J., Grossman, M., McKercher, T.: Professional CUDA C Programming, Wiley, 2014 — a practical treatment with many measured examples; chapter 2 covers the programming model including error handling, chapter 3 the execution model, chapter 4 global memory
Wilt, N.: The CUDA Handbook: A Comprehensive Guide to GPU Programming, Addison-Wesley, 2013 — the author maintains a free online edition and the sample code on the linked site; its chapters on the execution model, kernels and memory go far beyond this module
Ruetsch, G., Fatica, M.: CUDA Fortran for Scientists and Engineers, 2nd edition, Morgan Kaufmann, 2024 — the reference for the CUDA Fortran tabs, including data transfers and memory types
Reading materials for further learning¶
The external sources cited throughout this module are collected here. Each is defined once in references.bib and cited with {cite} where relevant in the episodes.
NVIDIA Corporation. CUDA C++ Programming Guide — Programming Model. URL: https://docs.nvidia.com/cuda/cuda-programming-guide/01-introduction/programming-model.html (visited on 2026-05-28).
NVIDIA Corporation. CUDA C++ Programming Guide — Hardware Implementation. URL: https://docs.nvidia.com/cuda/cuda-programming-guide/03-advanced/advanced-kernel-programming.html#hardware-implementation (visited on 2026-05-28).
NVIDIA Corporation. CUDA C++ Programming Guide — CUDA C++ Mathematical Standard Library Functions. URL: https://docs.nvidia.com/cuda/cuda-programming-guide/05-appendices/mathematical-functions.html#cuda-c-mathematical-standard-library-functions (visited on 2026-09-30).
NVIDIA Corporation. CUDA C++ Programming Guide — Function Execution Space Specifiers. URL: https://docs.nvidia.com/cuda/cuda-programming-guide/05-appendices/cpp-language-extensions.html#execution-space-specifiers (visited on 2026-05-28).
NVIDIA Corporation. CUDA Fortran Programming Guide — Subroutine / Function Qualifiers. URL: https://docs.nvidia.com/hpc-sdk/compilers/cuda-fortran-prog-guide/index.html#subroutine-function-qualifiers (visited on 2026-09-30).
NVIDIA Corporation. CUDA C++ Programming Guide — Error Checking. URL: https://docs.nvidia.com/cuda/cuda-programming-guide/02-basics/intro-to-cuda-cpp.html#error-checking-in-cuda (visited on 2026-05-28).