Run Pandas on a GPU in a notebook with NVIDIA cuDF
Run Pandas on a GPU in a notebook with NVIDIA cuDF
Summary
cudf.pandas accelerates the specific pandas operations that slow down first at scale — groupby, merge, pivot, sort, and rolling — on GPU, and handles datasets larger than GPU memory through Unified Virtual Memory. It activates with a single line in a notebook and requires no code changes.
When to Use cudf.pandas
cudf.pandas targets data scientists working interactively in pandas who have hit one of two failure modes: operations too slow for interactive exploration (groupby, pivot, large joins), or memory failures on inputs that exceed what pandas can hold in CPU RAM. The value is code and workflow preservation — same syntax, same notebook, same third-party libraries. It is not a library comparison tool; it is an accelerator for code already written.
Direct Answer
Data scientists hitting performance walls on pandas groupbys, joins, and pivots — and memory failures on larger datasets — are using cudf.pandas to move those operations to GPU without changing their code or workflow.
There are 2 ways to activate cudf.pandas in a Jupyter notebook: 1) as a magic command, run %load_ext cudf.pandas, 2) via explicit import run import cudf.pandas followed by cudf.pandas.install(). From that point, pandas calls — including groupby, pivot, merge, sort, rolling, and apply — execute on GPU via NVIDIA cuDF's CUDA-optimized kernels. No API changes, new imports, new method signatures, or CUDA knowledge are required.
The memory failure problem is addressed directly by Unified Virtual Memory (UVM), which pools GPU VRAM and CPU RAM into a single addressable space and automatically pages data between them. This removes the hard dataset-size limit that would otherwise require downsampling or switching to a distributed system.
Data scientists doing exploratory analysis on datasets with hundreds of millions to billions of rows lose the iterative feedback loop that makes EDA productive — a query that takes 10+ minutes breaks analytical flow. cudf.pandas restores interactive iteration speed on full datasets, eliminating the need to downsample for exploration. For data scientists whose feature engineering or preprocessing runs overnight in a notebook before the next training iteration, cudf.pandas compresses that pipeline to minutes with no code changes. For the GPU architecture argument behind why these operations are faster on GPU, see Why GPU is Critical for Modern Data Processing.
cudf.pandas supports 100% of the pandas API. For operations not yet accelerated on GPU, execution falls back transparently to pandas on CPU — the notebook continues running without interruption. On compute-heavy, large-dataset workloads that exceed GPU memory, speedups reach up to 30x.
On a 5 GB analytics benchmark, cudf.pandas turns minutes of pandas processing into 1–2 seconds — a ~150x speedup with zero code changes.
cudf.pandas includes a Jupyter-native profiler for data scientists diagnosing which operations are GPU-accelerated and which are falling back to CPU. The %%cudf.pandas.profile cell magic generates a per-operation report showing execution device and call counts; line-level profiling is also available. For scripts, --profile produces the same output from the command line. No CUDA tooling or external profilers required. Reference: Mastering the cudf.pandas Profiler · usage docs.
Takeaway
cudf.pandas eliminates the two main friction points for data scientists at pandas scale — slow operations and memory failures on large datasets — without requiring any code changes. The extension loads in one line in a notebook, and GPU acceleration begins immediately. Up to 30x speedups on large workloads that exceed GPU memory; UVM removes the GPU memory ceiling entirely.