Working with GPU packages#
The Anaconda Distribution includes several packages that use the GPU as an accelerator to increase performance, sometimes by a factor of five or more. These packages can dramatically improve machine learning and simulation use cases, especially deep learning. Read more about getting started with GPU computing in Anaconda.
While both AMD and NVIDIA are major vendors of GPUs, NVIDIA is currently the most common GPU vendor for machine learning and cloud computing. The information on this page applies only to NVIDIA GPUs.
GPU compatibility#
GPU acceleration requires the author of a project such as TensorFlow to implement GPU-specific code paths for algorithms that can be executed on the GPU. A GPU-accelerated project will call out to NVIDIA-specific libraries for standard algorithms or use the NVIDIA GPU compiler to compile custom GPU code. Only the algorithms specifically modified by the project author for GPU usage will be accelerated, and the rest of the project will still run on the CPU.
For most packages, GPU support is either a compile-time or run-time choice, allowing a variant of the package to be available for CPU-only usage. When GPU support is a compile-time choice, Anaconda will typically need to build two versions of the package, to allow the user to choose between the “regular” version of the project that runs on CPU only and the “GPU-enabled” version of the project that runs on GPU.
Due to the different ways that CUDA support is enabled by project authors, there
is no universal way to detect GPU support in a package. For many GPU-enabled
packages, there is a dependency on the cudatoolkit package. Other packages
such as Numba do not have a cudatoolkit dependency, because they can be used
without the GPU.
Hardware requirements#
NVIDIA released the CUDA API for GPU programming in 2006, and all new NVIDIA GPUs released since that date have been CUDA-capable regardless of market. Although any NVIDIA GPU released in the last 10 years will technically work with Anaconda, these are the best choices for machine learning and specifically model training use cases:
Tesla P100 or V100
Titan RTX
GeForce RTX 3050
Various recent Quadro models
Deployed models do not always require a GPU. When a GPU is required for a deployed model, there are other Tesla GPU models that are more optimized for inference than training, such as the Tesla P4, P40, and T4.
Cloud and on-premise data center deployments require Tesla cards, whereas the GeForce, Quadro, and Titan options are suitable for use in workstations.
Most users will have an Intel or AMD 64-bit CPU. Anaconda recommends having at least two to four times more CPU memory than GPU memory, and at least 4 CPU cores to support data preparation before model training. There are a limited number of Anaconda packages with GPU support for IBM POWER 8/9 systems as well.
Software requirements#
The best performance and user experience for CUDA is on Linux systems. Windows is also supported. No Apple computers have been released with an NVIDIA GPU since 2014, so they generally lack the memory for machine learning applications and only have support for Numba on the GPU.
Anaconda requires that the user has installed a recent NVIDIA driver that meets the version requirements in the table below. Anaconda does not require the installation of the CUDA SDK.
Ubuntu and some other Linux distributions ship with a third party open-source driver for NVIDIA GPUs called Nouveau. CUDA requires replacing the Nouveau driver with the official closed source NVIDIA driver.
All other CUDA libraries are supplied as conda packages.
GPU-enabled packages are built against a specific version of CUDA. Currently
supported versions include CUDA 11.8, 12.0, 12.1, and 12.2. The NVIDIA drivers are designed
to be backward compatible to older CUDA versions, so a system with NVIDIA driver
version 525.60.13 can support CUDA 12.0 packages and earlier. As a result, if a user
is not using the latest NVIDIA driver, they may need to manually pick a
particular CUDA version by selecting the version of the cudatoolkit conda
package in their environment. To select a cudatoolkit version, add a
selector such as cudatoolkit=12.1 to the version specification.
Required NVIDIA driver versions, excerpted from the NVIDIA CUDA Toolkit Release Notes:
CUDA Version |
Linux x86_64 Driver Version |
Windows x86_64 Driver Version |
|---|---|---|
CUDA 11.8.x |
>= 450.80.02 |
>= 452.39 |
CUDA 12.0.x |
>= 525.60.13 |
>= 527.41 |
CUDA 12.1.x |
>= 525.60.13 |
>= 527.41 |
CUDA 12.2.x |
>= 525.60.13 |
>= 527.41 |
Sometimes specific GPU hardware generations have a minimum CUDA version. As of August 27th, 2018, the only relevant constraint is that the Tesla V100 and Titan V (using the “Volta” GPU architecture) require CUDA 9 or later.
Available packages#
TensorFlow#
TensorFlow is a general machine learning library, but most popular for deep
learning applications. There are three supported variants of the tensorflow
package in Anaconda, one of which is the NVIDIA GPU version. This is selected by
installing the meta-package tensorflow-gpu.
conda install tensorflow-gpu
Other packages, such as Keras, depend on the generic tensorflow package name
and will use whatever version of TensorFlow is installed. This makes it straightforward to
switch between variants in an environment.
PyTorch#
PyTorch is another machine learning library with a deep learning focus. PyTorch detects GPU availability at run-time, so the user does not need to install a different package for GPU support.
conda install pytorch
Caffe (Linux only)#
Caffe was one of the first popular deep learning libraries.
conda install caffe-gpu
Chainer/CuPy#
Chainer is a deep learning library that uses NumPy or CuPy for computations.
conda install chainer
Chainer’s companion project CuPy is a GPU-accelerated clone of the NumPy API that can be used as a drop-in replacement for NumPy with a few changes to user code. When CuPy is installed, Chainer is GPU-accelerated. CuPy can also be used on its own for general array computation.
conda install cupy
XGBoost (Windows/Linux only)#
XGBoost is a machine learning library that implements gradient-boosted decision trees. Training several forms of trees is GPU-accelerated.
conda install py-xgboost-gpu
MXNet (Linux only)#
MXNet is a machine learning library supported by various industry partners, most
notably Amazon. Like TensorFlow, it comes in three variants, with the GPU
variant selected by the mxnet-gpu meta-package.
conda install mxnet-gpu
Numba#
Numba is a general-purpose JIT compiler for Python functions. It provides a way
to implement custom GPU algorithms in purely Python syntax when the
cudatoolkit package is present.:
conda install numba cudatoolkit
GPU support in Enterprise#
GPU-enabled conda packages can be used in Data Science & AI Workbench projects when the cluster has resource profiles that include GPUs. For more details see the GPU support section of the Workbench FAQ.