CUDA (Compute Unified Device Architecture) is a parallel computing platform developed by NVIDIA. CUDA allows developers to utilize the power of GPUs to accelerate their applications. CUDNN (CUDA Deep Neural Network) is a library developed by NVIDIA that accelerates the training and inference process in deep learning.

The use of CUDA and CUDNN is very important for deep learning application developers, because by using CUDA and CUDNN, the training and inference process of deep learning models will be faster than using CPU alone.

CUDA Installation

Before installation, we first make sure whether our GPU supports CUDA, run the following command in the terminal:

1

lspci | grep -i nvidia

If you already have the driver installed, we need to remove the driver first. To remove the NVIDIA driver, run the following command:

1
2
3
4
5

sudo apt purge nvidia* -y
sudo apt remove nvidia-* -y
sudo rm /etc/apt/sources.list.d/cuda*
sudo apt autoremove -y && sudo apt autoclean -y
sudo rm -rf /usr/local/cuda*

Make sure our operating system is updated, run the following command:

1
2

sudo apt update
sudo apt upgrade -y

Then install some other packages that are needed:

1

sudo apt install g++ freeglut3-dev build-essential libx11-dev libxmu-dev libxi-dev libglu1-mesa libglu1-mesa-dev

Add the CUDA repository to our operating system:

1
2

sudo add-apt-repository ppa:graphics-drivers/ppa
sudo apt update

To see the recommended driver for our GPU, run the following command:

1

ubuntu-drivers devices

An example output of the above command is as follows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

== /sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0 ==
modalias : pci:v000010DEd00001C8Dsv0000103Csd0000838Fbc03sc00i00
vendor   : NVIDIA Corporation
model    : GP107M [GeForce GTX 1050 Mobile]
driver   : nvidia-driver-470-server - distro non-free
driver   : nvidia-driver-470 - distro non-free
driver   : nvidia-driver-450-server - distro non-free
driver   : nvidia-driver-390 - distro non-free
driver   : nvidia-driver-535 - distro non-free recommended
driver   : nvidia-driver-535-server - distro non-free
driver   : nvidia-driver-418-server - distro non-free
driver   : nvidia-driver-545 - distro non-free
driver   : xserver-xorg-video-nouveau - distro free builtin

In the example above, the recommended driver is nvidia-driver-535, so we will install the driver:

1

sudo apt install nvidia-driver-535 libnvidia-gl-535 libnvidia-common-535

After the above command has finished running, reboot our operating system:

1

sudo reboot

Check NVIDIA Driver

After our operating system reboots, we need to make sure the NVIDIA driver is installed correctly. To check, run the following command:

1

nvidia-smi

If the NVIDIA driver is installed correctly, the following output will appear:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.171.04             Driver Version: 535.171.04   CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA GeForce GTX 1050        Off | 00000000:01:00.0  On |                  N/A |
| N/A   45C    P5              N/A / ERR! |     49MiB /  4096MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|    0   N/A  N/A      1619      G   /usr/lib/xorg/Xorg                           45MiB |
+---------------------------------------------------------------------------------------+

CUDA Toolkit Installation

For CUDA Toolkit installation, we need to match the version of Tensorflow that we want to use.

The following table shows the CUDA Toolkit version that matches the Tensorflow version:

For more details, please see the tensorflow documentation. : https://www.tensorflow.org/install/source#gpu

Here I will exemplify the installation of CUDA Toolkit version 11.8, here are the steps:

Open the CUDA Toolkit download page according to the desired version at : https://developer.nvidia.com/cuda-toolkit-archive

Select the desired CUDA Toolkit version, here I will choose version 11.8, then select the OS, Architecture, and distro that matches our operating system. For example I will use the installer type runfile (local). Then download the file.

Follow the steps given by the CUDA Toolkit installer, and make sure to install the NVIDIA drivers included in the CUDA Toolkit installer. For example, I will run the following command for CUDA Toolkit installation:

1

sudo sh cuda*.run

The above command will run the CUDA Toolkit installer, follow the steps given by the installer.

Because we have previously installed the required drivers, in the CUDA installation with runfile we do not need to install the NVIDIA driver included in the CUDA Toolkit installer. For that we need to change the NVIDIA driver installation option by unchecking the option. In the image above, we need to uncheck the Driver option.

Then follow the installation steps until it is complete.

After the installation is complete, we need to add the CUDA environment variable into the .bashrc file run the command line by line:

1
2
3
4

echo 'export PATH=/usr/local/cuda-11.8/bin:$PATH' >> ~/.bashrc
echo 'export LD_LIBRARY_PATH=/usr/local/cuda-11.8/lib64:$LD_LIBRARY_PATH' >> ~/.bashrc
source ~/.bashrc
sudo ldconfig

The above command will add the CUDA environment variable to the .bashrc file and update the environment variable. sudo ldconfig will update the libraries in our operating system.

CUDNN Installation

Open the CUDNN download page for the desired version at : https://developer.nvidia.com/rdp/cudnn-archive. Here I will install CUDNN version 8.6.0, select the desired version, then download the file.

After the CUDNN file has been downloaded, extract the file:

1
2

tar -xvf cudnn-linux-x86_64-8.6.0.163_cuda11-archive.tar.xz
sudo mv cudnn-linux-x86_64-8.6.0.163_cuda11-archive cuda

Then copy the necessary files into the CUDA Toolkit directory:

1
2
3

sudo cp -P cuda/include/cudnn.h /usr/local/cuda-11.8/include
sudo cp -P cuda/lib/libcudnn* /usr/local/cuda-11.8/lib64/
sudo chmod a+r /usr/local/cuda-11.8/lib64/libcudnn*

After that, we need to add the CUDNN environment variable to the .bashrc file:

1
2

echo 'export LD_LIBRARY_PATH=/usr/local/cuda-11.8/lib64:$LD_LIBRARY_PATH' >> ~/.bashrc
source ~/.bashrc

After that, we need to check whether the CUDA and CUDNN installation was successful or not, run the following command:

1

nvcc --version

If the CUDA and CUDNN installation is successful, the following output will appear:

1
2
3
4
5

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Sep_21_10:33:58_PDT_2022
Cuda compilation tools, release 11.8, V11.8.89
Build cuda_11.8.r11.8/compiler.31833905_0

Conclusion

Until now, we have successfully installed CUDA and CUDNN. With the installation of CUDA and CUDNN, we can utilize the GPU to speed up the training and inference process in deep learning. Furthermore, we can install deep learning libraries such as Tensorflow, Pytorch, and others that utilize CUDA and CUDNN to speed up the training and inference process of deep learning models. If you have any questions, please write them in the comments column. Thank you for reading this article.