Fastai on WSL 2 with Cuda

how to NOT get an operational corporate workstation with windows 10, cuda, wsl2, fastai, pytorch…
wsl
fastai
wsl2
cuda
Published

September 21, 2020

This is based on what is explained in https://forums.fast.ai/t/fastai-on-wsl-2-ubuntu-0-7-0-or-any-version/76651

install update of nvidia drivers

Based on Deep Learning Course Forums Platform: Windows 10 using WSL2 w/GPU fastai users

(xgboost) guillaume@LL11LPC0PQARQ:~/git/guillaume_blog$ nvidia-smi.exe
Mon Sep 21 16:00:46 2020
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 460.15       Driver Version: 460.15       CUDA Version: 11.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name            TCC/WDDM | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Quadro M1000M      WDDM  | 00000000:01:00.0  On |                  N/A |
| N/A   59C    P0    N/A /  N/A |    905MiB /  4096MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

install of WSL2 and convert existing images

Open a PowerShell window as an Administrator

Run wsl –set-default-version 2

update KB

{% include alert.html text=“–set-default-version 2 is not a valid option. KB4566116 should be installed” %}

This can be downloaded from Catalog Microsoft Update

update kernel version

If you see this message after running the command: WSL 2 requires an update to its kernel component. For information please visit https://aka.ms/wsl2kernel. You still need to install the MSI Linux kernel update package.

Download from https://docs.microsoft.com/en-us/windows/wsl/install-win10#step-4---download-the-linux-kernel-update-package

set default WSL to be version 2

PS C:\WINDOWS\system32> wsl --set-default-version 2

convert existing images

PS C:\WINDOWS\system32> wsl --list --verbose
  NAME            STATE           VERSION
* Ubuntu-18.04    Running         1
PS C:\WINDOWS\system32> wsl --set-version Ubuntu-18.04 2
La conversion est en cours. Cette opération peut prendre quelques minutes...
Pour plus d’informations sur les différences de clés avec WSL 2, visitez https://aka.ms/wsl2
La conversion est terminée.

It took a while (~1 hour) for my unique ubuntu image.

And at the end it has worked.

PS C:\WINDOWS\system32> wsl --list --verbose
  NAME            STATE           VERSION
* Ubuntu-18.04    Stopped         2

install of nvidia drivers under ubuntu

[Installation instructions)(https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&target_distro=Ubuntu&target_version=1804&target_type=deblocal)

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-ubuntu1804.pin
sudo mv cuda-ubuntu1804.pin /etc/apt/preferences.d/cuda-repository-pin-600
wget https://developer.download.nvidia.com/compute/cuda/11.0.3/local_installers/cuda-repo-ubuntu1804-11-0-local_11.0.3-450.51.06-1_amd64.deb
sudo dpkg -i cuda-repo-ubuntu1804-11-0-local_11.0.3-450.51.06-1_amd64.deb
sudo apt-key add /var/cuda-repo-ubuntu1804-11-0-local/7fa2af80.pub
sudo apt-get update
sudo apt-get -y install cuda
!cat /usr/local/cuda/version.txt
CUDA Version 11.0.228
!/usr/local/cuda/samples/4_Finance/BlackScholes/BlackScholes
[/usr/local/cuda/samples/4_Finance/BlackScholes/BlackScholes] - Starting...
CUDA error at ../../common/inc/helper_cuda.h:777 code=35(cudaErrorInsufficientDriver) "cudaGetDeviceCount(&device_count)"

There is an error when launching CUDA samples. Googling that error maybe my video card is running on low driver version?

I have posted on nvidia (cuda+wsl) forum: https://forums.developer.nvidia.com/t/cuda-sample-throwing-error/142537/18

(update 09-22: it is not possible to have cuda on wsl2 if not in Windows Insider build from Dev Channel. (20145 or higher))

{% include alert.html text=“because I am in version 1909 (18363.1049), it won’t work for me. ;(” %}

cuda for WSL

Here is a link that could be interesting: https://docs.nvidia.com/cuda/wsl-user-guide/index.html

According to this, I should not have installed cuda but cuda-toolkit. Do not choose the cuda, cuda-11-0, or cuda-drivers meta-packages under WSL 2 since these packages will result in an attempt to install the Linux NVIDIA driver under WSL 2.

Is it causing my issue?

apt-get install -y cuda-toolkit-11-0
!/usr/local/cuda/bin/nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2020 NVIDIA Corporation
Built on Wed_Jul_22_19:09:09_PDT_2020
Cuda compilation tools, release 11.0, V11.0.221
Build cuda_11.0_bu.TC445_37.28845127_0

install a new distro (ubuntu 20.04)

Because I cannot use windows store, I have to manually install https://docs.microsoft.com/fr-fr/windows/wsl/install-manual

Installation by just launching Ubuntu_2004.2020.424.0_x64.appx.

I have now 2 distros,

PS C:\WINDOWS\system32> wsl --list -v
  NAME            STATE           VERSION
* Ubuntu-18.04    Running         2
  Ubuntu-20.04    Running         2

WSL2 and network

There is a change of network architecture between WSL 1 and WSL 2. In WSL 2, a new network interface is available:

Carte Ethernet vEthernet (WSL) :

   Suffixe DNS propre à la connexion. . . :
   Adresse IPv6 de liaison locale. . . . .: 
   Adresse IPv4. . . . . . . . . . . . . .: 192.168.81.193
   Masque de sous-réseau. . . . . . . . . : 255.255.255.240
   Passerelle par défaut. . . . . . . . . :

Revert image to WSL1 to get back network access

PS C:\WINDOWS\system32> wsl --set-version Ubuntu-18.04 1
La conversion est en cours. Cette opération peut prendre quelques minutes...
La conversion est terminée.
PS C:\WINDOWS\system32> wsl --list -v
  NAME            STATE           VERSION
* Ubuntu-18.04    Stopped         1
  Ubuntu-20.04    Stopped         2

access to linux files from windows

For running state distros:

Files are available at \\wsl$\.

For stopped state distros:

Files are available at C:\Users\<users>\AppData\Local\Packages\CanonicalGroupLimited.Ubuntu*\LocalState\rootfs

some usefull wsl commands

List distros

!wsl --list --verbose
  NAME            STATE           VERSION

* Ubuntu-18.04    Running         1

  Ubuntu-20.04    Stopped         2

Stop a distro

!wsl --terminate Ubuntu-18.04

Update dns settings

as explained here

just switch from generateResolvConf = true to generateResolvConf = false in /etc/wsl.conf and edit /etc/resolv.conf

But still have issues, mainly I think linked to Symantec Endpoint protection.

Workaround network issue with WSL2

https://github.com/sakai135/wsl-vpnkit

installation setup

vpnkit

install docker for windows

install genisoimage in ubuntu (from http://archive.ubuntu.com/ubuntu/pool/main/c/cdrkit/genisoimage_1.1.11-3.1ubuntu1_amd64.deb)

install vpnkit

isoinfo -i /mnt/c/Program\ Files/Docker/Docker/resources/wsl/docker-for-wsl.iso -R -x /containers/services/vpnkit-tap-vsockd/lower/sbin/vpnkit-tap-vsockd > ./vpnkit-tap-vsockd
chmod +x vpnkit-tap-vsockd
sudo mv vpnkit-tap-vsockd /sbin/vpnkit-tap-vsockd
sudo chown root:root /sbin/vpnkit-tap-vsockd

npiperelay

install unzip in ubuntu (from http://archive.ubuntu.com/ubuntu/pool/main/u/unzip/unzip_6.0-25ubuntu1_amd64.deb)

download npiprelay (from https://github.com/jstarks/npiperelay/releases/download/v0.1.0/npiperelay_windows_amd64.zip )

install npiprelay

unzip npiperelay_windows_amd64.zip npiperelay.exe
rm npiperelay_windows_amd64.zip
mkdir -p /mnt/c/bin
mv npiperelay.exe /mnt/c/bin/
sudo ln -s /mnt/c/bin/npiperelay.exe /usr/local/bin/npiperelay.exe

socat

install socat in ubuntu (from http://archive.ubuntu.com/ubuntu/pool/main/s/socat/socat_1.7.3.3-2_amd64.deb)

Configure DNS for WSL

Disable WSL from generating and overwriting /etc/resolv.conf.

sudo tee /etc/wsl.conf <<EOL
[network]
generateResolvConf = false
EOL

Manually set DNS servers to use when not running this script. 1.1.1.1 is provided as an example.

sudo tee /etc/resolv.conf <<EOL
nameserver 1.1.1.1
EOL

wsl-vpnkit

from https://github.com/sakai135/wsl-vpnkit/archive/refs/heads/main.zip

unzip ~/git/wsl-vpnkit-main.zip -d ~/Applications/wsl-vpnkit

execution

~/Applications/wsl-vpnkit/wsl-vpnkit-main$ sudo ./wsl-vpnkit

proxychains

I can now use proxychains and everything works beautifully ;)

clean from local deb install to ubuntu repo

~/git$ ls *.deb
genisoimage_1.1.11-3.1ubuntu1_amd64.deb                proxychains_3.1-8.1_all.deb
libproxychains3_3.1-8.1_amd64.deb                      socat_1.7.3.3-2_amd64.deb
net-tools_1.60+git20180626.aebd88e-1ubuntu1_amd64.deb  unzip_6.0-25ubuntu1_amd64.deb
sudo apt remove genisoimage  net-tools  socat unzip
sudo proxychains apt install genisoimage libproxychains3 net-tools proxychains socat unzip

I don’t exactly see how to do it with proxychains.