How To Install Packages from the Jupyter Notebook
and why this is so messy
- Quick Fix: How To Install Packages from the Jupyter Notebook
- The Details: Why is Installation from Jupyter so Messy?
- The Root of the Issue
This is directly from https://jakevdp.github.io/blog/2017/12/05/installing-python-packages-from-jupyter/.
Here are my own experimentations following this article detailed explanations.
import sys
!conda install --yes --prefix {sys.prefix} matplotlib
!echo $PATH
!type python
You can optionally add the -a tag to see all available versions of the command in your current shell environment; for example:
!type -a python
!type -a conda
import sys
sys.path
By default, the first place Python looks for a module is an empty path, meaning the current working directory. If the module is not found there, it goes down the list of locations until the module is found. You can find out which location has been used using the __path__
attribute of an imported module:
import numpy
numpy.__path__
by printing the sys.path variables for each of the available python executables in my path, using Jupyter's delightful ability to mix Python and bash commands in a single code block:
paths = !type -a python
for path in set(paths):
path = path.split()[-1]
print(path)
!{path} -c "import sys; print(sys.path)"
print()
pip install
will install in the Python in the same path:
!type pip
conda install
will install in the active conda envt
!conda env list
The reason both pip and conda default to the conda pytorch environment is that this is the Python environment I used to launch the notebook.
!jupyter kernelspec list
!cat /home/explore/miniconda3/envs/pytorch/share/jupyter/kernels/python3/kernel.json
If you'd like to create a new kernel, you can do so using the jupyter ipykernel command; for example, I created the above kernels for my primary conda environments using the following as a template:
$ source activate myenv
$ python -m ipykernel install --user --name myenv --display-name "Python (myenv)"
The root of the issue is this: the shell environment is determined when the Jupyter notebook is launched, while the Python executable is determined by the kernel, and the two do not necessarily match. In other words, there is no guarantee that the python, pip, and conda in your $PATH will be compatible with the python executable used by the notebook.
Recall that the python
in your path can be determined using
!type python
The Python executable being used in the notebook can be determined using
sys.executable
when the 2 differs, boom!