Conda and jupyter tips
some useful commands I daily use
I manage all my python environments with conda
from miniconda
.
However I don't have a strong process to keep track of my environment specifications. Usually I manually create an env.txt
file under my projects. Keeping all commands I have used to create that environment.
!cat /home/explore/git/guillaume/mit_6S191_Intro_to_deep_learning/env\ \ mit_6S191.txt
What happens if I add packages in that environment. Or want to use that environment in another project. I have to remember the link between env name and project name.
That is not robust.
Keeping a yml file could be a solution to keep track of environment specifications. It doesn't answer to my last concern though (linking env name and project name)
But there is a limitation linked with channels.
!conda env export --from-history
In that example, fastai
package should come from fastai
channel but conda doesn't keep that information.
Using
conda install -n my_env rdkit::rdkit
could be an option.
Since conda keeps active environment in env variable CONDA_DEFAULT_ENV
, we can automatically create up-to-date yml file.
!echo $CONDA_DEFAULT_ENV
!conda env export --from-history > ~/temp/env_`echo $CONDA_DEFAULT_ENV`.yml
!ls ~/temp/env_`echo $CONDA_DEFAULT_ENV`.yml
But for it to be usable, I will have to install package using the <channel>::<package>
way.
!cat /home/explore/git/guillaume/mit_6S191_Intro_to_deep_learning/create_yml.sh
When managing conda environments, I very often fall on this documentation page which is simply great: https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html
Next time I visit this page, I will enter entries here to track my common commands.
I have already explained how to install jupyter extensions and the one I use. update jupyter to include extensions)
This is usefull to switch from environment to another without having to stop/restart jupyter.