If you have a Miniconda3 installation in your /work directory on Discovery, you may need to make some changes to continue to use those same Miniconda environments on Explorer. Since miniconda hard-codes paths through /work directories, a Miniconda base environment made for Discovery cannot be activated on Explorer (resulting in “No such file or directory” error).
You can, however, still use the environments that you created by loading a Miniconda3 module and activating the individual environment. This allows you to use those same Discovery environments on Explorer.
We recommend making the changes below if:
- You have JupyterLab installed in your Miniconda environment and want to use it on the Explorer OOD or in the terminal.
- You want to use your base Miniconda environment to create new environments in /projects on Explorer (this will likely be true for most of you).
I will show you two methods:
- A temporary solution to get you back to work immediately
- A long-term solution that, depending on how many environments you have, may take an hour or two to implement fully (my example with one environment took 20 minutes)
Temporary Solution
This is just to get an environment working on Explorer that runs through the Jupyter lab OOD application. This assumes you installed Jupyter lab in the environment on Discovery.
Please Note: These changes will make the environment work ONLY on Explorer (no going back to Discovery).
On Explorer, uninstall the JupyterLab package in your environment, and then reinstall it.
Why? The Jupyter executable in the Miniconda environment uses a hard-coded path through /work to run the Python executable, this errors out in the OOD causing the application to go directly to completed. By removing JupyterLab and installing it again in a /projects directory, we set the path to be through /projects and all is well for the Explorer OOD.
Long-Term Solution
This will show you how to install a new Miniconda3 in your /projects directory on Explorer and recreate (clone) all of your /work Discovery environments. This will result in a Miniconda that still works on Discovery (so you can go back), and a new Miniconda that now works on Explorer (so you can go forward). This also preserves state for past environments which may be necessary for publications and scientific reproducibility.
On Explorer, reinstall a new Miniconda3 and clone your environments from your old miniconda3 (created on Discovery).
Be sure to change “yourdirectory” to the name of your /projects space (or where you want to install miniconda3_explorer).
Clone environments from old miniconda3. You may want to keep the name of the environment the same, but that’s up to you.
The environment you are cloning is indicated by the full path after —clone
The new environment you are creating is indicated by the name after -n
Note you will note be able to see the Discovery created miniconda3 environments when you run “conda env list”. Environments are at typically at this path: miniconda3/envs/
You now have a miniconda3 with all of your Discovery compatible environments, and a new miniconda3_explorer for Explorer where all of your environments have been cloned.
As always we recommend activating conda environments with the command “source activate env_name” or “source activate /projects/path/to/env”
Please do not run “conda init” or use “conda activate env_name” as this edits your .bashrc and can lead to disrupted performance of the Open OnDemand website.
Serena has extensive experience with non-model organism genomics, including de novo genome or transcriptome assembly, genome annotation, genome liftover, variant calling, and population genetics. She is eager to help biologists with their computing needs and is skilled in experimental design (GWAS, Population genetics, and quantitative genetics), data analysis, and visualization in RStudio. Serena earned her PhD from UC Davis in Population Biology.
