[NON-ANVIL] Using Conda ENVs Inside BASH Scripts In cron Jobs

Anvil Q&A
1

Just spent a long time trying to get his working. The problem with most of the SO answers is that they are too simplistic or just don’t seem to work.

This, however, seems to work on Centos 7 :

# Bash

. $HOME/.bash_profile
conda activate myvenv
python /opt/stuff/my_python_program.py

The first line seems to correctly set up the temporary bash environment that cron starts, so as long as you have previously set up conda for your bash environment (conda init bash) then your temporary bash session should inherit that.

Would be grateful of someone more experienced in this than me would vet this for accuracy and caveats.

(and I cannot for the life of me remember where I read it)

2

Yes, that’s the correct way.

The last time I used unix was 30 years ago, but this seems to confirm my feeble memory.

1 Like
3

That was even the one I read and couldn’t find again :slight_smile:

Cheers, @stefano.menci

4

If you wanted you could use a shebang at the top of your python script to specify the environment it should run in a la:

#!/home/username/anaconda/envs/envname/bin/python

This would allow you to decouple your bash scripts from your environments and keep them in the python file. The major downside here is that you have to use the relative path of the environment instead of the name. So if you were doing something like a dockerized deployment this might be annoying.

Apparently you could also use the conda run command, but I’ve seen odd things about it on the conda Github tracker that says its broken. Here is an example: #!/usr/bin/env conda run -n my_env python

SO link explaining conda run: https://stackoverflow.com/questions/41914739/how-do-i-activate-a-conda-env-in-a-subshell

5

I tried that, but I found that it did not necessarily load all the environment vars. Some of the dependencies didn’t load.

1 Like
6

Oooh interesting. I’ll keep that in mind as I do this myself!

7

I’m not sure what the threshold for it failing is, because on some scripts it has worked fine. There’s obviously some tipping point (maybe certain dependencies).

1 Like