For instance, In my repo, I have a setup.py, how would I run pip install -e .

Okay, this worked --packages '-e .'

When I run from sfi.imagery import models. It works fine local. So the repo is setup for proper imports. But fails in clearML tasks

Not yet AgitatedDove14 Perhaps we can pair on this Monday.

Could it be something else is missing and hence the import fails ?

which should cover both cases

The config change worked but my PYTHONPATH is still not working as expected

basically, can I do local installs vs supplying a requirements.txt

This is to address the PYTHONPATH issues

Seems like it has everything I would need

When I run this line locally, it works fine
from sfi.imagery.models.chip_classifier.eval import eval_chip_classifier

BoredHedgehog47 were you able to locate the issue ?

If you look lower, it is there '/home/npuser/.clearml/venvs-builds/3.7/task_repository/commons-imagery-models-py'

Would be great if the docker_bash_setup_script had output I could see

it has sfi suffix and regular root

PYTHONPATH: /home/npuser/.clearml/venvs-builds/3.7/task_repository/commons-imagery-models-py/sfi:/home/npuser/.clearml/venvs-builds/3.7/task_repository/commons-imagery-models-py:/home/npuser/.clearml/venvs-builds/3.7/task_repository/commons-imagery-models-py/sfi/imagery/models/training::/home/npuser/.clearml/venvs-builds/3.7/task_repository/commons-imagery-models-py/sfi:/usr/lib64/python37.zip:/usr/lib64/python3.7:/usr/lib64/python3.7/lib-dynload:/home/npuser/.clearml/venvs-builds/3.7/lib64/python3.7/site-packages:/home/npuser/.clearml/venvs-builds/3.7/lib/python3.7/site-packages

I was hoping to use docker_bash_setup_script but it didn't work when I ran pip install -e . in the respective script

Seems related to this https://github.com/allegroai/clearml/issues/241

AgitatedDove14 How would I install using a setup.py in a clearML task?

yea IDK, the git repo is a python library. Is it possible to run something like pip install -e . so I can utilize the setup.py?

Traceback (most recent call last): File "sfi/imagery/models/training/ldc_train_end_to_end.py", line 26, in <module> from sfi.imagery.models.chip_classifier.eval import eval_chip_classifier ModuleNotFoundError: No module named 'sfi.imagery.models'

/home/npuser/.clearml/venvs-builds/3.7/task_repository/commons-imagery-models-pyYep I see it now, could you simulate locally (i.e have the other folders in the path as well)?
could it be you also have a file somewhere that is called sfi or imagery or models or chip_classifier that it accidently tries to import first from ?

SysPath: ['/home/npuser/.clearml/venvs-builds/3.7/task_repository/commons-imagery-models-py/sfi/imagery/models/training', '/home/npuser/.clearml/venvs-builds/3.7/task_repository/commons-imagery-models-py/sfi', '/home/npuser/.clearml/venvs-builds/3.7/task_repository/commons-imagery-models-py', '/usr/lib64/python37.zip', '/usr/lib64/python3.7', '/usr/lib64/python3.7/lib-dynload', '/home/npuser/.clearml/venvs-builds/3.7/lib64/python3.7/site-packages', '/home/npuser/.clearml/venvs-builds/3.7/lib/python3.7/site-packages']

Yea I've done that already but I can do it again

PYTHONPATH is still not working as expected

inside your code if you do :
import os print("PYTHONPATH", os.environ["PYTHONPATH"])what are you getting?

note /home/npuser/.clearml/venvs-builds/3.7/task_repository/commons-imagery-models-py is the correct path

BoredHedgehog47
is this ( https://clearml.slack.com/archives/CTK20V944/p1665426268897429?thread_ts=1665422655.799449&cid=CTK20V944 ) the same issue (or solution) ?

Notice you have in the Path:
/home/npuser/.clearml/venvs-builds/3.7/task_repository/commons-imagery-models-py/sfiBut you should have:
/home/npuser/.clearml/venvs-builds/3.7/task_repository/commons-imagery-models-py/

note

/home/npuser/.clearml/venvs-builds/3.7/task_repository/commons-imagery-models-py

is the correct pat

So how come it is failing?
Can you also print sys.path just to be sure ?