Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
When My Remote Task Is Installing The Python Dependencies

When my remote task is installing the python dependencies --packages requests for example, is there any caching "magic" that is done by the k8 agent? Or is it always the equivalent of pip install <deps> --no-cache-dir ?

  
  
Posted one year ago
Votes Newest

Answers 31


image

  
  
Posted one year ago

basically, can I do local installs vs supplying a requirements.txt

  
  
Posted one year ago

Traceback (most recent call last): File "sfi/imagery/models/training/ldc_train_end_to_end.py", line 26, in <module> from sfi.imagery.models.chip_classifier.eval import eval_chip_classifier ModuleNotFoundError: No module named 'sfi.imagery.models'

  
  
Posted one year ago

which should cover both cases

  
  
Posted one year ago

PYTHONPATH: /home/npuser/.clearml/venvs-builds/3.7/task_repository/commons-imagery-models-py/sfi:/home/npuser/.clearml/venvs-builds/3.7/task_repository/commons-imagery-models-py:/home/npuser/.clearml/venvs-builds/3.7/task_repository/commons-imagery-models-py/sfi/imagery/models/training::/home/npuser/.clearml/venvs-builds/3.7/task_repository/commons-imagery-models-py/sfi:/usr/lib64/python37.zip:/usr/lib64/python3.7:/usr/lib64/python3.7/lib-dynload:/home/npuser/.clearml/venvs-builds/3.7/lib64/python3.7/site-packages:/home/npuser/.clearml/venvs-builds/3.7/lib/python3.7/site-packages

  
  
Posted one year ago

PYTHONPATH is still not working as expected

inside your code if you do :
import os print("PYTHONPATH", os.environ["PYTHONPATH"])what are you getting?

  
  
Posted one year ago

note /home/npuser/.clearml/venvs-builds/3.7/task_repository/commons-imagery-models-py is the correct path

  
  
Posted one year ago

The config change worked but my PYTHONPATH is still not working as expected

  
  
Posted one year ago

note

/home/npuser/.clearml/venvs-builds/3.7/task_repository/commons-imagery-models-py

is the correct pat

So how come it is failing?
Can you also print sys.path just to be sure ?

  
  
Posted one year ago

When I run from sfi.imagery import models. It works fine local. So the repo is setup for proper imports. But fails in clearML tasks

  
  
Posted one year ago

Could it be something else is missing and hence the import fails ?

  
  
Posted one year ago

Yea I've done that already but I can do it again

  
  
Posted one year ago

SysPath: ['/home/npuser/.clearml/venvs-builds/3.7/task_repository/commons-imagery-models-py/sfi/imagery/models/training', '/home/npuser/.clearml/venvs-builds/3.7/task_repository/commons-imagery-models-py/sfi', '/home/npuser/.clearml/venvs-builds/3.7/task_repository/commons-imagery-models-py', '/usr/lib64/python37.zip', '/usr/lib64/python3.7', '/usr/lib64/python3.7/lib-dynload', '/home/npuser/.clearml/venvs-builds/3.7/lib64/python3.7/site-packages', '/home/npuser/.clearml/venvs-builds/3.7/lib/python3.7/site-packages']

  
  
Posted one year ago

/home/npuser/.clearml/venvs-builds/3.7/task_repository/commons-imagery-models-pyYep I see it now, could you simulate locally (i.e have the other folders in the path as well)?
could it be you also have a file somewhere that is called sfi or imagery or models or chip_classifier that it accidently tries to import first from ?

  
  
Posted one year ago

Seems like it has everything I would need

  
  
Posted one year ago

it has sfi suffix and regular root

  
  
Posted one year ago

What's the log you are seeing?

  
  
Posted one year ago

AgitatedDove14 How would I install using a setup.py in a clearML task?

  
  
Posted one year ago

Not yet AgitatedDove14 Perhaps we can pair on this Monday.

  
  
Posted one year ago

BoredHedgehog47 were you able to locate the issue ?

  
  
Posted one year ago

This is to address the PYTHONPATH issues

  
  
Posted one year ago

Would be great if the docker_bash_setup_script had output I could see

  
  
Posted one year ago

I was hoping to use docker_bash_setup_script but it didn't work when I ran pip install -e . in the respective script

  
  
Posted one year ago

If you look lower, it is there '/home/npuser/.clearml/venvs-builds/3.7/task_repository/commons-imagery-models-py'

  
  
Posted one year ago

yea IDK, the git repo is a python library. Is it possible to run something like pip install -e . so I can utilize the setup.py?

  
  
Posted one year ago

When I run this line locally, it works fine
from sfi.imagery.models.chip_classifier.eval import eval_chip_classifier

  
  
Posted one year ago

For instance, In my repo, I have a setup.py, how would I run pip install -e .

  
  
Posted one year ago

Okay, this worked --packages '-e .'

  
  
Posted one year ago
4K Views
31 Answers
one year ago
one month ago
Tags