So I run the same script as part of a git repo - but unfortunately the package is still missing.
I'm not sure if it matters but 'kwcoco' is being imported inside one of the repo's functions and not on the script's header.
Hi CrookedWalrus33
When we enqueue the task to run remotely, not all conda packages are installed,
Yes it actually lists all the python packages inside "installed packages" regradless of whether they are coming from pip / conda. Internally it holds the conda part in a separate section (maybe we should present it?!)
and the task is failing (they
Can you provide the log for the Task executed by the agent?
I'm not sure if it matters but 'kwcoco' is being imported inside one of the repo's functions and not on the script's header.
Should work.
when you run pip freeze inside the same env what are you getting ?
Also, is there anyother import that is missing? (basically 'clearml' tryies to be smart, and see if maybe the script itself, even though inside a repo, is not actually importing anything from the repo itself, and if this is the case it will only analyze the original script. Basically are you importing any "local" modules inside the original script is the way it detects it)
It's in my local conda environment though.
Meaning this is a wheel installed manually in conda? or is it a folder inside the conda environment ?
Let me try to add some color to this process analysis process.
Basically clearml will try to statically analyze the code (i.e. look for import/from packages)
Then it will list them in a pip requirements.txt format under installed packages.
When running inside conda environment, it will check which packages were installed via "conda install" (instead of pip install) and mark them internally. This process ensures that when the clearml-agent is running with conda package manager, it "knows" which packages need to be installed with Conda and which with pip.
I'm hoping this helps 🙂
Going back to the original issue. My point is, it seems the "kwcoco" is not being detected, am I correct?
Are you running the code inside a git repository (this means the static analysis will go over all the files in the project), or is it a single script (in which case only the main python script is analyzed) ?
BTW: You can also ask clearml to list the Entire local conda/pip environment with:Task.force_requirements_env_freeze()
It was installed by 'pip install kwcoco' while my conda env was active.
Well I guess my question is, how does conda know ehere to install it form, if this is not on the public channels ? is there a specific conda channel you added (or preconfigured) ?
AgitatedDove14
It was installed by 'pip install kwcoco' while my conda env was active.
Not sure if it answers your question..
CrookedWalrus33 from the log it seems the code is trying to use "kwcoco" but it is not listed under any "Installed packages" nor do you see any attempt to install it. Can you confirm ?
Thank for the great explanation! Now it makes much more sense.
You are right about the issue that 'kwcoco' isn't being detected, and Im actually running this as a single script, and the kwcoco not imported directly (but from within another package).
Ill try running it from a repo and see how it works.
Maybe I should understand better how this actually works.
It's not a rare use case to use conda environments, but install some packages using pip (when they're not available at any conda channel). Will that use case be supported on clearml?
No. But this isn't the only package that was installd using pip inside a conda env.
Compare it with Pandas as an example - was installed using pip nad it's actually instaled inside the worker as well (using pip).
AgitatedDove14 Yes, thats correct.
It's in my local conda environment though.
Nice!
script, and the kwcoco not imported directly (but from within another package).
fyi: usually the assumption is that clearml will only list the directly imported packages, as these will pull the respective required packages when the agent will be installing them ... (meaning that if in the repository you are never actually directly importing kwcoco, it will not be listed (the package that you do import directly, the you mentioned is importing kwcoco, will be listed). I hope this helps 🙂
CrookedWalrus33 , Hi 🙂
Can you please provide which packages are missing?
Yes.
Its kwcoco
This is my conda env export :
name: clearml
channels:
- defaults
dependencies: - ca-certificates=2021.10.26=hecd8cb5_2
- certifi=2021.10.8=py39hecd8cb5_0
- libcxx=12.0.0=h2f01273_0
- libffi=3.3=hb1e8313_2
- ncurses=6.3=hca72f7f_2
- openssl=1.1.1l=h9ed2024_0
- pip=21.2.4=py39hecd8cb5_0
- python=3.9.7=h88f2d9e_1
- readline=8.1=h9ed2024_0
- setuptools=58.0.4=py39hecd8cb5_0
- sqlite=3.36.0=hce871da_0
- tk=8.6.11=h7bc2e8c_0
- tzdata=2021e=hda174b7_0
- wheel=0.37.0=pyhd3eb1b0_1
- xz=5.2.5=h1de35cc_0
- zlib=1.2.11=h1de35cc_3
- pip:
- attrs==20.3.0
- backports-entry-points-selectable==1.1.1
- chardet==4.0.0
- charset-normalizer==2.0.8
- clearml==1.1.4
- clearml-agent==1.1.1
- cycler==0.11.0
- cython==0.29.24
- distlib==0.3.3
- ffmpeg-python==0.2.0
- filelock==3.4.0
- fonttools==4.28.2
- furl==2.1.3
- future==0.18.2
- idna==2.10
- imageio==2.12.0
- joblib==1.1.0
- jsonschema==3.2.0
- kiwisolver==1.3.2
- kwarray==0.5.21
- kwcoco==0.2.17
- kwimage==0.7.14
- matplotlib==3.5.0
- networkx==2.6.3
- numpy==1.21.4
- opencv-python==4.5.4.60
- ordered-set==4.0.2
- orderedmultidict==1.0.1
- packaging==21.3
- pandas==1.3.4
- parse==1.19.0
- pathlib2==2.3.6
- pillow==8.4.0
- platformdirs==2.4.0
- psutil==5.8.0
- pycocotools==2.0.3
- pyhocon==0.3.58
- pyjwt==2.0.1
- pyparsing==2.4.7
- pyrsistent==0.18.0
- python-dateutil==2.8.2
- pytz==2021.3
- pywavelets==1.2.0
- pyyaml==5.4.1
- requests==2.25.1
- scikit-image==0.18.3
- scikit-learn==1.0.1
- scipy==1.7.3
- scriptconfig==0.5.8
- setuptools-scm==6.3.2
- shapely==1.8.0
- sortedcontainers==2.4.0
- threadpoolctl==3.0.0
- tifffile==2021.11.2
- tomli==1.2.2
- tqdm==4.62.3
- typing==3.7.4.3
- ubelt==0.10.1
- uritools==3.0.2
- urllib3==1.26.7
- virtualenv==20.10.0
- xarray==0.20.1