What's the "working dir" ? (where in the repo the script is executed from)

test_clearml , so directly from top-level.

Thanks for your help again. I will just use detect_with_conda_freeze: true . Seems like a perfect solution for me!

Okay. It works now. I don't know what went wrong before. Probably a user error 😅

Let me try it another time. Maybe something else went wrong.

Wait, nvm. I just tried it again and now it worked.

Good to know!
I think the current solutions are fine. I will try it first and probably will have some more questions/problems 🙂

I will create a minimal example.

Many thanks ReassuredTiger98 !

I will create a minimal example.

Don't know whether I do something wrong. Locally it works, but when executed via queue I get:

File "run_task.py", line 14, in <module> main() File "run_task.py", line 9, in main printme = importlib.import_module("some_package.file_to_import").printme File "/home/tim/.clearml/venvs-builds/3.7/lib/python3.7/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1006, in _gcd_import File "<frozen importlib._bootstrap>", line 983, in _find_and_load File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "<frozen importlib._bootstrap>", line 1006, in _gcd_import File "<frozen importlib._bootstrap>", line 983, in _find_and_load File "<frozen importlib._bootstrap>", line 965, in _find_and_load_unlocked ModuleNotFoundError: No module named 'some_package'

Nono, I got to thank you for this awesome tool!

ReassuredTiger98 oh wow I did not realize you actually call importlib to import your libraries (any reason not to call import ?)
And yes, I think we will miss it as the package analysis is actually static text analysts of the code

When experimenting we use a entrypoint script which we pass the specific experiment to.

The package is just subdir by the way. So it should not be in installed packages anyways, right?

The package is just subdir by the way. So it should not be in installed packages anyways, right?

Correct, also when the agent is spinning the code it will automatically add the root of the git repository to the pythonpath so you should be able to load the package.

ReassuredTiger98

It seems like clearml is not able to fetch the dependencies correctly when 

importlib

 is used.

If you have an example please let me know we'll try to fix it :)

Is it possible to read the dependencies manually from a conda environment.yml?

You can set detect_with_conda_freeze: true in clearml.conf, it will just use the entire conda env
https://github.com/allegroai/clearml/blob/28b85028fe4da3ab963b69e8ac0f7feef73cfcf6/docs/clearml.conf#L170

Here is the minimal example: https://github.com/mctigger/test_clearml

ReassuredTiger98 regrading the agent error, can you see the package some_packge in the "Installed Packages" in the UI? Was it installed ? are you using pip or conda as package manager in the agent (check the clearml.conf) is the agent running in docker mode ?

For example in our case we do reinforcement learning and the we would call a script like this: python run_openai_gym.py some_ http://package.my _agent .

Perfect! That sounds like a good solution for me.

Just like in the git repo.

ReassuredTiger98 I can verify the code snippet reproduces the issues with packages missing from "installed package".
If you feel this is important, please open a GitHub issue.
Also, you can manually add packages:
Task.add_requirements('package_name_here', 'optional version here')
So when you manually load the package you can make sure it will be listed, do remember to call it before the Task.init call.