SweetBadger76 thanks,

The only thing I am not certain about is. What does agent means in ClearML world? Is the queue manager or the pipelien?

thank you, guys! I think now it works! Amazing step-by-step support! This is sublime!

for future ref., this is a summary of what I have done:

create a project on clearml webUI create a queue on clearml webUI run an agent /homes/yosefhaie/.conda/envs/devops/bin/clearml-agent daemon --create-queue --queue <queue-name> use this test script:` from clearml import PipelineController

We will use the following function an independent pipeline component step

notice all package imports ins...

so it's not auto-generated. What's the spec of this conf file?

I see! SuccessfulKoala55 what is the right way to config it? via vim or is there any commandline tool?

so it looks for requirement.txt?

PIP Mode

By default, ClearML Agent works in PIP Mode, in which it uses

as the package manager. When ClearML runs, it will create a virtual environment (or reuse an existing one, see

). Task dependencies (Python packages) will be installed in the virtual environment.

how does it know what are the dependencies of a task?

if this was possible we wouldn't need pip

I think I am missing something

would love to hear your thoughts

there was an issue with the layout of my simple git example. So if you do the above it should work

solved with Alon's help 1x1

ran clearml-agent daemon --queue rudolf --detached

how this is possible?

This is extremely helpful! I decided to go with pipeline from functions.

Everything looks great, but the tasks are pending. Am I missing some executor or something like that?

locally and remotely

or how can I add some libraries I'd like it to pip install in the new environment

we have a diff name for this file in the repo

now the question how to make the production stuff to work

from clearml import PipelineController

nice! Thank you

TimelyPenguin76

thanks again... I have tried what you have suggested. It does inject the dir, but not cloning the whole content.

os.system("tree") from within demo.py pipeline step function:

└── test-demo-clearml.git ├── demo.py ├── __init__.py ├── local_dir └── step_one_task.pyvs.
from the actual tree (under ~/clearml_using_local_lib )
` tree
.
├── demo.py
├── init.py
└── local_dir
├── hello.py
├── init.py
└── pycache
└── __i...

Yep Correct

I created a worker that listens to the queue, but the worker doesn't pull the tasks (they are pending)

it's an undecidable problem

Task.add_requirements

the question how does ClearML know to create env and what files does it copy to the task
/homes/yossi/.clearml/venvs-builds/3.7/

the only way any computer could figure this out is by running it

based on what reqeuirment.txt manifest?

if a==True: import torch else: import tensorflow

consider this case: