thanks,
I've opened an issue
https://github.com/allegroai/clearml/issues/430

great!
Is there a way to add this for an existing task's draft via the web UI?

Thanks AgitatedDove14 . I'll try that

that was my next question 🙂
How does this design work with a stateful search algorithm?

the hack doesn't work if conda is not installed 😞

Can you elaborate on what you would do with it? Like an OS environment disable the entire setup itself ? will it clone the code base ?

It will not do any setup steps. Ideally it would just pull an experiment from a dedicated HPO queue and run it inplace

It's a very convenient way of doing a parameter sweep on with minimal setup effort

hows does this work with HPO?
the tasks are generated in advance?

Regardless, it would be very convenient to add a flag to the agent which point it to an existing virtual environment and bypassing the entire setup process. This would facilitate ramping up new users to clearml who don't want the bells and whistles and would just a simple HPO from an existing env (which may not even exist as part of a git repo)

You mean running everything on a single machine (manually)?

Yes, but not limited to this.
I want to be able to install the venv in multiple servers and start the "simple" agents in each one on them. You can think of it as some kind of one-off agent for a specific (distributed) hyperparameter search task

AgitatedDove14 , I'm running an agent inside a docker (using the image on dockerhub) and mounted the docker socket to the host so the agent can start sibling containers. How do I set the config for this agent? Some options can be set through env vars but not all of them 😞

cudnn isn't cuda, it's a separate library.
are you running on docker on bare metal? you should have cuda installed at /usr/local/cuda-<>

note that the cuda driver was only recently added to nvidia-smi

just seems a bit cleaner and more DevOps/k8s friendly to work with the container version of the agent 🙂

I'm trying to achieve a workflow similar to the one in wandb for parameter sweep where there are no venvs involved other than the one created by the user 😅

Of course conda needs to be installed, it is using a pre-existing conda env, no?! what am I missing

its not a conda env, just a regular venv (poetry in this specific case)

And the assumption is the code is also there ?

yes. The user is responsible for the entire setup. the agent just executes python <path to script> <current hpo args>

An easier fix for now will probably be some kind of warning to the user that a task is created but not connected

got it

Ok working now 🙂 had a mistake in the folder name

lol great hack. I'll check it out.
Although I'd be really happy if there was a solution in which I can just spawn an ad-hoc worker 🙂

try:
sudo updatedb locate libcudart

just docker-compose up with the latest compose file from the repo

just to be clear, multiple CUDA runtime version can coexist on a single machine, and the only thing that points to which one you are using when running an application are the library search paths (which can be set either with LD_LIBRARY_PATH , or, preferably, by creating a file under /etc/ld.so.conf.d/ which contains the path to your cuda directory and executing ldconfig )

thanks!

oops 🙂

as a workaround I just stick the epoch number in the series argument of report_scatter2d , with the same title name

this is the cuda driver api. you need libcudart.so

still the same 😞

so you dont have cuda installed 🙂

JitteryCoyote63 I still don't understand what is the actual CUDA version you are using on your machine