correct, you can pass it as keys on the "task_filter" argument, e.g:Task.get_tasks(..., task_filter={'status': ['failed']})
would those containers best be started from something in services mode?
Yes as long as the machine has enough cpu/ram
Notice that the services mode will start a second parallel Task after the first one is done setting up the env, if running with CLEARML_AGENT_SKIP_PYTHON_ENV_INSTALL, with containers that have git/python/clearml-agent preinstalled it should be minimal.
or is it possible to get no-overhead with my approach of worker-inside-docker?
No do not do that, see above e...
AdventurousRabbit79 you mean like minio / ceph ?
BTW: see if this works:$ CLEARML_API_HOST_VERIFY_CERT=0 clearml-init
Yep 🙂 but only in RC (or github)
basically the default_output_uri will cause all models to be uploaded to this server (with specific subfolder per project/task)
You can have the same value there as the files_server.
The files_server is where you have all your artifacts / debug samples
Yes, I mean trains-agent. Actually I am using 0.15.2rc0. But, I am using local files, I mean I clone trains and trains-agent repos and install them. Their versions are 0.15.2rc0
I see, that's why we get the git ref, not package version.
What are you seeing in the Task that was cloned (i.e. the one the HPO created not the original training task)?
by that I mean, configuration section, do you have the Args there ? (seems like the pic you attached, but I just want to make sure)
Also in the train.py file, do you also have Task.init ?
Like get the tasks that uses the most metrics API?
SubstantialElk6 feel free to tweet them on their very inaccurate comparison table 🙂
ngrok to connect to the remote server at the office?
That makes sense, I guess this is the equivalent of using a VPN, from that point onward clearml-session can directly access the remote machine, right?
SoggyBeetle95 you can configure the credentials in the clearml.conf running on the agent machines:
https://github.com/allegroai/clearml-agent/blob/a5a797ec5e5e3e90b115213c0411a516cab60e83/docs/clearml.conf#L320
(I'm assuming these are storage credentials)
If you need general purpose env variables, you can ad them here:
https://github.com/allegroai/clearml-agent/blob/a5a797ec5e5e3e90b115213c0411a516cab60e83/docs/clearml.conf#L149
with ["-e", "MY_VAR=MY_VALUE"]
Oh then this should just workcp -R --link b a/You can achieve the same symbol link link from python as well
GrievingTurkey78 where do you see this message? Can you send the full server log
?
I think for it to work you have to have ssh running on the host machine (the socket client itself), no?
 is the "installed packages" part editable? good to know
Of course it is, when you clone a Task everything is Editable 🙂
Isn't it a bit risky manually changing a package version?
worst case it will crash quickly, and you reset/edit/enqueue 🙂
(Should work though)
Hi TrickyRaccoon92
Yes please update me once you can, I would love to be able to reproduce the issue so we could fix for the next RC 🙂
Hi JitteryCoyote63 ,
These properties are usually not available on the UI and are used internal, hence the lack of documentation. Regrading parent property, it will hold a parent Task.id (str) , that said it has no real effect on the Task itself. You can however search for Tasks with a specific parent ID (For examples, this is how the the hyper parameter class is using this property)
compression=ZIP_DEFLATED if compression is None else compressionwdyt?
Sure, ReassuredTiger98 just add them after the docker image in the "Base Docker image" section under the execution Tab. The same applies for setting it from code.
example:nvcr.io/nvidia/tensorflow:20.11-tf2-py3 -v /mnt/data:/mnt/dataYou can also always force extra docker run arguments by changing the clearml.conf on the agent itself:
https://github.com/allegroai/clearml-agent/blob/822984301889327ae1a703ffdc56470ad006a951/docs/clearml.conf#L121
Hi @<1618056041293942784:profile|GaudySnake67>Task.create is designed to create an External task not from the current running process.Task.init is for creating a Task from your current code, and this is why you have all the auto_connect parameters. Does that make sense ?
Let me check something
AstonishingSeaturtle47 I think there's a workaround for the GitHub multiple repo issue. See https://gist.github.com/gubatron/d96594d982c5043be6d4
Container environment setup overhead?
Fixing that would make this feature great.
Hmm, I guess that is doable, this is a good point, search for the GUID is not always trivial (or maybe at least we can put in the description the project/dataset/version )