CrookedMonkey33 , let me take a look if I can find an AMI 🙂
Hi BroadSeaturtle49 , what versions of clearml-agent
& clearml
are you using? What OS is this?
The ec2 machines will be spun in your account. Any image accessible by those machines will be usable. You can add a bash script to run on the ec2 instances on startup and login. The agent on the ec2 instance will basically run a 'docker run' command.
Are you running on a self hosted ClearML server or the SaaS?
Hi EcstaticBaldeagle77 ,
I'm not sure I follow. Are you using the self hosted server - and you'd like to move data from one self hosted server to another?
This will disable storing the uncommitted changes
Hi RoundMole15 , what version of clearml
are you using? Also how is the model being saved without ClearML?
Hi @<1531807732334596096:profile|ObliviousClams17> , I think for your specific use case it would be easiest to use the API - fetch a task, clone it as many times as needed and enqueue it into the relevant queues.
Fetch a task - None
Clone a task - None
Enqueue a task (or many) - [None](https://clear.ml/docs/latest/docs/references/api/ta...
Hi @<1572395190897872896:profile|ShortWhale75> , that is not the correct way to use workers & queues.
First of all, Task.init
will mark your task as running so this error makes sense.
The idea is first you run the code locally on your machine, once everything is logged (packages, repo, uncomitted changes & configurations) you can clone the task and then enqueue it into the agent.
Programmatically, you would watch to fetch an existing task in the system, clone it and then enqueue the n...
You can disable automatic model logging using auto_connect_frameworks
in Task.init()
https://clear.ml/docs/latest/docs/references/sdk/task#taskinit
This however will also disable automatic reporting of scalers. You can also manually force the upload of the final model with
https://clear.ml/docs/latest/docs/references/sdk/model_outputmodel#class-outputmodel
Hi DepressedFish57 ,
But authentificate by log/pass is disable on host side.
Can you please clarify?
By separate ssh key - do you mean a different git password?
Can you see if in the APIserver logs something happened during this time? Is the agent still reporting?
Hi @<1564060263047499776:profile|ThoughtfulCentipede62> , I think the issue is still open. Can you please open a GitHub issue to track it so we can make sure it is resolved?
Hi @<1556812486840160256:profile|SuccessfulRaven86> , can you please add an example configuration that reproduces this?
Are you sure you pasted the credentials correctly? Does it give you feedback on which key/secret you used during the process? Which version of ClearML-Agent are you on?
FreshKangaroo33 ,
On the top right of the experiments view you have a cog wheel, if you click on it, it will give you an option to add hyper parameters to the table. I think from the API calls from there you can figure something out 🙂
FreshKangaroo33 , what do you mean by syntax examples?
I think this should give you some context on usage 🙂
https://github.com/allegroai/clearml/blob/master/examples/reporting/hyper_parameters.py
Hi @<1648134232087728128:profile|AlertFrog99> , I don't think there is an automatic way to do this out of the box but I guess you could write some automation that does that via the API
Hi @<1562610703553007616:profile|CloudyCat50> , you can use Task.set_tags()
to 're-set' tags and omit the tag you want removed.
What is being reported that isn't auto-logged?
Hi @<1578555761724755968:profile|GrievingKoala83> , you can do it directly through the UI by adding parameters, see examples here - None
Hi AdventurousButterfly15 ,
When running code locally, how are the installed packages detected? Does it detect your entire venv or does it detect only the packages that were used?
Hi AttractiveShrimp45 , hyperparameter optimization is available in open source https://github.com/allegroai/clearml/tree/master/examples/optimization/hyper-parameter-optimization .
It is indeed lacking a GUI and does require some more tinkering to tune it to your needs.
https://clear.ml/docs/latest/docs/guides/optimization/hyper-parameter-optimization/examples_hyperparam_opt are the relevant docs 🙂
You can specify any extra requirements in the steps themselves
The agent prints its configuration before the execution step, I don't see agent.git_pass
set anywhere in the log. Are you sure you set it up on the correct machine? This needs to be set up on the machine running the agent.
Hi @<1526371965655322624:profile|NuttyCamel41> , can you add the full log?
They need to switch to your workspace, create credentials on your workspace and then use them instead of their own. Makes sense?
Also, in the original experiment, what pytorch version is detected?