UnevenDolphin73 , can you verify that the process is not running on the machine? for example with htop
or top
You write code for a new pipeline π
Hi @<1603560525352931328:profile|BeefyOwl35> , The agent uses it's own entry point, so yes you do need to specify it even if it's in the dockerfile π
Hi @<1523702307240284160:profile|TeenyBeetle18> , if they are already on GS then you can use add_external_files to log them.
None
What do you think?
Hmm my bad, I wasn't aware of this π
Can you add a print out to verify that you selected the correct project?
Hi @<1691258549901987840:profile|PoisedDove36> , did you do all the db migrations during the upgrade or did you go straight to 1.5 form 1.0?
What OS are you using?
you're always running a single task at a time. The whole point is that everything is reported to the task (auto-magic bindings, console logs etc.), so there cannot be any ambiguity. You can close the current task ( task.close()
) and init a new one if you'd like, but you can't init several at the same time.
Hi @<1639799308809146368:profile|TritePigeon86> , you mean that in order to initialize machines in ec2 you need to provide some external ip or you need to pass the external ip as a parameter in order for the job to run?
Hi, can you add a log of the run? Also what version of ClearML Agent are you using
Hi @<1523704674534821888:profile|SourLion48> , making sure I understand - You push a job into a queue that an autoscaler is listening to. A machine is spun up by the autoscaler and takes the job and it runs. Afterwards during the idle time, you push another job to the same queue, it is picked up by the machine that was spun up by the autoscaler and that one will fail?
Hi @<1523701181375844352:profile|ExasperatedCrocodile76> , and now the worker clones the repo correctly?
@<1523701137134325760:profile|CharmingStarfish14> ,interesting, so what are you suggesting? Creating Jira tasks from special tags on ClearML?
Hi @<1546303277010784256:profile|LivelyBadger26> , how did you set the random seed? I think you can also disable ClearML's random seed override and set one with Pytorch
Are you using a self deployed server?
Hi @<1541954607595393024:profile|BattyCrocodile47> , how does ClearML react when you run the scripts this way? The repository is logged as usual?
What do you mean read params file?
When in table view (rows) there is a small icon next to the 'Started' column. There you can configure time periods you'd like to view π
Hi @<1679661969365274624:profile|UnevenSquirrel80> , are you running a self hosted server?
Hi @<1676762887223250944:profile|FancyKangaroo34> , it would be possible for example if the docker image has a different python version from what you ran on previously
Do you have a screenshot of your settings?
Hi @<1636537816684957696:profile|CooperativeGoat65> , you can change the api.files_server
section of the configuration file to point to your s3 bucket
What exactly are you looking to set up?
Regarding project move - Do you move it between subprojects within a project and after F5 you see the experiment again?
and just making sure - by pipeline we're talking about the ClearML pipelines, correct?
https://clear.ml/docs/latest/docs/references/sdk/automation_controller_pipelinecontroller
SoggyBeetle95 , in the ClearML UI you should see a small notification at the top when there is a new version available