Reputation
Badges 1
45 × Eureka!its working now, thanks that was the problem.
yeah i see it now in the requirements of the task, that's weird, I'll create a new environment and check it again, thanks
we had a few experiments that were stuck for a few hours until we noticed that and we also had 1 that was stuck for 2 days (on the weekend). and they weren't auto aborted.
Hi @<1523701205467926528:profile|AgitatedDove14> ,
That solved my problem thank you, from my deep dive I've found the problem there was a package called install
that changed it's name to pip-install
and its requirement was setuptools
.
Thanks again for the help.
And I'm looking at None as an example of a clearml.conf file and i can't seem to find sdk.development.worker.console_cr_flush_period
this flag.
Hi @<1523701070390366208:profile|CostlyOstrich36> ,
but how do I configure this if I'm not hosting the clearml server?
where can i find the services.conf file?
Yes it does, thank you @<1523701070390366208:profile|CostlyOstrich36>
@<1523701087100473344:profile|SuccessfulKoala55> in the file example here there is no reference to console_cr_flush_period
@<1523701087100473344:profile|SuccessfulKoala55> yes the working dir is set to the correct path and yet it cannot import the train module
It’s running a agent without docker, we aren’t using docker
@<1523701087100473344:profile|SuccessfulKoala55> and @<1523701070390366208:profile|CostlyOstrich36> Ok so I found the problem but its weird,
when the agent is setting up the enviorment its installing torch=1.11.0 and not installing the one in the requirements which is torch=1.11.0+cu113,
I've checked the clearml.conf and i do have this flag set:
force_repo_requirements_txt: true
and I have a local whl of torch=1.11.0+cu113 with a path set to its location in the requirements.txt ...
Yes, same one
@<1523701087100473344:profile|SuccessfulKoala55> But when i use this setting it the packages download only from the torch repo and not a local repo correct? or does it use the url-extra-link? and is there a way to cancel the auto cuda detect?