BitterRabbit31

3 Questions, 7 Answers

Active since 10 January 2023

Last activity one year ago

Reputation

Badges 1

7 × Eureka!

Questions 3
Answers 7

0 Votes

1 Answers

1K Views

0 Votes 1 Answers 1K Views

Ok, I Faced Quite Funny Issue. Sorry For Spamming In This Chat, But I Am Just Ramping Up With Clearml And Its A Bit Turbulent.. Issue (As I Understand It) Is Following: My Package That I Use For Model Trainings Has The Same Name As Some Package In Pip (I

ok, I faced quite funny issue. sorry for spamming in this chat, but i am just ramping up with ClearML and its a bit turbulent.. Issue (as I understand it) is...

clearml

2 years ago

0 Votes

10 Answers

1K Views

0 Votes 10 Answers 1K Views

Hey. Is It Possible To Keep Docker Container Once Job Finished/Failed? I See In Logs That "--Rm" Is Passed To "Docker Run". Is It Possible To Avoid It?

hey. is it possible to keep docker container once job finished/failed? I see in logs that "--rm" is passed to "docker run". Is it possible to avoid it?

clearml

2 years ago

0 Votes

4 Answers

1K Views

0 Votes 4 Answers 1K Views

Hey. I Want To Run Task Remotely, But Neither The Clearml Server Nor Worker Can Clone The Repository With Training Code (They Dont Have Access). Can I Somehow Tell The Task That It Has To Use Local Copy Of The Package? I.E.:

hey. I want to run task remotely, but neither the clearml server nor worker can clone the repository with training code (they dont have access). can i someho...

clearml

2 years ago

0 Hey. Is It Possible To Keep Docker Container Once Job Finished/Failed? I See In Logs That "--Rm" Is Passed To "Docker Run". Is It Possible To Avoid It?

but who exactly executes agent in this case? does it happen on my machine, server or worker?

2 years ago

0 Hey. Is It Possible To Keep Docker Container Once Job Finished/Failed? I See In Logs That "--Rm" Is Passed To "Docker Run". Is It Possible To Avoid It?

if want to debug why it failed if it failed due to tiny issue - maybe to fix it locally and restart the training command from within docker, not to wait for docker to get assembled again

2 years ago

0 Hey. Is It Possible To Keep Docker Container Once Job Finished/Failed? I See In Logs That "--Rm" Is Passed To "Docker Run". Is It Possible To Avoid It?

ah, stupid me. I was looking everywhere but not in clearml-agent 🙂 thanks!

2 years ago

0 Hey. Is It Possible To Keep Docker Container Once Job Finished/Failed? I See In Logs That "--Rm" Is Passed To "Docker Run". Is It Possible To Avoid It?

yeah. I am getting logs, but they are extremely puzzling to me. I would appreciate to actually have access to whole package structure.. indeed. can you maybe point where the docker command is composed. looking for it for past 30 mins or so. not so familiar with internals really 😕

2 years ago

0 Hey. I Want To Run Task Remotely, But Neither The Clearml Server Nor Worker Can Clone The Repository With Training Code (They Dont Have Access). Can I Somehow Tell The Task That It Has To Use Local Copy Of The Package? I.E.:

was rather on infrastructure level - worker doesnt have access to the repository. I found workaround of using mirror repo that is accessible by worker..

2 years ago

0 Hey. Is It Possible To Keep Docker Container Once Job Finished/Failed? I See In Logs That "--Rm" Is Passed To "Docker Run". Is It Possible To Avoid It?

nono. that one is clear. i am about general workflow...

but from diagrams it looks like its worker who runs that worker.py you pointed above.

2 years ago

okay, after digging a little, I think the snippet above is not valid. "standalone script" - assumes just a single python file. I still need repo where training scripts live. It just cant be cloned on remote worker. Is it possible to somehow pack local repo to be executed on remote worker?

2 years ago