MelancholyBeetle72

3 Questions, 12 Answers

Active since 10 January 2023

Last activity one year ago

Reputation

Badges 1

11 × Eureka!

Questions 3
Answers 12

0 Votes

11 Answers

1K Views

0 Votes 11 Answers 1K Views

Continuing On

Continuing on https://allegroai-trains.slack.com/archives/CTK20V944/p1607012505242500 we'd like to minimize startup time for the agent-started experiments si...

mlops

3 years ago

0 Votes

4 Answers

1K Views

0 Votes 4 Answers 1K Views

Feature Request: Group Series In The Plots Section Like In The Scalars Section. I'D Like To Group Pr Curves From Different Iterations. That'S It

Feature request: group series in the Plots section like in the Scalars section. I'd like to group PR curves from different iterations. That's it 🙂

clearml

3 years ago

0 Votes

8 Answers

1K Views

0 Votes 8 Answers 1K Views

Hi All. I Am Using The Recently Added Trainslogger In Pytorch-Lightning And Experiencing Incoherent Behavior With Model Checkpoint Upload. I Made An Issue On Pytorch-Lightning Github

Hi all. I am using the recently added TrainsLogger in Pytorch-Lightning and experiencing incoherent behavior with model checkpoint upload. I made an issue on...

clearml

4 years ago

0 Omg Look Who Just Joined The Pytorch Ecosystem

Finally the recognition Trains deserves!

4 years ago

0 Continuing On

This is as far as I could get.

3 years ago

0 Continuing On

Thanks AgitatedDove14 !
Is there a way to programmatically set the base docker image and extra docker arguments for enqueued tasks? I'm afraid I have no access to trains.conf , and manually editing enqueued experiments in the web UI is not an option.

3 years ago

0 Continuing On

Am I right that docker_cmd should be like "docker run --mount <...> image" ?

3 years ago

0 Continuing On

ah, Task.set_base_docker(docker_cmd) , I reckon

3 years ago

0 Continuing On

Yes, exactly

3 years ago

0 Continuing On

Maybe I could use a pre-built docker image with a mounted volume instead?

3 years ago

0 Hi All. I Am Using The Recently Added Trainslogger In Pytorch-Lightning And Experiencing Incoherent Behavior With Model Checkpoint Upload. I Made An Issue On Pytorch-Lightning Github

Issue: https://github.com/allegroai/trains/issues/123

4 years ago

0 Hi All. I Am Using The Recently Added Trainslogger In Pytorch-Lightning And Experiencing Incoherent Behavior With Model Checkpoint Upload. I Made An Issue On Pytorch-Lightning Github

is there a way for pytorch lightning not to rename the model?

I think no, since pytorch-lightning ensures atomarity of save operation by this logic.

I think the best solution is for Trains to create a copy of the file and upload it in the background

Yes, sounds good.
Yeah, I'll open an issue on Trains Github

4 years ago

0 Hi All. I Am Using The Recently Added Trainslogger In Pytorch-Lightning And Experiencing Incoherent Behavior With Model Checkpoint Upload. I Made An Issue On Pytorch-Lightning Github

Great, thanks 🙂

4 years ago

0 Hi All. I Am Using The Recently Added Trainslogger In Pytorch-Lightning And Experiencing Incoherent Behavior With Model Checkpoint Upload. I Made An Issue On Pytorch-Lightning Github

Wow, astonishingly fast 🙂
Ok, I'll check my case and leave feedback

4 years ago

0 Continuing On

If I follow the pre-built docker image option, what are the correct configurations?
Also, can the image not be pulled from dockerhub but used from the local build instead?

3 years ago