Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Profile picture
HealthyStarfish45
Moderator
9 Questions, 44 Answers
  Active since 10 January 2023
  Last activity one year ago

Reputation

0

Badges 1

43 × Eureka!
0 Votes
15 Answers
1K Views
0 Votes 15 Answers 1K Views
I Have A Local Version Of Trains Server (Installed On A Vm In Gcp), One Problem I Run Into Was That Task.Init() Would Hang, And In The Ui I Could See "Drafts" Of Experiments. After Doing Docker-Compose Down And Up, Everything Went Back To Normal. Unfortun
I have a local version of trains server (installed on a VM in GCP), one problem I run into was that Task.init() would hang, and in the UI I could see "drafts...
4 years ago
0 Votes
30 Answers
1K Views
0 Votes 30 Answers 1K Views
What Is The Suggested Way Of Running Trains-Agent With Slurm? I Was Able To Do A Very Naive Setup: Trains-Agent Runs A Slurm Job. It Has The Disadvantage That This Slurm Job Is Blocking A Gpu Even If The Worker Is Not Running Any Task. Is There An Easy Wa
What is the suggested way of running trains-agent with SLURM? I was able to do a very naive setup: trains-agent runs a slurm job. It has the disadvantage tha...
4 years ago
0 Votes
2 Answers
947 Views
0 Votes 2 Answers 947 Views
I Have Installed The Trains Pycharm Plugin, But Based On The
I have installed the trains pycharm plugin, but based on the https://allegro.ai/docs/integrations/integration_pycharm/ I do not really know how to use it to ...
4 years ago
0 Votes
17 Answers
880 Views
0 Votes 17 Answers 880 Views
I Wanted To Ask About Html Reporting, If I Want To Do A More Fancy Visualization (Like Overlay Of Two Images Depending On Mouse Hovering), I Have To Inject This Html Into The Reporting Code, Right? I Am Asking, As Perhaps It Is Possible To Have Custom Wid
I wanted to ask about html reporting, if I want to do a more fancy visualization (like overlay of two images depending on mouse hovering), I have to inject t...
4 years ago
0 Votes
4 Answers
990 Views
0 Votes 4 Answers 990 Views
Afaiu By Default Trains Logs All Tensorboard Things, Can This Be Turned Off?
AFAIU by default trains logs all tensorboard things, can this be turned off?
4 years ago
0 Votes
11 Answers
996 Views
0 Votes 11 Answers 996 Views
If I Am Using The Demo Servers, Do I Need To Do Something Special To Use
If I am using the demo servers, do I need to do something special to use report_image ?
4 years ago
0 Votes
2 Answers
966 Views
0 Votes 2 Answers 966 Views
Are Trains Able To Discover The Docker Image And Docker Command That Is Used For Running The Current Experiment, Or Do I Need To Set It Up Manually By
Are trains able to discover the docker image and docker command that is used for running the current experiment, or do I need to set it up manually by task.s...
4 years ago
0 Votes
2 Answers
939 Views
0 Votes 2 Answers 939 Views
In The Experiment List Page I Can Add Custom Columns (Like Metrics) Which Is Great, But Can I Save The View To Switch Between The Different Views, Or At Least To Have A Fixed View Per Experiment?
In the experiment list page I can add custom columns (like metrics) which is great, but can I save the view to switch between the different views, or at leas...
4 years ago
0 Votes
8 Answers
967 Views
0 Votes 8 Answers 967 Views
Default Installation Of Trains-Server Exposes Only Logging Via Http Not Https, In Case The Server Has A Public Ip, What Is The Recommended Way For Increasing Security?
Default installation of trains-server exposes only logging via http not https, in case the server has a public IP, what is the recommended way for increasing...
4 years ago
0 In The Experiment List Page I Can Add Custom Columns (Like Metrics) Which Is Great, But Can I Save The View To Switch Between The Different Views, Or At Least To Have A Fixed View Per Experiment?

it does, thanks!

4 years ago
0 Default Installation Of Trains-Server Exposes Only Logging Via Http Not Https, In Case The Server Has A Public Ip, What Is The Recommended Way For Increasing Security?

got it, thanks!

4 years ago
0 I Wanted To Ask About Html Reporting, If I Want To Do A More Fancy Visualization (Like Overlay Of Two Images Depending On Mouse Hovering), I Have To Inject This Html Into The Reporting Code, Right? I Am Asking, As Perhaps It Is Possible To Have Custom Wid

For the images themselves, you can get heir urls

how can I do it?

4 years ago
0 I Wanted To Ask About Html Reporting, If I Want To Do A More Fancy Visualization (Like Overlay Of Two Images Depending On Mouse Hovering), I Have To Inject This Html Into The Reporting Code, Right? I Am Asking, As Perhaps It Is Possible To Have Custom Wid

Not sure yet, I will get back to you on this later, in 1-2 weeks, thanks.

4 years ago
0 I Wanted To Ask About Html Reporting, If I Want To Do A More Fancy Visualization (Like Overlay Of Two Images Depending On Mouse Hovering), I Have To Inject This Html Into The Reporting Code, Right? I Am Asking, As Perhaps It Is Possible To Have Custom Wid

have some kind of an add-on not as a widget but in an external system (this is not the preffered way of course)

4 years ago
0 I Wanted To Ask About Html Reporting, If I Want To Do A More Fancy Visualization (Like Overlay Of Two Images Depending On Mouse Hovering), I Have To Inject This Html Into The Reporting Code, Right? I Am Asking, As Perhaps It Is Possible To Have Custom Wid

Yes, replace image viewer with a custom widget, but perhaps we can implement this externally (I am not an expert on UI to be honest)

4 years ago
0 If I Am Using The Demo Servers, Do I Need To Do Something Special To Use

since I am using the demo server, I should make sure that in the configuration file the images will be uploaded to the appropriate server, right? Can you please point me to the proper line in the config file?

4 years ago
0 Afaiu By Default Trains Logs All Tensorboard Things, Can This Be Turned Off?

that was quick, thanks!

4 years ago
0 What Is The Suggested Way Of Running Trains-Agent With Slurm? I Was Able To Do A Very Naive Setup: Trains-Agent Runs A Slurm Job. It Has The Disadvantage That This Slurm Job Is Blocking A Gpu Even If The Worker Is Not Running Any Task. Is There An Easy Wa

I hope you can do this without containers.

4 years ago
0 What Is The Suggested Way Of Running Trains-Agent With Slurm? I Was Able To Do A Very Naive Setup: Trains-Agent Runs A Slurm Job. It Has The Disadvantage That This Slurm Job Is Blocking A Gpu Even If The Worker Is Not Running Any Task. Is There An Easy Wa

that's ok, I think that the race condition will be a non-issue. Thanks for checking!

4 years ago
0 What Is The Suggested Way Of Running Trains-Agent With Slurm? I Was Able To Do A Very Naive Setup: Trains-Agent Runs A Slurm Job. It Has The Disadvantage That This Slurm Job Is Blocking A Gpu Even If The Worker Is Not Running Any Task. Is There An Easy Wa

AgitatedDove14 thanks, that will be helpful!

4 years ago
0 What Is The Suggested Way Of Running Trains-Agent With Slurm? I Was Able To Do A Very Naive Setup: Trains-Agent Runs A Slurm Job. It Has The Disadvantage That This Slurm Job Is Blocking A Gpu Even If The Worker Is Not Running Any Task. Is There An Easy Wa

I implemented the first version and it seems to work.

4 years ago
0 What Is The Suggested Way Of Running Trains-Agent With Slurm? I Was Able To Do A Very Naive Setup: Trains-Agent Runs A Slurm Job. It Has The Disadvantage That This Slurm Job Is Blocking A Gpu Even If The Worker Is Not Running Any Task. Is There An Easy Wa

AgitatedDove14 going back to the slurm subject, I have local trains installed on the cluster with slurm so I am ready to test. At the same time I was thinking whether a simple solution would do the job:
a) [scale up agents] monitor the trains queue, if there is something that was not started for some amount of time, and the number of agents is not yet at the maximum, then add an agent,
b) [scale down agents] if all the tasks are running and there are idle agents, kill an idle agent.
Or do yo...

4 years ago
0 What Is The Suggested Way Of Running Trains-Agent With Slurm? I Was Able To Do A Very Naive Setup: Trains-Agent Runs A Slurm Job. It Has The Disadvantage That This Slurm Job Is Blocking A Gpu Even If The Worker Is Not Running Any Task. Is There An Easy Wa

yes, happy to help! In fact I am also interested in the k8s glue, since in one of our use cases we are using jobs and not pods (to allow for spot instances in the cloud), but I need to dig digger into the architecture to understand what we need exactly from k8s glue.

4 years ago
0 I Have A Local Version Of Trains Server (Installed On A Vm In Gcp), One Problem I Run Into Was That Task.Init() Would Hang, And In The Ui I Could See "Drafts" Of Experiments. After Doing Docker-Compose Down And Up, Everything Went Back To Normal. Unfortun

ok, thanks!

4 years ago
0 I Have A Local Version Of Trains Server (Installed On A Vm In Gcp), One Problem I Run Into Was That Task.Init() Would Hang, And In The Ui I Could See "Drafts" Of Experiments. After Doing Docker-Compose Down And Up, Everything Went Back To Normal. Unfortun

yes, but the local output was completely empty

4 years ago
0 I Have A Local Version Of Trains Server (Installed On A Vm In Gcp), One Problem I Run Into Was That Task.Init() Would Hang, And In The Ui I Could See "Drafts" Of Experiments. After Doing Docker-Compose Down And Up, Everything Went Back To Normal. Unfortun

and the experiment did not produce any logs, shall I enable some debug flag?

4 years ago
0 I Wanted To Ask About Html Reporting, If I Want To Do A More Fancy Visualization (Like Overlay Of Two Images Depending On Mouse Hovering), I Have To Inject This Html Into The Reporting Code, Right? I Am Asking, As Perhaps It Is Possible To Have Custom Wid

yes, this is what I found as well

4 years ago
0 What Is The Suggested Way Of Running Trains-Agent With Slurm? I Was Able To Do A Very Naive Setup: Trains-Agent Runs A Slurm Job. It Has The Disadvantage That This Slurm Job Is Blocking A Gpu Even If The Worker Is Not Running Any Task. Is There An Easy Wa

AgitatedDove14 I looked at the K8s glue code, having something similar but for SLURM would be great!

4 years ago
0 Are Trains Able To Discover The Docker Image And Docker Command That Is Used For Running The Current Experiment, Or Do I Need To Set It Up Manually By

thanks for explaining!

4 years ago
0 What Is The Suggested Way Of Running Trains-Agent With Slurm? I Was Able To Do A Very Naive Setup: Trains-Agent Runs A Slurm Job. It Has The Disadvantage That This Slurm Job Is Blocking A Gpu Even If The Worker Is Not Running Any Task. Is There An Easy Wa

AgitatedDove14 I do not want to push you in any way, but if you could give me an estimate of the slurm glue code, that would be helpful. I should have a local installation of the trains server to experiment with next week.

4 years ago
0 What Is The Suggested Way Of Running Trains-Agent With Slurm? I Was Able To Do A Very Naive Setup: Trains-Agent Runs A Slurm Job. It Has The Disadvantage That This Slurm Job Is Blocking A Gpu Even If The Worker Is Not Running Any Task. Is There An Easy Wa

I will only cancel daemons which are idle.

4 years ago
0 I Have A Local Version Of Trains Server (Installed On A Vm In Gcp), One Problem I Run Into Was That Task.Init() Would Hang, And In The Ui I Could See "Drafts" Of Experiments. After Doing Docker-Compose Down And Up, Everything Went Back To Normal. Unfortun

apiserver logs were clean, only 200s there

4 years ago
0 I Wanted To Ask About Html Reporting, If I Want To Do A More Fancy Visualization (Like Overlay Of Two Images Depending On Mouse Hovering), I Have To Inject This Html Into The Reporting Code, Right? I Am Asking, As Perhaps It Is Possible To Have Custom Wid

yes

4 years ago
0 What Is The Suggested Way Of Running Trains-Agent With Slurm? I Was Able To Do A Very Naive Setup: Trains-Agent Runs A Slurm Job. It Has The Disadvantage That This Slurm Job Is Blocking A Gpu Even If The Worker Is Not Running Any Task. Is There An Easy Wa

AgitatedDove14 I need to finish working now, will be back in the evening and have a look.

4 years ago
0 What Is The Suggested Way Of Running Trains-Agent With Slurm? I Was Able To Do A Very Naive Setup: Trains-Agent Runs A Slurm Job. It Has The Disadvantage That This Slurm Job Is Blocking A Gpu Even If The Worker Is Not Running Any Task. Is There An Easy Wa

AgitatedDove14 Is there a way to say to a worker that it should not take new tasks? If there is such a feature then one could avoid the race condition.

4 years ago
0 I Have Installed The Trains Pycharm Plugin, But Based On The

ok, got it, thanks!

4 years ago
0 I Have A Local Version Of Trains Server (Installed On A Vm In Gcp), One Problem I Run Into Was That Task.Init() Would Hang, And In The Ui I Could See "Drafts" Of Experiments. After Doing Docker-Compose Down And Up, Everything Went Back To Normal. Unfortun

thanks, next time I will provide you will all the logs

4 years ago
0 What Is The Suggested Way Of Running Trains-Agent With Slurm? I Was Able To Do A Very Naive Setup: Trains-Agent Runs A Slurm Job. It Has The Disadvantage That This Slurm Job Is Blocking A Gpu Even If The Worker Is Not Running Any Task. Is There An Easy Wa

Unfortunately there is no docker, there is only singularity. This cluster is used by many users and docker is not secure enough.

4 years ago
0 I Have A Local Version Of Trains Server (Installed On A Vm In Gcp), One Problem I Run Into Was That Task.Init() Would Hang, And In The Ui I Could See "Drafts" Of Experiments. After Doing Docker-Compose Down And Up, Everything Went Back To Normal. Unfortun

SuccessfulKoala55 20 minutes at least

4 years ago
Show more results compactanswers