Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Profile picture
HealthyStarfish45
Moderator
9 Questions, 44 Answers
  Active since 10 January 2023
  Last activity 2 years ago

Reputation

0

Badges 1

43 × Eureka!
0 Votes
2 Answers
2K Views
0 Votes 2 Answers 2K Views
I Have Installed The Trains Pycharm Plugin, But Based On The
I have installed the trains pycharm plugin, but based on the https://allegro.ai/docs/integrations/integration_pycharm/ I do not really know how to use it to ...
5 years ago
0 Votes
30 Answers
2K Views
0 Votes 30 Answers 2K Views
What Is The Suggested Way Of Running Trains-Agent With Slurm? I Was Able To Do A Very Naive Setup: Trains-Agent Runs A Slurm Job. It Has The Disadvantage That This Slurm Job Is Blocking A Gpu Even If The Worker Is Not Running Any Task. Is There An Easy Wa
What is the suggested way of running trains-agent with SLURM? I was able to do a very naive setup: trains-agent runs a slurm job. It has the disadvantage tha...
5 years ago
0 Votes
11 Answers
2K Views
0 Votes 11 Answers 2K Views
If I Am Using The Demo Servers, Do I Need To Do Something Special To Use
If I am using the demo servers, do I need to do something special to use report_image ?
5 years ago
0 Votes
4 Answers
2K Views
0 Votes 4 Answers 2K Views
Afaiu By Default Trains Logs All Tensorboard Things, Can This Be Turned Off?
AFAIU by default trains logs all tensorboard things, can this be turned off?
5 years ago
0 Votes
8 Answers
2K Views
0 Votes 8 Answers 2K Views
Default Installation Of Trains-Server Exposes Only Logging Via Http Not Https, In Case The Server Has A Public Ip, What Is The Recommended Way For Increasing Security?
Default installation of trains-server exposes only logging via http not https, in case the server has a public IP, what is the recommended way for increasing...
5 years ago
0 Votes
15 Answers
2K Views
0 Votes 15 Answers 2K Views
I Have A Local Version Of Trains Server (Installed On A Vm In Gcp), One Problem I Run Into Was That Task.Init() Would Hang, And In The Ui I Could See "Drafts" Of Experiments. After Doing Docker-Compose Down And Up, Everything Went Back To Normal. Unfortun
I have a local version of trains server (installed on a VM in GCP), one problem I run into was that Task.init() would hang, and in the UI I could see "drafts...
5 years ago
0 Votes
17 Answers
2K Views
0 Votes 17 Answers 2K Views
I Wanted To Ask About Html Reporting, If I Want To Do A More Fancy Visualization (Like Overlay Of Two Images Depending On Mouse Hovering), I Have To Inject This Html Into The Reporting Code, Right? I Am Asking, As Perhaps It Is Possible To Have Custom Wid
I wanted to ask about html reporting, if I want to do a more fancy visualization (like overlay of two images depending on mouse hovering), I have to inject t...
5 years ago
0 Votes
2 Answers
2K Views
0 Votes 2 Answers 2K Views
In The Experiment List Page I Can Add Custom Columns (Like Metrics) Which Is Great, But Can I Save The View To Switch Between The Different Views, Or At Least To Have A Fixed View Per Experiment?
In the experiment list page I can add custom columns (like metrics) which is great, but can I save the view to switch between the different views, or at leas...
5 years ago
0 Votes
2 Answers
2K Views
0 Votes 2 Answers 2K Views
Are Trains Able To Discover The Docker Image And Docker Command That Is Used For Running The Current Experiment, Or Do I Need To Set It Up Manually By
Are trains able to discover the docker image and docker command that is used for running the current experiment, or do I need to set it up manually by task.s...
5 years ago
0 What Is The Suggested Way Of Running Trains-Agent With Slurm? I Was Able To Do A Very Naive Setup: Trains-Agent Runs A Slurm Job. It Has The Disadvantage That This Slurm Job Is Blocking A Gpu Even If The Worker Is Not Running Any Task. Is There An Easy Wa

AgitatedDove14 thanks, that will be helpful!

5 years ago
0 What Is The Suggested Way Of Running Trains-Agent With Slurm? I Was Able To Do A Very Naive Setup: Trains-Agent Runs A Slurm Job. It Has The Disadvantage That This Slurm Job Is Blocking A Gpu Even If The Worker Is Not Running Any Task. Is There An Easy Wa

I implemented the first version and it seems to work.

5 years ago
0 What Is The Suggested Way Of Running Trains-Agent With Slurm? I Was Able To Do A Very Naive Setup: Trains-Agent Runs A Slurm Job. It Has The Disadvantage That This Slurm Job Is Blocking A Gpu Even If The Worker Is Not Running Any Task. Is There An Easy Wa

AgitatedDove14 I need to finish working now, will be back in the evening and have a look.

5 years ago
0 What Is The Suggested Way Of Running Trains-Agent With Slurm? I Was Able To Do A Very Naive Setup: Trains-Agent Runs A Slurm Job. It Has The Disadvantage That This Slurm Job Is Blocking A Gpu Even If The Worker Is Not Running Any Task. Is There An Easy Wa

But I can't do this from the web ui, can I?

5 years ago
0 What Is The Suggested Way Of Running Trains-Agent With Slurm? I Was Able To Do A Very Naive Setup: Trains-Agent Runs A Slurm Job. It Has The Disadvantage That This Slurm Job Is Blocking A Gpu Even If The Worker Is Not Running Any Task. Is There An Easy Wa

so far everything works, the only problem I can think of is a race condition, but I will probably ignore it, which happens in the following scenario:
a) a worker finishes its current run, turns into an idle state,
b) my script scrapes the status of the worker, which is idle,
c) a new task is enqueued and picked by the worker,
d) the worker is killed after it managed to pull a task from the queue, so the task will be cancelled as well.

5 years ago
0 What Is The Suggested Way Of Running Trains-Agent With Slurm? I Was Able To Do A Very Naive Setup: Trains-Agent Runs A Slurm Job. It Has The Disadvantage That This Slurm Job Is Blocking A Gpu Even If The Worker Is Not Running Any Task. Is There An Easy Wa

that's ok, I think that the race condition will be a non-issue. Thanks for checking!

5 years ago
0 What Is The Suggested Way Of Running Trains-Agent With Slurm? I Was Able To Do A Very Naive Setup: Trains-Agent Runs A Slurm Job. It Has The Disadvantage That This Slurm Job Is Blocking A Gpu Even If The Worker Is Not Running Any Task. Is There An Easy Wa

AgitatedDove14 I do not want to push you in any way, but if you could give me an estimate of the slurm glue code, that would be helpful. I should have a local installation of the trains server to experiment with next week.

5 years ago
0 What Is The Suggested Way Of Running Trains-Agent With Slurm? I Was Able To Do A Very Naive Setup: Trains-Agent Runs A Slurm Job. It Has The Disadvantage That This Slurm Job Is Blocking A Gpu Even If The Worker Is Not Running Any Task. Is There An Easy Wa

AgitatedDove14 I looked at the K8s glue code, having something similar but for SLURM would be great!

5 years ago
0 I Have A Local Version Of Trains Server (Installed On A Vm In Gcp), One Problem I Run Into Was That Task.Init() Would Hang, And In The Ui I Could See "Drafts" Of Experiments. After Doing Docker-Compose Down And Up, Everything Went Back To Normal. Unfortun

thanks, next time I will provide you will all the logs

5 years ago
0 What Is The Suggested Way Of Running Trains-Agent With Slurm? I Was Able To Do A Very Naive Setup: Trains-Agent Runs A Slurm Job. It Has The Disadvantage That This Slurm Job Is Blocking A Gpu Even If The Worker Is Not Running Any Task. Is There An Easy Wa

I will only cancel daemons which are idle.

5 years ago
0 I Wanted To Ask About Html Reporting, If I Want To Do A More Fancy Visualization (Like Overlay Of Two Images Depending On Mouse Hovering), I Have To Inject This Html Into The Reporting Code, Right? I Am Asking, As Perhaps It Is Possible To Have Custom Wid

yes

5 years ago
0 Hi! I Have Problem With Login To Trains. We Have Created Users That Until Yesterday Have No Problem To Access App, But Now It Throws Invalid User/Password Combination For Everyone. I Have Checked Apiserver Configuration And Everything Looks Ok. Do You Kno

No, they were not SuccessfulKoala55

5 years ago
0 I Have A Local Version Of Trains Server (Installed On A Vm In Gcp), One Problem I Run Into Was That Task.Init() Would Hang, And In The Ui I Could See "Drafts" Of Experiments. After Doing Docker-Compose Down And Up, Everything Went Back To Normal. Unfortun

ok, thanks!

5 years ago
0 I Have A Local Version Of Trains Server (Installed On A Vm In Gcp), One Problem I Run Into Was That Task.Init() Would Hang, And In The Ui I Could See "Drafts" Of Experiments. After Doing Docker-Compose Down And Up, Everything Went Back To Normal. Unfortun

yes, but the local output was completely empty

5 years ago
0 Afaiu By Default Trains Logs All Tensorboard Things, Can This Be Turned Off?

that was quick, thanks!

5 years ago
0 Default Installation Of Trains-Server Exposes Only Logging Via Http Not Https, In Case The Server Has A Public Ip, What Is The Recommended Way For Increasing Security?

got it, thanks!

5 years ago
0 I Wanted To Ask About Html Reporting, If I Want To Do A More Fancy Visualization (Like Overlay Of Two Images Depending On Mouse Hovering), I Have To Inject This Html Into The Reporting Code, Right? I Am Asking, As Perhaps It Is Possible To Have Custom Wid

AgitatedDove14 if I use report_image can I get a URL to it somehow?

5 years ago
0 If I Am Using The Demo Servers, Do I Need To Do Something Special To Use

there was a problem with index order when converting from pytorch tensor to numpy array

5 years ago
0 What Is The Suggested Way Of Running Trains-Agent With Slurm? I Was Able To Do A Very Naive Setup: Trains-Agent Runs A Slurm Job. It Has The Disadvantage That This Slurm Job Is Blocking A Gpu Even If The Worker Is Not Running Any Task. Is There An Easy Wa

yes, happy to help! In fact I am also interested in the k8s glue, since in one of our use cases we are using jobs and not pods (to allow for spot instances in the cloud), but I need to dig digger into the architecture to understand what we need exactly from k8s glue.

5 years ago
0 In The Experiment List Page I Can Add Custom Columns (Like Metrics) Which Is Great, But Can I Save The View To Switch Between The Different Views, Or At Least To Have A Fixed View Per Experiment?

it does, thanks!

5 years ago
0 What Is The Suggested Way Of Running Trains-Agent With Slurm? I Was Able To Do A Very Naive Setup: Trains-Agent Runs A Slurm Job. It Has The Disadvantage That This Slurm Job Is Blocking A Gpu Even If The Worker Is Not Running Any Task. Is There An Easy Wa

I hope you can do this without containers.

5 years ago
0 What Is The Suggested Way Of Running Trains-Agent With Slurm? I Was Able To Do A Very Naive Setup: Trains-Agent Runs A Slurm Job. It Has The Disadvantage That This Slurm Job Is Blocking A Gpu Even If The Worker Is Not Running Any Task. Is There An Easy Wa

Unfortunately there is no docker, there is only singularity. This cluster is used by many users and docker is not secure enough.

5 years ago
0 If I Am Using The Demo Servers, Do I Need To Do Something Special To Use

AgitatedDove14 FYI: I am using pytorch

5 years ago
0 I Have A Local Version Of Trains Server (Installed On A Vm In Gcp), One Problem I Run Into Was That Task.Init() Would Hang, And In The Ui I Could See "Drafts" Of Experiments. After Doing Docker-Compose Down And Up, Everything Went Back To Normal. Unfortun

apiserver logs were clean, only 200s there

5 years ago
0 I Have A Local Version Of Trains Server (Installed On A Vm In Gcp), One Problem I Run Into Was That Task.Init() Would Hang, And In The Ui I Could See "Drafts" Of Experiments. After Doing Docker-Compose Down And Up, Everything Went Back To Normal. Unfortun

and the experiment did not produce any logs, shall I enable some debug flag?

5 years ago
0 What Is The Suggested Way Of Running Trains-Agent With Slurm? I Was Able To Do A Very Naive Setup: Trains-Agent Runs A Slurm Job. It Has The Disadvantage That This Slurm Job Is Blocking A Gpu Even If The Worker Is Not Running Any Task. Is There An Easy Wa

AgitatedDove14 going back to the slurm subject, I have local trains installed on the cluster with slurm so I am ready to test. At the same time I was thinking whether a simple solution would do the job:
a) [scale up agents] monitor the trains queue, if there is something that was not started for some amount of time, and the number of agents is not yet at the maximum, then add an agent,
b) [scale down agents] if all the tasks are running and there are idle agents, kill an idle agent.
Or do yo...

5 years ago
0 I Have Installed The Trains Pycharm Plugin, But Based On The

ok, got it, thanks!

5 years ago
0 I Have A Local Version Of Trains Server (Installed On A Vm In Gcp), One Problem I Run Into Was That Task.Init() Would Hang, And In The Ui I Could See "Drafts" Of Experiments. After Doing Docker-Compose Down And Up, Everything Went Back To Normal. Unfortun

SuccessfulKoala55 20 minutes at least

5 years ago
0 If I Am Using The Demo Servers, Do I Need To Do Something Special To Use

AgitatedDove14 thanks for the additional information:
yes, the report_image problem was resolved after I reordered dimensions in the tensor. is there an advantage in using tensorboard over your reporting? html reporting looks powerfull, can one inject some javascript inside?

5 years ago
0 What Is The Suggested Way Of Running Trains-Agent With Slurm? I Was Able To Do A Very Naive Setup: Trains-Agent Runs A Slurm Job. It Has The Disadvantage That This Slurm Job Is Blocking A Gpu Even If The Worker Is Not Running Any Task. Is There An Easy Wa

sure, we can deal with the drivers

5 years ago
Show more results compactanswers