I am using pip as a package manager, but i start the trains-agent inside a conda env 😄

The experiment finished completely this time again

ok, so there is no way to cache it and detect when the ref changes?

But I am not sure it will connect the parameters properly, I will check now

of which task/process?

Not sure about that, I think you guys solved it with your PipelineController implementation. I would need to test it before giving any feedback 🙂

yes

Thanks!3. I don't know, I never used Highcharts 🙂

Ok yes, I get it, this info is also available at the very beginning of the logs, where the agent logs the full docker run command, this docker_cmd is a shorter version?

I am not using hydra, I am reading the conf with:
config_dict = read_yaml(conf_yaml_path) config = OmegaConf.create(task.connect_configuration(config_dict))

I mean, inside a parent, do not show the project [parent] if there is nothing inside

continue_last_task is almost what I want, the only problem with it is that it will start the task even if the task is completed

I hit F12 to check projects.get_all_ex but nothing is fired, I guess the web ui is just frozen in some weird state

Ping CostlyOstrich36 AgitatedDove14 SuccessfulKoala55 Just making sure this wasn't missed 🙂

This is how I start the agent that is running the two experiments in parallel:
python3 -m trains_agent --config-file "~/trains.conf" daemon --queue default --log-level DEBUG --detached

Hi CostlyOstrich36 ! no I am running on venv mode

no it doesn't! 3. They select any point that is an improvement over time

Awesome, thanks!

CostlyOstrich36 I don’t see such number, can you please share a screenshot of where to look at?

and then call task.connect_configuration probably

Alright, I had a look in the /tmp/.trains_agent_daemon_outabcdef.txt logs, not many insights from here. For the moment, I simply started a new trains-agent daemon in services mode and I will wait to see what happens.

Thanks! With this I’ll probably be able to reduce the cluster size to be on the safe side for a couple of months at least :)

Seems to works, I started a last one to confirm!

can it be that the merge op takes so much filesystem cache that the rest of the system becomes unresponsive?

Thanks for the hint, I’ll check the paid version, but I’d like first to understand how much efforts it would be to fix the current situation by myself 🙂

Seems like it just went unresponsive at some point

4 cpus, 8Gb

ProxyDictPostWrite._to_dict() will recursively convert to dict and OmegaConf will not complain then

yes but they are in plain text and I would like to avoid that

I asked this question some time ago, I think this is just not implemented but it shouldn’t be difficult to add? I am also interested in such feature!