Furthermore, let's say I have 6 GPUs on a machine, and I'd like trains to treat this machine as 2 workers (gpus 0-2, 3-5), is there a way to do that?

Let me update you about this in a couple of minutes

Will try, thanks

let me try

If I'd be exact that's a trains agent task that creates in a new subprocess another trains agent task

Run a remote task with trains agent that would create inside another task that would again run remotely as well and check the permissions of the second task created file?

But that's not the case

Well the original task is run with my user

So I'd guess they would inherit my user as well

Changing the mountpoint for the agent is not possible

Could you maybe check this?

For me

Also, why there is a project id where there is a project name that exists? I don't even know how to display a project's id haha

Quite hard for me to check locally, could you check it using trans-agent?

Thanks

Nope, I didn't change anything

Great! Thanks.

otherwise if you empty the installed packages and the requirements.txt is in one of the parents folder of the files that ran trains should detect it automatically

If this could be so, I'd be happy to have this as a feature, this really impacts my pipeline flow.

I am aware this is the current behavior, but could it be changed to something more intelligent? 😇

So what can I do?

When you say I can still get race/starvation cases, you mean in the enterprise or regular version?

I see, will keep that in mind. Thanks Martin!

This is the path:
/Remote/moshe/Experiments/trains_bs_pipe_new/ypi/OKAY/Try_That/baseline/evaluation_validation/results/images/bottom_scores/0.0_slot02_extracted_23_01__1035__1.png

I've investigated it some more, It isn't path related as far as I can tell, as these same paths worked 2 weeks ago and a normal path doesn't work now

I bet it has something to do with the server or DB, any clue?

trains ( 0.16.1-320 )