Nice job

yeah sure

You can use this to reproduce

in my case, we need to evaluate the result across many random seeds, so each task needs to log the result independently.

now it has log, but only the initial one

i can only see the init log

working very well actually

Thanks, it seems i need to forward all three ports

yes, that’s the error i got

not sure if it’s the problem with map

You are awesome, thanks again!

i’m trying to install it my lab server, but the same problem happen, when i try to create credentials, it say error but this time it give more info:
Error 301 : Invalid user id: id=f46262bde88b4928997351a657901d8b, company=d1bd92a3b039400cbafc60a7a5b1e52b

one quick question is that do i need to do some task.close() at the end of each process?

btw, i’m running the code with multiprocessing, so the log is duplicated 3 times

but the thing is that i can only use master to log everything

Not for now, i think it can only run on multiple GPU at one node

i see, now we are trying to let the agent pop up the experiment separately and see if they can communicate with each other, right?

Yeah, i’m done with the test, not i can run as what you said

never done this before, let me do a quick search

yes

I tried from clearml.backend_api.session import client no luck

I’ll get back to you after i get this done

is there any document for this?

and one experiment takes 40 hours to run, so i let them run in parallel

Guess I’ll need to implement job schedule myself

Ok, i’ll try to do that asap

But it seems buggy

i tired to run trains-compose without -d to say the log,
trains-agent-services | trains_agent: ERROR: Connection Error: it seems api_server is misconfigured. Is this the TRAINS API server http://apiserver:8008 ?
trains-agent-services | http://192.5.53.86:8081 http://192.5.53.86:8080 http://apiserver:8008
I didn’t assign anything to TRAINS_HOST_IP, not sure if the apiserver:8008 caused the problem