Hi HealthyStarfish45 , how long would Task.init() hang for?
thanks, next time I will provide you will all the logs
Well, it's a bit tricky, but here's what you'll need to configure. Basically we'd like to see what requests are being sent to the server, and the best way to do that is using the urllib3 log used by Python's requests module. Since this log is "muted" by default (to prevent it from littering the experiment's log), you'll need to add the following in your trains.conf file:sdk.log.disable_urllib3_info: false logging.loggers.urllib3 { level: DEBUG handlers: [urllib_console] } logging.handlers.urllib_console { class: logging.StreamHandler level: DEBUG }
specifically the log for trains-apiserver
apiserver logs were clean, only 200s there
So a long time... I would start by getting all the docker containers logs
yes, but the local output was completely empty
and the experiment did not produce any logs, shall I enable some debug flag?
By experiment log I actually meant the console output you have locally when running the experiment, not what is showing in the UI - from your description it's pretty obvious there's some issue preventing Trains from communicating property with the server 🙂
And of course the full log for the experiment
This will print any server endpoint accessed by Trains during the experiment execution to the console