allegroai/trains image hash f038c8c6652d

i ran it this week

seems like if i remove the plt.figure(figsize=(16, 8)) i start to see the figure title but not figure itself

screenshot

seems like CORS issue in the console logs

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at http://localhost:8081/Trains%20Test/LightGBM.56ca0c9c9ebf4800b7e4f537295d942c/metrics/LightGBM%20Feature%20Importance%20above%200.0%20threshold/plot%20image/LightGBM%20Feature%20Importance%20above%200.0%20threshold_plot%20image_00000000.png . (Reason: CORS request did not succeed).

seems like port forwarding had an issue.. fixed that.. now running test again to see if things workout as expected

i am simply proxying it using ssh port forwarding

will report back

ah ok.. anyway to avoid it or change it on my side

simply changing to show doesn't work in my case as i am displaying CM.. what about if i use matshow

it may be that i am new to trains but in my normal notebook flow they both are images and i as trains user expected it to be under the Plot section as i think this is an image.. as in nutshell all matplotlib plots display data as an image 🙂

yeah i still see it.. but that seems to be due to dns address being blocked by our datacenter

as i am seeing now my plots but they are lending into metrics section not plot section.

so as you say.. i don't think the issue i am seeing is due to this error

whereas i am using simple matplotlib now

any logs i can check or debug my side

you replied it already.. it was execute_remotely called with exit_true argument

there are multiple scripts under test/scripts folder.. example is running one script from that folder

allegroai/trains-agent-service image hash 03dc85869afe

in the above example task id is from a newly generated task like Task.init() ?

not so sure.. ideally i was looking for some function calls which enables me to create a sort of DAG which get scheduled at given interval and DAG has status checks on up streams task ... so if upstream task fails.. downstream tasks are not run

looking at the above link, it seems i might be able to create it with some boilerplate as it has concept of parent and child... but not sure how status checks and dependency get sorted out

an example achieving what i propose would be greatly helpful

as if its couple of weeks away.. i can wait

test package is not installed but its in the current working directory

trains-agent version as mentioned is 0.16.1 and server is 0.16.1 as well

not just fairness but the scheduled workloads will be starved of resources if say someone run training which by default take all the available cpus

the use case i have is to allow people from my team to run their workloads on set of servers without stepping over each other..

this looks good... also do you have any info/eta on next controller/service release you mentioning