ok... is there any way to enforce using a given system wide env.. so agent doesn't need to spend time with env. creation

trains-agent version as mentioned is 0.16.1 and server is 0.16.1 as well

once i removed the seaborn plot then CM plots becomes visible again

i ran it this week

AgitatedDove14 it seems i am having issues when i restart the agent... it fails in creating/setting up the env again... when i clean up the .trains/venv-builds folder and run a job for agent.. it is able to create the env fine and run job successfully.. when i restart the agent it fails with messages like
` Requirement already satisfied: cffi@ file:///home/conda/feedstock_root/build_artifacts/cffi_1595805535531/work from file:///home/conda/feedstock_root/build_artifacts/cffi_1595805535...

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at http://localhost:8081/Trains%20Test/LightGBM.56ca0c9c9ebf4800b7e4f537295d942c/metrics/LightGBM%20Feature%20Importance%20above%200.0%20threshold/plot%20image/LightGBM%20Feature%20Importance%20above%200.0%20threshold_plot%20image_00000000.png . (Reason: CORS request did not succeed).

as when it run first time after .trains/venv-build cleaning, it output this message for this package - pycparser @ file:///home/conda/feedstock_root/build_artifacts/pycparser_1593275161868/work wondering why it specify this package as for most other packages it just prints the version number

yes delete experiments which are old or for some other reason are not required to keep around

i think for now it should do the trick... was just thinking about the roadmap part

seems like port forwarding had an issue.. fixed that.. now running test again to see if things workout as expected

ah ok.. anyway to avoid it or change it on my side

i understand.. its just if i have a docker image with correct env.. i would prefer if trains-agent can use that directly

this looks good... also do you have any info/eta on next controller/service release you mentioning

ok so controller task is a simple place holder which run infinitely and fetch a task template and queue it..

seems like if i remove the plt.figure(figsize=(16, 8)) i start to see the figure title but not figure itself

TimelyPenguin76 yeah when i run matplotlib with show plots does land under Plot section... so its mainly then the imshow part.. i am wondering why the distinction and what is the usual way to emit plots to debug samples

it may be that i am new to trains but in my normal notebook flow they both are images and i as trains user expected it to be under the Plot section as i think this is an image.. as in nutshell all matplotlib plots display data as an image 🙂

TimelyPenguin76 also is there any reason for trating show and imshow differently

looking at the above link, it seems i might be able to create it with some boilerplate as it has concept of parent and child... but not sure how status checks and dependency get sorted out

ok will report back

i know it support conda.. but i have another system wide env which is not base .. say ml so wondering if i can comnfigure trains-agent to use that... not standard practice but just asking if it is possible

an example achieving what i propose would be greatly helpful

yeah i still see it.. but that seems to be due to dns address being blocked by our datacenter

there are multiple scripts under test/scripts folder.. example is running one script from that folder

or is there any plan to fix it in upcoming release

not so sure.. ideally i was looking for some function calls which enables me to create a sort of DAG which get scheduled at given interval and DAG has status checks on up streams task ... so if upstream task fails.. downstream tasks are not run

it still tries to create a new env

couldn't find the licensing price for enterprise version

seems like CORS issue in the console logs