Reputation
Badges 1
151 × Eureka!as I have wandb docker set up on the same VM for teting
I also get this from the logging
TRAINS Monitor: GPU monitoring failed getting GPU reading, switching off GPU monitoring
task_reporting = Task.init(project_name='project', task_name='report') tasks = Task.get_tasks(project_name='project', task_name='partial_task_name_here') for t in tasks: t.get_last_scalar_metrics() task_reporting.get_logger().report_something
Instead of get_last_scalar_metrics()
, I am using t._data.hyperparams['summary'] to get the metrics I needed
I see, I will look into the documentation of it, thanks Jake.
https://github.com/allegroai/clearml/commit/164fa357ed01704b11db67b8a7ac19791fbc49d1
This works. So it is still in master and should be included in 1.0.5?
Cool! Will have a look at the fix when it is done. Thanks a lot AgitatedDove14
It's for addition filtering only right? My use case is to prevent user accidentally querying the entire database.
I want to achieve something similar we would do in SQL
select * from user_query limit 100;
it's ok, I don''t think this is very important. thx
Thanks for your help. I will stick with task.connect() first. I have submit a Github Issue, thanks again AgitatedDove14
It's good that you have version your dataset with name, I have seen many trained model that people just replace the dataset directly.
using configuration directly it actually worse than using a dictionary for hyperparmaeters. It would do the diff line by line (notice the right experiment)
It would be nice if there is an "export" function to just export all/selected experiment table view
Great discussion, I agree with you both. For me, we are not using clearml-data, so I am a bit curious how does a "published experiment" locked everything (including input? I assume someone can still just go inside the S3 bucket and delete the file without Clearml noticing).
From my experience, absolute reproducibility is code + data + parameter + execution sequence. For example, random seed or some parallelism can cause different result and could be tricky to deal with sometimes. We did bu...
Ok, will prepare a PR and script to reproduce the error
I want the support for click as well, or is there any adhoc solution?
I have tried adding the line to conf but seems not working as well... are u able to run with proper logging?
https://github.com/allegroai/trains/blob/master/examples/frameworks/matplotlib/matplotlib_example.py
https://allegro.ai/docs/examples/frameworks/matplotlib/matplotlib_example/
I also don't understand why the last chart goes to DEBUG sample instead? the code does not seems to have difference with previous block.
https://github.com/quantumblacklabs/kedro-examples/blob/master/kedro-tutorial/conf/base/catalog.yml
I am actually using Kedro (a pipeline library), you can check out the yaml config here. There will be a lot of cases that I need to insert a new argument or dataset in between
Could u give me some pointers where ClearML auto-capture log/stdout? I suspect as Kedro has configuration on logging
and somehow ClearML fail to catch it.
Now my problem is clearml-agent pick up the job but fail to run the docker.
I am running on Window 10 Machine, is this not compatible?
How often does Trains record the usage? I tried running a dummy programming which sleeps for 20 seconds and I don't see any plot.
Is this configurable?