
Reputation
Badges 1
25 × Eureka!I tried to export them to json and they don't take more than 50KB each, but maybe they take more memory internally?
Ballpark should be the same.
I'm already at 300MB of usage with just 15 tasks
Maybe it was not updated yet? meaning you had more and deleted? (I think this is updated asynchronously, with max of 24h)
Exactly! nice π
If you wan to change the Args, go to the Args section in the Configuration tab, when the Task is in draft mode you can edit them there
- Correct. Basically the order is restapi body dictionary-> preprocess -> process -> post-process -> restapi dictionary return
CourageousLizard33 if the two series are on the same graph, just click on the series in the legend, you can enable/disable it, and the scale will adjust automatically.
Regarding grouping, this is a feature that can be turned off, the idea is that we split the tag to title/series... So if you have the same prefix you get to group the TF scalars on the same graph, otherwise they will be on a diff title graph. That said you can make force it to have a series per graph like in TB. Makes sense?
Hi RobustRat47
the easiest way to reproduce the entire environment on you local machine:clearml-agent build --id <task_id> --target ~/debug-full-env/
This will install an entire venv including code and applying git changes:
You can also create a container with everything:
https://clear.ml/docs/latest/docs/clearml_agent#task-container
CourageousLizard33 so you have a Linux server running Ubuntu VM with Docker inside?
I would imagine that you could just run the docker on the host machine, no?
BTW, I think 8gb is a good recommendation for a VM it's reasonable enough to start with, I'll make sure we add it to the docs
Thank you so much @<1572395184505753600:profile|GleamingSeagull15> !
looks like your
faq.clear.ml
site is missing from your main sites sitemap files,
Thank you for noticing! I'll check with the webdevs
Also missing the
robots
meta tag on that site,
π
Last tip is to add a link on the
faq.clear.ml
site back to
clear.ml
for search index relevancy ( connects the two sites as being related in content...
PunySquid88 RC1 is out with a fix:pip install trains-agent==0.14.2rc1
DefeatedCrab47 If I remember correctly v1+ has their arguments coming from argparse .
Are you using this feature ? 2. How do you set the TB HParam ? Currently Trains does not support TB HParams, the reason is the set of HParams needs to match a single experiment. Is that your case?
UI for some anomalous file,
Notice the metrics are not files/artifacts, just scalars/plots/console
Hi FreshKangaroo33
clearml.conf is HOCON format, to parse you can use pyhocon:
https://github.com/chimpler/pyhocon
Or the built in version of clearml:from clearml.utilities.pyhocon import ConfigFactory config_dict = ConfigFactory.parse_string(text).as_plain_ordered_dict()
You can also just get the parsed objectfrom clearml.config import config_obj
Actually you cannot breakpoint at "atexit" calls (or at least doesn't work with my gdb)
But I would add a few prints here:
https://github.com/allegroai/clearml/blob/aa4e5ea7454e8f15b99bb2c77c4599fac2373c9d/clearml/task.py#L3166
Hi @<1695969549783928832:profile|ObedientTurkey46>
How can I connect clearml to a relational database, and have sql query as a dataset? (e.g. dataset.add_references(query = βselect * from images where label = β1β)).
hmm interesting, you have a couple of options that I can think of:
- You can have your query and an argument to the Task, which means it is logged and can be changed later from the UI when you are relaunching it.
- You can have the query an an argument for a preprocessin...
Hi ConvolutedSealion94
Just making sure, you spinned the docker-compose of the clearml serving as well ?
(BTW: any reason not to use the agent?)
Hmm yeah I think that makes sense. Can you post here the arguments?
I'm assuming you have something like '1.23a' in the arguments?
RobustRat47 what's the Triton container you are using ?
BTW, the Triton error is:model_repository_manager.cc:1152] failed to load 'test_model_pytorch' version 1: Internal: unable to create stream: the provided PTX was compiled with an unsupported toolchain.
https://github.com/triton-inference-server/server/issues/3877
TRAINS_WORKER_NAME=first_agent trains-agent --gpus 0
andTRAINS_WORKER_NAME=second_agent trains-agent --gpus 0
time.sleep(time_sleep)
You should not call time.sleep in async functions, it should be asyncio.sleep,
None
See if that makes a difference
Hi ColossalAnt7
Following on SuccessfulKoala55 answer
I saw that there is a config file where you can specify specific users and passwords, but it currently requires
- mount the configuration file (the one holding the user/pass) into the pod from a persistent volume .
I think the k8s way to do this would be to use mounted config maps and secrets.
You can use ConfigMaps to make sure the routing is always correct, then add a load-balancer (a.k.a a fixed IP) for the users a...
ConvolutedSealion94 if you do bash:cd ~/work/repo/code/ git status
what are you getting ?
Notice that if you pass string it will split it based on spaces
Hi @<1578193378640662528:profile|MoodySeaurchin4>
but is it possible to log some metrics too, like rmse or the likes? If so, how would you do it?
Sure, I'm assuming this is part of the output ? if not, this means this is part of your code, and if this is the case then yes you should use collect_custom_statistics_fn
None
`collect_custom_statistics_fn({'rmse'...
ahh, because task_id is the "real" id of a task
Yes the ID is a global system wide unique ID (regardless of the project etc.)
Maybe we will call tasks as
slug_yyyymmdd
Notice that you can just copy-paste the link in the address bar, it will bring you to the exact same view, meaning easily shared among users π You can, but I would actually use the Task ID. This also means that programatically you can do , task=Task,get_task(task_id_here)
and interact and query a...
I'm sorry JitteryCoyote63 No π
I do know that the enterprise addition have these features (a.k.a vault & permissions), basically to answer these types of situations.