Reputation
Badges 1
25 × Eureka!Whats the trains server IP? It seems everything is configured with local host?
Hi SubstantialElk6
Generically, we would 'export' the preprocessing steps, setup an inference server, and then pipe data through the above to get results. How should we achieve this with ClearML?
We are working on integrating the OpenVino serving and Nvidia Triton serving engiones, into ClearML (they will be both available soon)
Automated retraining
In cases of data drift, retraining of models would be necessary. Generically, we pass newly labelled data to fine...
This, however, requires that I slightly modify the clearml helm chart with the aws-autoscaler deployment, right?
Correct 🙂
Yes that's the part that is supposed to only pull the GPU usage for your process (and sub processes) instead of globally on the entire system
LazyFish41 just making sure, you built a container from the docker file, and used it as base docker image for the Task, is that correct ?
Also notice the cleaml-agent will not change the entry point of the docker meaning if the entry point does not end with plain bash, it will not actually run anything
JumpyPig73 Do you see all the configurations under the Args section in the "Configuration" Tab ?
(Maybe I'm wrong and the latest RC does Not include the python-fire support)
I think this is due to the label map including some keys with a
.
in them.
Hi TenseOstrich47 what do you mean "label"
I have to leave i'll be back online in a couple of hours.
Meanwhile see if the ports are correct (just curl to all ports see if you get an answer) if everything is okay, try again to run the text example
Hi FunnyTurkey96
Which pip are you using, basically pip changed the dependency resolver after 20.1
Change: https://github.com/allegroai/clearml-agent/blob/aede6f4bac71c8fc56e7cf982318a48527953a3c/docs/clearml.conf#L57pip_version: "<20.2"
See if that helps
Hi @<1623491856241266688:profile|TenseCrab59>
It is kind of dark or are you asking about the graphs ?
HarebrainedBear62 this is what I have.
clearml-data will store all the files for you, and version the entire thing, make is a breeze to abstract the dataset from the code. Querying data is available using Apache Drill (though currently it is still not built into the platform, but we are planning to get there soon) Since this is Image based data/meta-data, I know the paid tier of ClearML, has n additional dedicated data management solution specifically for images, with full ability to query m...
or do you mean agent can convert https url to ssh??
Yep it does that automatically if you set: force_git_ssh_protocol: true
https://github.com/allegroai/clearml-agent/blob/42606d9247afbbd510dc93eeee966ddf34bb0312/docs/clearml.conf#L25
I was having this confusion as well. Did behavior for execute_remote change that it used to be Draft is Aborted now?
Actually it was changed, it used to reset the Task (then push into into execution queue if needed), with clearml v1.0, we now support pushing aborted Tasks back into queues, so execute_remotely aborts the Task (instead of reseting it)
(you can always manually reset it)
Hi SubstantialElk6
Unicodeencodeerror:'ascii' codec can't encode characters in position 296-297: ordinal not in range (128) (edited)
I'm assuming this is the usual UTF8 missing from the container.
Can you try to launch it with PYTHONIOENCODING=utf-8
?
Hi FiercePenguin76
Is catboost actually using TB or is it just writing to .tfevent on its own ?
Yes, just set system_site_packages: true
in your clearml.conf
https://github.com/allegroai/clearml-agent/blob/d9b9b4984bb8a83914d0ec6d53c86c68bb847ef8/docs/clearml.conf#L57
Hi TrickySheep9
Could you post the pipeline code here?
Also which clearml version are you using ?
Hi TrickySheep9
Hmm I think you are correct, exit remotely will not work inside a jupyter notebook because it will not be able to close it.
I was just revising workflows that might be similar, wdyt?
https://clearml.slack.com/archives/CTK20V944/p1620506210463400?thread_ts=1614234125.066600&cid=CTK20V944
BTW: I tested the code you previously attached, and it showed the plot in the "Plots" section
(Tested with latest trains from GitHub)
MagnificentSeaurchin79 YEY!!!!
Very cool!
Do you feel like making it public, I have the feeling a lot of people will appreciate it, this is very useful 🙂
LudicrousParrot69
Yes please add to GitHub 🙂 The problem is, if this is on single Task than we loose the ability have the nice interactive abilities (selecting diff scalars / parameters) etc...
I think the only way is using the API, with task.query_tasks and filter, would that have helped?
(also could you make sure all posts regrading the same question are put in the thread of the first post to the channel?)
what just happened next time and what is happening underneath.
Not sure I follow, is there still an issue ?
Thank you so much @<1572395184505753600:profile|GleamingSeagull15> !
looks like your
faq.clear.ml
site is missing from your main sites sitemap files,
Thank you for noticing! I'll check with the webdevs
Also missing the
robots
meta tag on that site,
🙏
Last tip is to add a link on the
faq.clear.ml
site back to
clear.ml
for search index relevancy ( connects the two sites as being related in content...
Hi @<1572395181150310400:profile|DeterminedHare56>
Yes Slack is not the best for knowledge sharing, but it is the easiest for users to communicate over, and it is the easiest to setup and scale.
Specifically you can find historical log of the Slack channel here: None
Which we hoped google will index, but seems like this is still not working as expected, if you have any inputs it will be great to improve it