Are you inheriting from their docker file ?
Yes, there is no real limit, I think the only requirements id docker v19+
@<1523701868901961728:profile|ReassuredTiger98> it works on my machine 😞
Are you suggesting the conf file did not set the default size? It sounds like a bug, can you verify?
It might be the file upload was broken?
You can disable it with:
Task.init('example', 'train', auto_connect_frameworks={'pytorch': False})
HugeArcticwolf77 actually it is more than that, you can embed the graphs now in the markdown, when you hove over any plot/graph/image you now have a new button that copies the embed test, so that you can directly copy it into your markdown editor (internal or external)
More documentation and screenshots are coming after the holiday, mean time you can check:
https://clear.ml/docs/latest/docs/webapp/webapp_reports
https://clear.ml/docs/latest/assets/images/webapp_report-695dddd2ec8064938bf8...
FlatOctopus65
In my local environment
pipeline_package
is installed in development mode
In order to install the package you need to specify the git repo of the package, this is how the pipeline would know where to bring it from.
Either install it locally with "pip install git+ https://github.com/ ...." or add tp the packages argument of the Pipeline wrapper packages = ["git+ https://github.com/ "] `
wdyt?
Oh this is so internally, the background thread can signal it is not deferred, are you saying there is bug or the code is odd?
Yes you can 🙂 (though not on the open-source version)
@<1523701868901961728:profile|ReassuredTiger98> what are you getting with:
nvidia-smi
And here:
ls -la /usr/local/
Hi @<1523701868901961728:profile|ReassuredTiger98>
is there something like a clearml context manager to disable automatic logging?
Sure just do a wildcard with the files you actually want to autolog the rest will be ignored:
None
task = Task.init(..., auto_connect_frameworks={'pytorch' : '*.pt'}
Hi @<1529633468214939648:profile|CostlyElephant1>
Is it possible to get user ID of the current user
On the Task.data object itself there should be a filed named " user " that's the user ID of the owner (creator) of the Task.
You can filter based on this id with
Tasks.get_tasks(..., task_filter={'user': ["user-id-here"]})
wdyt?
Hi TightDog77 _
HTTPSConnectionPool(host='
', port=443): Max retries exceeded with url: /upload/storage/v1/b/models/o?uploadType=resumable (Caused by SSLError(SSLError(1, '[SSL: DECRYPTION_FAILED_OR_BAD_RECORD_MAC] decryption failed or bad record mac (_ssl.c:2633)')))
This seems like a network error to GCP, (basically GCP python package thows it)
Are you always getting this error? is this something new ?
Hi DeliciousBluewhale87
This is the latest clearml-serving (stable release at GTC at the end of the month)
https://github.com/allegroai/clearml-serving/tree/dev
Generally speaking, clearml-sering is a control plane, preprocessing, ML inference, with Nvidia Triton for DL inference (fully transparent).
It allows you to spin an entire fully dynamic & scalable serving on top of k8s cluster. Once you spin the base containers, you can configure them live with a CLI, this includes adding new en...
Are you sure trains-server not trains package (i.e. backend)
Hmm I think this was the fix (only with TF2.4), let me check a sec
Hi @<1523702652678967296:profile|DeliciousKoala34>
What's the clearml-server version you are working with?
Can you check with the latest RC?
pip3 install clearml==1.9.2rc2
I pass my dataset as parameter of pipeline:
@<1523704757024198656:profile|MysteriousWalrus11> I think you were expecting the dataset_df dataframe to be automatically serialized and passed, is that correct ?
If you are using add_step, all arguments are simple types (i.e. str, int etc.)
If you want to pass complex types, your code should be able to upload it as an artifact and then you can pass the artifact url (or name) for the next step.
Another option is to use pipeline from dec...
2,3 ) the question is whether the serving is changing from one tenant to another, does it?
Hi @<1704304350400090112:profile|UpsetOctopus60>
https://clear.ml/docs/latest/docs/deploying_clearml/clearml_server_kubernetes_helm
Just use the helm charts. It's the easiest
Is it also possible to specify different user/api_token for different hosts? For example I have a github and a private gitlab that I both want to be able to access.
ReassuredTiger98 my apologies I just realize you can use ~/.git-credentials for that. The agent will automatically map the host .git-credentials into the docker :)
PunyBee36 to get https add an aws elb before the server , the elb will add the https to any outside connection
Feels like we've been over this
LOL, I think I can't wrap my head around the use case 🙂
When running locally, this is "out of the box", as we can init and close before and after each model.
I finally got it! Task.init should be dubbed "init Main task" , automagic kicks in Only when it is the only one existing. You remote execution is "linear" Task after Task, in theory a good candidate for pipeline.
Basically option (2) , the main task is being "replaced" (which loca...
Could it be that this is the callback that causes it?
None
Depending on your security restrictions, but generally yes.
in order to work with ssh cloning, one has to manually install openssh-client to the docker image, looks like that
Correct, you have to have SSH inside the container so that git can use it.
You can always install with the following setup inside your agent's clearml.conf:extra_docker_shell_script: ["apt-get install -y openssh-client", ]
https://github.com/allegroai/clearml-agent/blob/73625bf00fc7b4506554c1df9abd393b49b2a8ed/docs/clearml.conf#L145