Reputation
Badges 1
383 × Eureka!I then install this wrapper SDK in my containers, notebook instances etc
AlertBlackbird30 :
--remote-gateway [REMOTE_GATEWAY] Advanced: Specify gateway ip/address to be passed to interactive session (for use with k8s ingestion / ELB)I see this in clearml-session - what’s the intent here?
AgitatedDove14 - i am disabling pytorch like above but still see auto models . I even see model import when running evaluation from a loaded model
AgitatedDove14 - looks like the serving is doing the savemodel stuff?
https://github.com/allegroai/clearml-serving/blob/main/clearml_serving/serving_service.py#L554
sure, will do AlertBlackbird30
Thanks AgitatedDove14 - is the datasets feature pretty new?
AgitatedDove14 - just saw about start_remotely - https://clear.ml/docs/latest/docs/references/sdk/automation_controller_pipelinecontroller#start_remotely
This means the services agent will take care of taking it to completion right?
Or WARNING should be param not found, but using General/param etc
I have convention of root task for experiments and a sub-project Pipelines for running “sandboxed” pipeline task. So I do some magic like this
Yeah got it, thanks!
Good question 🙂
this is what I am seeing in the logs:
` No tasks in queue 9154efd8a1314550b1c7882981720861
No tasks in Queues, sleeping for 5.0 seconds
No tasks in queue 9154efd8a1314550b1c7882981720861
No tasks in Queues, sleeping for 5.0 seconds
No tasks in queue 9154efd8a1314550b1c7882981720861
No tasks in Queues, sleeping for 5.0 seconds
No tasks in queue 9154efd8a1314550b1c7882981720861
No tasks in Queues, sleeping for 5.0 seconds
K8S Glue pods monitor: Failed parsing kubectl output...
But if haven’t done a task yet
The agent ip? Generally what’s the expected pattern to deploy and scale this for multiple models?
AgitatedDove14 - these instructions are out of date? https://allegro.ai/clearml/docs/docs/deploying_clearml/clearml_server_kubernetes_helm.html
Yes. It's a pickle file that I have added via OutputModel
OK i found what’s happening:
I had an additional Task.init() - just the blank one, to get the project name. Adding the disable to that as well fixed the issue
I am on 1.0.3
I am doing something like this with a yaml based pipelines DSL
and then do Task.current_task() it is None as expected
Any updates on trigger and schedule docs 🙂
I always have my notebooks in git repo but suddenly it's not running them correctly.
And other question is clearml-serving ready for serious use?
AlertBlackbird30 - i don’t understand why it can’t be a focus though. Probably missing some context.
Also the pipeline ran as per this example - https://github.com/allegroai/clearml/blob/master/examples/pipeline/pipeline_controller.py
Task.add_requirements would fit the bill yeah, thanks