Hi HappyLion37
It seems that you are "reusing" the Tasks. Which means the second time you open them you are essentially resetting the old run and starting all over.
Try to do:
task1 = Task.init('examples', 'step one', reuse_last_task_id=False) print('do stuff') task1.close() task2 = Task.init('examples', 'step two', reuse_last_task_id=False) print('do some more stuff') task2.close()

Hmm that is odd, it seemed to missed the fact this is a jupyter notbook.
What's the clearml version you are using ?

I see if this is the case try to set
'output_uri="file:///full/path/to/dir"'
Notice it has to have the full path there and the file:// prefix

Are any files uploaded? Like artifacts etc?

, i thought there will be some hooks for deploying where the integration with k8s was also taken care automatically.

Hi ObedientToad56
Yes you are correct, basically now you have a docker-compose (spinning everything, even though per example you can also spin a standalone container (mostly for debugging).
We are working on a k8s helm chart so the deployment is easier, it will be based on these docker-compose :
https://github.com/allegroai/clearml-serving/blob/main/docker/docker-comp...

CharmingBeetle38 try adding "General/" before the arguments. This means batch_size becomes General/batch_size. This is only because we are accessing the parameters externally, when the task is executed it is resolved automatically

I "think" I have a clue on the issue that is lost here in the translation:
Specifically to me it all comes down to the definition of "pipeline"
From the clearml perspective:
Manual Task - code that is executed by the user (or any other mechanism Outside of the agent)
Remote Task - code that is executed by the Agent

Pipeline is a Task
Pipeline can be "manual task" but also "remote task"
Pipeline generates "remote tasks"
Task status (e.g. pipeline status as it is also a Task) can be: draft, a...

You mean one machine with multiple clearml-agents ?
(worker is a unique ID of an agent, so you cannot have two agents with the exact same worker name)
Or do you mean two agents pulling from the same queue ? (that is supported)

Hi GrievingTurkey78
How are you getting different version than what is used in run time? it analyzes the PYTHONPATH just as python does ? How can I reproduce it? Currently you can use Task.add_requirements(package_name, package_version=None) This will not force it though, it is a recommendation (if it fails to find the package itself) maybe we can add force ?!What do you think?

Could it be it defaulted to the demo server instead of your own server?

ok so i accidentally (probably with luck) noticed the max_connection: 2 in the azure.storage config.

NICE!!!! 🎊
But wait where is that set?
None
Should we change the default or add a comment ?

Back to the feature request, if this is taken care of (both adding a missed package, and the S3 upload), do you still believe there is a room for this kind of feature?

So I can set output_uri = "s3://<bucket_name>/prefix" and the local models will be loaded into the s3 bucket by ClearML ?

Yes, magic 🙂

Yep I changed it

This means it will totally ignore the overrides and just take the OmegaConf, this is by design. You either use the overrides, or you configure the OmegaConf. LovelyHamster1 Does that make sense ?

Hmm maybe this is the issue, :

Conda error: UnsatisfiableError: The following specifications were found to be incompatible with a past
explicit spec that is not an explicit spec in this operation (cudatoolkit):

  - pytorch~=1.8.0 -> cudatoolkit[version='>=10.1,<10.2|>=10.2,<10.3']

This makes no sense, conda is saying pytorch=1.8 needs cudatoolkit <10.2/10.3 but actually it needs cudatoolkit 11.1

Hi LovelyHamster1
You mean when as a section name or a variable?
Could you change this example to include a variable that breaks the support ?
https://github.com/allegroai/clearml/blob/master/examples/frameworks/hydra/hydra_example.py

What I'm trying to do is to filter is between two datetimes...Is that possible?

could you expand ?

At the moment I'm querying by paging through the tasks as you recommended, and then filtering with standard python list-comprehension filters...Which is less than ideal.

At least let's do that better:
Use Task._query_tasks:
Task._query_tasks(order_by=['-started'], page_size=10, page=0, only_fields=['id', 'started'])You will get "lighter" objects returned, then you can filter them with code (but the request will be a lots faster)
SuccessfulKoala55 any suggestion on improving that ?

BTW: StickyMonkey98 if you feel like writing a few examples I think it will be easy to push into the docs, so that at least we improve iteratively...

Hi StickyMonkey98

I'm (again) having trouble with the lack of documentation regarding Task.get_tasks(task_filter={STUFF}).

Yes we really have to add documentation there... Let me add that to the todo list

How do I filter tasks by time started? It seems there's a "started" property, and the web ui uses "started" as a key-word in the url query, but task_filter results in an error when I try that...Is there some other filter keyword for filtering by start-time??

last 10 started ...

Glad to hear 🙂

The agent ip? Generally what’s the expected pattern to deploy and scale this for multiple models?

Yes the agent's IP, and with multiple agents, one would probably use k8s for the nodes, then configure ingest. This is the next step for the cleaml-serving, adding support for KFServing or manually configuring the ingest. wdyt?

That's why I want to keep it as separate tasks under a single pipeline.

Hmm Yes, if this is the case then you definitely have to have two Tasks (with execution info on each one).
So you could just create a "draft" pipeline Task and report everything to it? Does that make sense ?
(By design a pipeline is in charge of spinning the Tasks and pulling the data/metric from them if needed, in your case it sounds like you need the Tasks to push the data/metric onto the pipeline Task, this is ...

What's the matplotlib version ? and python version?

@<1523701868901961728:profile|ReassuredTiger98> it works on my machine 😞

AgitatedTurtle16 could you check with the latest clearml RC (I remember a similar issue was fixed).
pip install clearml==0.17.5rc3Then run again
clearml-task ...

Wow, those guys....

https://allegro.ai/docs/webapp/webapp_project/#customizing-the-experiments-table

With offline mode,
Later if you need you can actually import the execution (including artifacts etc.) you just need the zip file it creates when you are done.

Good news, there is an offline mode.
Task.set_offline(True)
If you want your code to be aware, you can do:
from trains import Task if Task.current_task(): Task.current_task().get_logger().report_confusion_matrix(...)