but now since 

Task.current_task()

 doesn't work on the pipeline object we have a serious problem

How is that possible ?
Is there a small toy code that can reproduce it ?

[Assuming the above is what you are seeing]
What I "think" is happening is that the Pipeline creates it's own Task. When the pipeline completes, it closes it's own Task, basically making any later calls to Tasl.current_task() return None, because there is no active Task. I think this is the reason that when you are calling process_results(...) you end up with None.
For a quick fix, you can do
pipeline = Pipeline(...) MedianPredictionCollector.process_results(pipeline._task)Maybe we should...

WackyRabbit7 just making sure I understand:
MedianPredictionCollector.process_results Is called after the pipeline is completed.
Then inside the function, Task.current_task() returns None.
Is this correct?

(I suspect you are correct, but I'm missing some information in order to understand where the problem is)
WackyRabbit7 can you send mock code that explains how you create the pipeline ?

we can add non-clearml code as a step in the pipeline controller.

Yes 🙂 , btw you can kind of already do that, with pre/post function callbacks (notice they are running from the same scope as the actual pipeline controller).
What exactly did you have in mind to put there ?

JitteryCoyote63 could you test the latest RC 😉
pip install clearml-agent==0.17.2rc4

Thanks GiganticTurtle0 !
I will try to reproduce with the example you provided. regardless I already took a look at the code, and I'm pretty sure I know what the issue is. We will be pushing a few fixes after the weekend, I'm hoping this one will be included as well 🙂

Hi TenderCoyote78

I'm trying to clearml-agent in my dockerfile,

I'm not sure I'm following, Are you traying to create a docker container containing the agent inside? for what purpose ?
(notice that the agent can spin any off the shelf container, there is no need to add the agent into the container it will take of itself when it is running it)

Specifically to your docker file:

RUN curl -sSL

| sh

No need for this line

COPY clearml.conf ~/clearml.conf

Try the ab...

... if we have direct access to the Kubernetes worker when we run K8S glue?

Correct, if you have a direct access to the Node (on your k8s cluster) from your laptop (assuming the clearml-session is running from the laptop), everything should work

Maybe we should rename it?! it actually creates a Task but will not auto connect it...

An easier fix for now will probably be some kind of warning to the user that a task is created but not connected

That is a good point, maybe if you do not have a "main" Task, then we print the warning (with some flag to disable the warning) ?

MelancholyElk85 that looks great, let me see how quickly we can push it (I think 1.1.5 needs to be pushed very soon, I'll check if we can have it before 🙂 )

LOL 😊

Correct 🙂

👍
Okay But we should definitely output an error on that

👍

It should be under script.diff:
'script': {'binary': '', 'repository': '', 'tag': '', 'branch': '', 'version_num': '', 'entry_point': '', 'working_dir': '', 'requirements': {'pip': ''}, 'diff': ''}For some reason this is empty in your case, are you seeing it in the UI?
If you are querying the current task (i.e. running) it might not be there yet.
You can call this internal function that returns only after the repo detection is done.
task._wait_for_repo_detection()

JitteryCoyote63 The release was delayed due a last minute issue, should be released later today. Anyhow the code is updated on GitHub, so you can start implementing :) let me know if I can be of help :)

Hi CluelessElephant89

Hi guys, if I spot issue with documentations, where should I post them?

The best way from our perspective PR the fix 🙂 this is why we put it on GitHub

Hi @<1655744373268156416:profile|StickyShrimp60>

My hydra OmegaConf configuration object is not always being picked up, and I am unable to consistently reproduce it.
... I am using clearml v1.14.4,

Hmm how can we reproduce it? what are you seeing what it does "miss" the hydra, i.e. are you seeing any Hydra section? how are you running the code (manually , agent ?)

Hi ExuberantBat52
I do not think you can... i would use aws secret manager to push the entire user list config file wdyt?

@<1523706266315132928:profile|DefiantHippopotamus88> seems like you are missing the ports 🙂

CLEARML_WEB_HOST="

"
CLEARML_API_HOST="

"
CLEARML_FILES_HOST="

"

Hi SkinnyPanda43
In your local machine do not pass output_uri at all, so nothing will be uploaded.
On the agent's configuration file configure, default_output_uri to the S3 bucket
(Notice you can always override them in the UI, see the bottom of the execution Tab)
https://github.com/allegroai/clearml-agent/blob/e93384b99bdfd72a54cf2b68b3991b145b504b79/docs/clearml.conf#L312

Depending on your security restrictions, but generally yes.

Hi ProudChicken98
How about saving it as a local YAML and upload the file itself as an artifact?

Hmm would uploading it as YAML string be better?

Ohh so even easier:
print(client.workers.get_all())

At the moment I'm querying by paging through the tasks as you recommended, and then filtering with standard python list-comprehension filters...Which is less than ideal.

At least let's do that better:
Use Task._query_tasks:
Task._query_tasks(order_by=['-started'], page_size=10, page=0, only_fields=['id', 'started'])You will get "lighter" objects returned, then you can filter them with code (but the request will be a lots faster)
SuccessfulKoala55 any suggestion on improving that ?

this results at the end of an experiment in an object to be saved under a given name regardless if it was dynamic or not?

Yes, at the end the name of the artifact is what it will be stored under (obviously if you reuse the name you basically overwrites the artifact)

and the inet of the same card ?