Correct, which makes sense if you have a stochastic process and you are looking for the best model snapshot. That said I guess the default use case would be min/max (and not the global variant)

Great to hear!

Also what's the additional p doing at the last line if the screenshot ?

Hi WackyRabbit7

I have a pipeline controller task, which launches 30 tasks. Semantically there are 10 applications, and I run 3 tasks for each (those 3 are sequential, so in the UI it looks like 10 lines of 3 tasks).

👍

In one of those 3 tasks that run for every app, I save a dataframe under the name "my_dataframe".

I'm assuming as an artifact:

What I want to achieve is once all tasks are over, to collect all those "my_dataframe" artifacts (10 in number), extract a sin...

We should probably change it so it is more human readable 🙂

SteadyFox10 With pleasure 🙂
BTW: you can retrieve the Task id from its name with
Task.get_tasks(project_name='my project', task_name='my task name')See https://allegro.ai/docs/task.html?highlight=get_tasks#trains.task.Task.get_tasks

Hi @<1661180197757521920:profile|GiddyShrimp15>
I think the is a better channel for this kind of question
(they will be able to help with that)

cuda 10.1, I guess this is because no wheel exists for torch==1.3.1 and cuda 11.0

Correct

how can I enforce a specific wheel to be installed?

You mean like specific CUDA wheel ?
you can simple put the http link to the wheel in the "installed packages", it should work

Verified, and already fixed with 1.0.6rc2

Okay that actually makes sense, let me check I think I know what's going on

Hi ConvincingSwan15

For the train.py do I need a setup.py file in my repo to work corerctly with the agent ? For now it is just the path to train,py

I'm assuming the train.py is part of the repository, no?
If it is, how come the agent after cloning the repository cannot find it ?
Could it be it was accidentally not added to the git repo ?

FrothyShark37 any chance you can share snippet to reproduce?

@<1523707653782507520:profile|MelancholyElk85> what are you trying to change ? maybe there is a better way?
BTW: if you do step_base_task.export_task() you can use the parts that you need in the dict and pass them to:
task_overrides argument in add_step (you might need to flatten the nested arguments with '.' , and thinking about it, maybe we should do that automatically?!)

Hmm okay, this does point to a mem leak, any chance this is reproducible?

EnviousStarfish54
Can you check with the latest clearml from github?
pip install git+

FloppyDeer99 what am I seeing in the screenshot ?

But why the url in es is different from it in web UI?

They are not really different, but sometimes the "url quote" is an issue (this is the process a browser will take a string url like a/b and convert it to a%2fb ),
I remember that there was an issue involving double quoting (this is when you have: a/b -> a%2fb -> a%252fb ), notice the last one replace "%" with "%25" as in your example...
Let me know i...

Here you go 🙂
(using trains_agent for easier all data access)
from trains_agent import APIClient client = APIClient() log_events = client.events.get_scalar_metric_data(task='11223344aabbcc', metric='valid_average_dice_epoch') print(log_events)

still it is a chatgpt interface correct ?

Actually, no. And we will change the wording on the website so it is more intuitive to understand.
The idea is you actually train your own model (not chatgpt/openai) and use that model internally, which means everything is done inside your organisation, from data through training and ending with deployment. Does that make sense ?

Hi SubstantialElk6
where exactly in the log do you see the credentials ?

/tmp/.clearml_agent.234234e24s.cfg

What's the exact setup ? (I mean are you using the glue? if that's the case I think the temp config file is only created inside the pod/docker so upon completion it will be deleted along side the pod.

We use an empty queue to enqueue our tasks in, just to trigger the scheduler

it's only importance is that the experiment is not enqueued anywhere else, but the trigger then enqueues it

👍

It's just that the trigger is never triggered
(Except when a new task is created - this was not the case)

Is the trigger controller running on the services queue ?

Now I need to figure out how to export that task id

You can always look it up 🙂
How come you do not have it?

One example is a node that resizes the images, this node receives as input a Dataset and iterates over each image, resizes it an outputs a new Dataset, which is used in the next node downstream in the Pipeline.

I agree, this sounds like a "function" rather than a job, so better suited for Kedro.

organization structure 

 and see for yourself (this pipeline has two nodes 

train_model

 and 

predict

 )

Interesting! let me dive into that and ...

Out of curiosity, what ended up being the issue?

Hi JitteryCoyote63

Or even better: would it be possible to have a support for HTML files as artifacts?

If you report html files as debug media they will be previewed, as long as the link is accessible.
You can check this example:
https://github.com/allegroai/trains/blob/master/examples/reporting/html_reporting.py

In the artifacts, I think html are also supported (maybe not previewed as nicely but clickable.
Regrading the s3 link, I think you are supposed to get a popup window as...

However, SNPE performs quantization with precompiled CLI binary instead of python library (which also needs to be installed). What would be the pipeline in this case?

I would imagine a container with preinstalled SNPE compiler / quantizer, and a python script triggering the process ?

one more question: in case of triggering the quantization process, will it be considered as separate task?

I think this makes sense, since you probably want a container with the SNE environment, m...

I'm hoping we are ready to release

Hi ConvolutedChicken69
assuming you are runnign the agent in venv mode you can do something like:
$ CLEARML_AGENT_SKIP_PIP_VENV_INSTALL=1 clearml-agent daemon --queue defaultThis will basically only clone the code and use the default python the clearml-agent itself is using.
Does that help?
BTW:

it gets an error as it can't find it with pip.

What's the error? how come the package cannot be installed ?

Great, you can test directly from the master 🙂
pip3 install -U git+

So dynamic or static are basically the same thing, just in dynamic, I can edit the artifact while running the experiment?

Correct

Second, why would it be overwritten if I run a different run of the same experiment?

Sorry, I meant in the same run, if you reuse the artifact name you will be overwriting it. Obviously different runs different artifacts :)

Yes, look for the clearml serving session ID in the web UI (just go to the home screen and put the UID in the search 🙂 )