Hi @<1593413673383104512:profile|MiniatureDragonfly17>
These are the specific model input/output layers name.
The way Triton analyses PyTorch model is usually
input__0 then input__1 for the input layers and output__0 and so on for the results:
You can see an example here:
None

--input-size 1 28 28 --input-name "INPUT__0" --input-type float32    --output-size -1 10 --output-name "OUTPUT__0" --outpu...

Hmmm are you saying the Dataset Tasks do not have the "dataset" system_tag as well as the type ?

Is this a common case? maybe we should change the run_pipeline_steps_locally argument to False?
(The idea of run_pipeline_steps_locally=True is that it will be easier to debug the entire pipeline on the same machine)

GrievingTurkey78
Both are now supported, they basically act the same way 🙂
and log overrides + the final omegaconf

how can I start up the clearml agent using the clearml-agent image instead of SDK?

Not sure I follow, what do you mean instead of the SDK? and what is the "clearml-agent image" ?

In the UI, no 😞
From code, Yes 🙂

Hi

The Squash operation copies all the data and is no longer linked to previous commits?

Yes, basically the idea is if you have data version that relies on many parents that needs to be merged, the squash will create a merged copy and push it all as a single version, and then yes the parent versions are no longer needed

I thought this operation is like git squash but it seems to me

yeah... we did not want to actually delete the parents because unlike git, the operation is done ...

After it finishes the 1st Optimzation task, what's the next job which will be pulled ?

The one in the highest queue (if you have multiple queues)
If you use fairness it will pull in round robin from all queues, (obviously inside every queue it is based on the order of jobs).
fyi, you can reorder the jobs inside the queue from the UI 🙂
DeliciousBluewhale87 wdyt?

Sure. JitteryCoyote63 so what was the problem? can we fix something?

JitteryCoyote63 is this still an issue?

PompousParrot44 the fundamental difference is that artifacts are uploaded manually (i.e. a user will specifically "ask" to upload an artifact), models are logged automatically and a user might not want them uploaded (imagine debugging sessions, or testing).
By adding the 'upload_uri' arguments, you can specify to trains that you want all models to be automatically uploaded (not just logged).
Now here is the nice thing, when running using the trains-agent, you can have:
Always upload the mod...

@<1538330703932952576:profile|ThickSeaurchin47> can you try the artifacts example:
None
and in this line do:

task = Task.init(project_name='examples', task_name='Artifacts example', output_uri="

")

I think it was just pushed, including nested call you have to use the new argument for the decorator, helper_function
https://github.com/allegroai/clearml/blob/400c6ec103d9f2193694c54d7491bb1a74bbe8e8/clearml/automation/controller.py#L2392

They inherit from one another, so it does make sense. Also the add_tags is on the "main" Task and not the backend parent

ohh AbruptHedgehog21 if this is the case, why don't you store the model with torch.jit.save and use Triton to run the model ?
See example:
https://github.com/allegroai/clearml-serving/tree/main/examples/pytorch
(BTW: if you want a full custom model serve, in this case you would need to add torch to the list of python packages)

Hi ExasperatedCrocodile76
This is quite the hack, but doable 🙂
`
file_path = task.connect_configuration(name = 'augmentations', configuration = 'augmentations.py')

import importlib

module_name = 'augmentations'

spec = importlib.util.spec_from_file_location(module_name, file_path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module) `
https://stackoverflow.com/a/54956419

potential sources of slow down in the training code

Is there one?

Notice the args will be set on the connect call, so the check on whether they are empty should come after

actually no it is not, alpine is Not a good baseline, is is very very slim missing a ton of stuff.
I would use bullseye or slim (depending how many aux things you need on the container)
https://hub.docker.com//python/tags?page=1&name=bullseye
https://hub.docker.com//python/tags?page=1&name=slim-bullseye

GloriousPanda26 wouldn't it make more sense that multi run would create multiple experiments ?

When I'm setting up my Pipeline, I can't go "here are some brand new tasks, please run them",

I think this is the main point. Can you create those Tasks via Task.create and get what you want? If so, then sure you can do that:
` def create_step_task(a_node):
task = Task.create(...)
return task

pipe.add_step(
name="stage_process",
parents=["stage_data"],
base_task_factor=create_step_task
) `wdyt?

As for the node, this confusing bit is that this is text from the docs...

seems like pip 20.1.1 has the issue, but >= 22.2.2 do not.

Notice we changed the value there, it now has two versions, pne for python 3.10 < and one for python 3.10>=
The main reason is that pip changed their resolving algorithm, and the new one can break its own dependencies (i.e. pip freeze > requirements.txt -> pip install might not actually work)
None

well cudnn is actually missing from the base image...

Hmm, and when you zoom out, still cropped ?

Oh sure that makes sense, clone the experiment in the UI (right click, clone) then everything is editable :) both uncommitted changes, and branch / commit

Hey LethalDolphin75 , when it works, could you PR it?

Sorry found the code on the Task, duh 🙂
` # get_ipython().magic('pip install clearml')
import clearml
from clearml import Task
task = Task.init(project_name='examples', task_name='test param', reuse_last_task_id=False)
param = {
'tuple_double_quotes_r': (r"value\blah", 1),
'tuple_double_quotes': ("value\blah", 1),
'tuple_single_quotes': ('value\blah', 1),
"double_quotes_r": r"value\blah",
'double_quotes': "value\blah",
'single_quotes': 'value\blah'
...

You can always specify diff clearml.conf files with --config-file 🙂

That is exactly that, the trains-agent is replicating the code from the git repo, and trying to apply the git diff (see uncommitted changes section). Obviously it failed 🙂

BTW: if you need you can do the following:
` from clearml import Task
from clearml.automation import PipelineController

task = Task.init(project_name='pipelines', task_name='pipeline test')
task.set_base_docker(...)

the pipeline object is using the Current Task, hence docker image is set

pipe = PipelineController(...)

pipe.start() `