Hi @<1523704757024198656:profile|MysteriousWalrus11>

"parents": [

  "step_two", 

  "step_four" 

], 

Seems like step 5 depends on steps 2+4 , how did you create it? what did the console say ?
Could it be your not actually passing any output from step3 ? how is it dependent on it ?

Seems correct.
I'm assuming something is wrong with the key/secret quoting ?!
Could you generate another one and test it ?
(you can have multiple key/secretes on the same user)

MoodyCentipede68 from your log

clearml-serving-triton | E0620 03:08:27.822945 41 model_repository_manager.cc:1234] failed to load 'test_model_lstm2' version 1: Invalid argument: unexpected inference output 'dense', allowed outputs are: time_distributed

This seems the main issue of triton failing to.load
Does that make sense to you? how did you configure the endpoint model?

Well from the error it seems there is no layer called "dense" , hence triton failing to find the layer returning the reult. Does that make sense?

Let me rerun the code and check

MoodyCentipede68 can you post the full docker-compose log (from spinning it until you get the error?)
You can just pipe the output to a file with :
docker-compose ... up > log.txt

PompousParrot44 obviously you can just archive a task and run the cleanup service, it will actually delete archived tasks older than X days.
https://github.com/allegroai/trains/blob/master/examples/services/cleanup/cleanup_service.py

TeenyFly97 the TL;DR is:
Task.close() should be called when you previously used Task.init (i.e the code creating the task)
Task.mark_stopped() should be called to stop a remote Task running.
I hope it helps 🙂

Its stored on the Task, you can see it under the execution tab in the UI

Hi @<1600299043865497600:profile|MagnificentSeaurchin90>
Any chance you can provide more info on the error?

if I want to compare two experiments the scalar plots do not load ( loading forever ).

I'm assuming the issue is the Plots tab? or is it the Scalars? what do you have in the Plots? can you send an image of the single experiment ?

at least you did not change permission of your K8s etcd folder 😄

BTW: could it be the Task.init is Not called on the "module.name" entry point, but somewhere internally ?

Hi PanickyMoth78

I had several pipeline components getting it and uploading files to is concurrently.

Should not be a problem

I've attached it's log file which only mentions skipping one file (a warning)

So what exactly is the error you are getting?

how can I for example convert it back to a pandas dataframe?

You can always report csv file with report_media as well, or if this is not for debugging maybe an artifact ?

It seems to follow a structure specific to clearml,

Actually plotly.js 🙂

conda list | grep matplotlib ?

there is a bug wherein both

Task.current_task()

and

Logger.current_logger()

return

None

.

This is not a bug this means something broke, the environment variable CLEARML_TASK_ID Has to be set inside the agent's process
How are you running it? (also log 🙂 , you can DM so it is not public here)

PanickyMoth78

Is it limited to

accounts? (

unfortunately, yes 😊 , but I'm sure sales will be able to hook you up ...

Hi MistakenDragonfly51
Notice that Models are their own entity, you can query them based on tags/projects/names etc.
Querying and getting Models is done by Model class:
https://clear.ml/docs/latest/docs/references/sdk/model_model#modelquery_models

task.get_models()

is always empty. (edited)

How come there are no Models on the Task? (in other words how come this is empty?)

Wait, that makes no sense to me. The API from python and the API from the UI are getting the same data from the backend ...
What are you getting with?
from clearml import Task task = Task.get_task(task_id=<put task id here>) print(task.models)

Well it seems we forgot that one 😞 I'll quickly make sure it is there.
As a quick solution (no need to upgrade)
task.models["output"]._models.keys()

SmarmyDolphin68 if you can reproduce the behavior in a standalone script , it will really accelerate fixing this issue

Regrading the project name:
set_project will support project_name in the next version 🙂 project_id=[p.id for p in Task.get_projects() if p.name==project_name][0]

AttractiveCockroach17
Can you print the configuration to console when you start he run (you will get a local print and then later the remote print), are they the same? Are the 3 runs the same (local / remote print)

Woot woot! 🤩

LovelyHamster1 NICE! 👍

which to my understanding has to be given before a call to an argparser,

SmarmySeaurchin8 You can call argparse before Task.init, no worries it will catch the arguments and trains-agent will be able to override them :)

Hi SkinnyPanda43
Every "commit" is a new version, so sync changes you need to either create a new version (with parent version as the previous one), and sync the local folder (or manually add/remove files).
If you do not need to actually store the "current" version, you can just reset the Task, and sync it again.
wdyt?

(torchvision vs. cuda compatibility, will work on that),

The agent will pull the correct torch based on the cuda version that is available at runtime (or configured via the clearml.conf)

I want each remote task to execute one instance of the hydra multirun, but I suspect the remote will try to run the full multirun by itself

if config.clearml.remote and task.running_locally(): task.execute_remotely( queue_name=config.clearml.queue_name, clone=True, exit_process=False ) returnI think this ensures the local execution actually triggers the remote one, so it should be as you expect, no?