Hi PricklyRaven28 ! What dict do you connect? Do you have a small script we could use to reproduce?

JumpyDragonfly13 The function was auto-generated from the backend schema, so it was auto-included. This function is actually used by the UI when you press the Download full log button, but the SDK can't call it because the response is simply an octet-stream representing the log (the SDK expects a JSON containing some fields).
Anyway, you could've also done something like this to get the task log:
` from clearml import Task
t = Task.get_task(task_id="cecc9d51581d4fe3948b857d21af1ff3")
p...

Hi JumpyDragonfly13 ! Try using get_task_log instead of download_task_log

Hi OutrageousSheep60 ! The fix for Dataset.list_datasets() will be out in the next release of ClearML SDK. Sorry for the inconvenience!

Hi @<1523701523954012160:profile|ShallowCormorant89> ! This is not really supported, but you could use continue_on_fail to make sure you get to your last step: None

Hi @<1676400486225285120:profile|GracefulSquid84> ! Each step is indeed a clearml task. You could try using the step ID. Just make sure you pass the ID to the HPO step (you can do that by simply returning the Task.current_task().id

how does your model look like?

Hi @<1715900760333488128:profile|ScaryShrimp33> ! You can set the log level by setting the CLEARML_LOG_LEVEL env var before importing clearml. For example:

import os
os.environ["CLEARML_LOG_LEVEL"] = "ERROR"  # or str(logging.CRITICAL/whatever level) also works 

Note that the ClearML Monitor warning is most likely logged to stdout, in which case this message can't really be suppressed, but model upload related message should be

you could also try using gloo as the backend (it uses CPU) just to check that the subprocesses spawn properly

Hi @<1578555761724755968:profile|GrievingKoala83> ! It looks like lightning uses the NODE_RANK env var to get the rank of a node, instead of NODE (which is used by pytorch).
We don't set NODE_RANK yet, but you could set it yourself after launchi_multi_node :

import os    
current_conf = task.launch_multi_node(2)
os.environ["NODE_RANK"] = str(current_conf.get("node_rank", ""))

Hope this helps

I think I understand. In general, if your communication worked without clearml, it should also work when using clearml.
But you won't be able to upload an artifact using None for example, to the shared memory. Same thing for debug samples etc.

Hi @<1657918706052763648:profile|SillyRobin38> ! If it is compatible with http/rest, you could try setting api.files_server to the endpoint or sdk.storage.default_output_uri in clearml.conf (depending on your use-case).

Hi @<1590514584836378624:profile|AmiableSeaturtle81> ! Having tqdm installed in your environment might help

btw, to avoid clutter you could also archive runs you don't need anymore

Hi @<1628202899001577472:profile|SkinnyKitten28> ! What code do you see that is being captured?

Hi GiganticMole91 . You could use something like
` from clearml.automation import DiscreteParameterRange

HyperParameterOptimizer(
...,
hyper_parameters=[DiscreteParameterRange("epochs", values=[100]), ...] # epochs is static, ... represent the other params
) to get the same behaviour --params-override ` provides

you should be able to see it here: None

Is there any way to look at all the tasks that used that version of the dataset?
Not easily. You could query the runtime properties of all tasks and check for datasets used.
But what I would do is tag the task that uses a certain dataset, and then you should be able to query by tags

Thank you

That's unfortunate. Looks like this is indeed a problem 😕 We will look into it and get back to you.

Do you want to remove steps/add steps from the pipeline after it has ran basically? If that is the case, then it is theoretically possible, but we don't expose and methods that would allow you to do that...
What you would need to do is modify all the pipeline configuration entries you find in the CONFIGURATION section (see the screenshot), Not sure if that is worth the effort. I would simply create another version of the pipeline with the added/removed steps
![image](https://clearml-web-asset...

That is a clear bug to me. Can you please open a GH issue?

Maybe you want to use some other functions then the ones I quoted, so feel free to read the docs, you should be able to do this

Hi @<1717350332247314432:profile|WittySeal70> ! The pre_execute_callback runs before the task is even created. For better control, I recommend using status_change_callback

1 more thing: It's likely that you should do task.launch_multi_node(args.nodes * args.gpus) instead, as I see that the world size set by lightning corresponds to this value

Hi @<1581454875005292544:profile|SuccessfulOtter28> ! The logger is likely outdated. Can you please open a Github issue about it?

Hi @<1547390464557060096:profile|NuttyKoala57> ! You can use wildcards in auto_connect_framework to filter your models. Check the docs under init: None . You might also want to check out this GH thread for an another way to do this: None

Hi @<1765547897220239360:profile|FranticShark20> ! Do you have any other logs that could help us debug this, such as tritonserver logs?
Also, can you use model.onnx as the model file name both in the upload and default_model_filename, just to make sure this is not a file extension problem (this can happen with triton)

Hi @<1668427950573228032:profile|TeenyShells80> , the parent_datasets should be a list of dataset IDs or clearml.Dataset objects, not dataset names. Maybe that is the issue

I think that will work, but I'm not sure actually. I know for sure that something like us-east-2 is supported