I expect either 'var1' to be 'b' or - better - there to be log of the change, so that I would be able to see how the value changed over time.

@<1523701087100473344:profile|SuccessfulKoala55> Also, I think that - in this case, but also in other cases - the issue is not just the documentation, but also the design of the SDK.

KindChimpanzee37 , this time, I was away for a week 🙂 . I do not think, that I made the mistake you suggested. At the top of the script I wrote
project_name = 'RL/Urbansounds'and then later
` self.original_dataset = Dataset.get(dataset_project=project_name, dataset_name='UrbanSounds example')

This will return the pandas dataframe we added in the previous task

self.metadata = Task.get_task(task_id=self.original_dataset.id).artifacts['metadata'].get() `

I have... its included in None

@<1523701070390366208:profile|CostlyOstrich36> , I am build a PoC, evaluating if we should use ClearML for our entire ML team and go Scale or Enterprise pricing. For that I need to know all/most capabilities and concepts of ClearML to see if ClearML is future-proof.

TL;DR: difficult to narrow it down, but we (amongst other things), we need a model store

@<1523701205467926528:profile|AgitatedDove14> : "does that make sense ?" Not really.

"you do not need to automatically Add/Log/Track things into the Task in the current process." - I do not need to automatically do [...]? You mean I can do it automatically, but alternatively I can do it manually? Do you mean I use close within a process to prevent automatic logging/adding/tracking? But, as far as I know, after I used close I am not able to log etc. manually either. So...

"Mark...

@<1523701205467926528:profile|AgitatedDove14> : In general: If I do not build a package out of my local repository/project , I cannot reference anything
from the local project/repository directly, right? I must make a package out of it, or I must reference it with the repo argument, or I must reference respective functions using the helper_functions argument. Did I get this right?

I am not sure if it the fact the name of the file ends with .model is an issue - but that would be somewhat crazy design...

@<1523701070390366208:profile|CostlyOstrich36> indeed

>pip show clearml
WARNING: Ignoring invalid distribution -upyterlab (c:\users\...\lib\site-packages)
WARNING: Ignoring invalid distribution -illow (c:\users\...\lib\site-packages)
Name: clearml
Version: 1.6.4
Summary: ClearML - Auto-Magical Experiment Manager, Version Control, and MLOps for AI
Home-page: None
`Auth...

Ok, I checked: A is terminated. This is not what I thought would happen and not what I intended with my documentation. I should clarify that.

Thank you I found the error.
myPar = task.connect(myPar, name='from TaskParameters')
is required.

I see that with task.mark_completed(), I am able to programmatically complete the task.

Here is my code:

from clearml import Task, TaskTypes
from clearml.task_parameters import TaskParameters, param, percent_param

class MyParams(TaskParameters):

    iterations = param(
        type=int,
        desc="Number of iterations to run",
        range=(0, 100000),
    )

    target_accuracy = percent_param(
        desc="The target accuracy of the model",
    )

myPar = MyParams(iterations=1000, target_accuracy=0.95)
parameters_to_track1 = {'var1': 'a', 'hyper_par': 1}
parameter...

@<1523701205467926528:profile|AgitatedDove14>

@<1523701070390366208:profile|CostlyOstrich36> : Thanks, where can I find more information on ClearML's model repository. I hardly find any in the documentation.

Also, that leaves the question open, what Model is for. I described how I understand, the workflow should look like, but my question remains open...

@<1523701205467926528:profile|AgitatedDove14> : I am writing quite a bit of documentation on the topic of pipelines. I am happy to share the article here, once my questions are answered and we can make a pull request for the official documentation out of it.

KindChimpanzee37 : First I went to the dataset and clicked on "Task information ->" in the right bottom corner of the "VERSION INFO". I supposed that is the same as what you meant with "right click on more information"? Because I did not find any option to "right click on more information". The "Task information ->" leads me to a view in the experiment manager. I posted the two screen shots.

PS: It is weird to me that the datamanager leads me to the experiment manager, specifically an experi...

Maybe this is only logged after it is not "running" anymore, but I am not sure how to "complete" a task programmatically.

As far as I understand, the workflow is like this. I define some model. Then I register it as an OutputModel. Then I train it. During training I save snapshots (not idea how, though) and then I save the final model when training is finished. This way the Model is a) connected to the task and b) available in the model store of ClearML.

Later, in a different task, I can load an already trained model with InputModel. This InputModel is read-only (regarding the ClearML model store), but I can ma...

@<1523701070390366208:profile|CostlyOstrich36> : After more playing around, it seems that ClearML Server does not store the models or artifacts itself. These are stored somewhere else (e.g., AWS S3-bucket) or on my local machine and ClearML Server is only storing configuration parameters and previews (e.g., when the artifact is a pandas dataframe). Is that right? Is there a way to save the models completely on the ClearML server?

@<1523701083040387072:profile|UnevenDolphin73> : If I do, what should I configure how?

GrittyStarfish67 : In terms of "has a good name" you literally mean the name or do you mean, they have a good reputation 😄 ?

@<1523701070390366208:profile|CostlyOstrich36>

My training outputs a model as a zip file. The way I save and load the zip file to make up my model is custom made (no library is directly used), because we invented the entire modelling ourselves. What I did so far:

output_model = OutputModel(task=..., config_dict={...}, name=f"...")
output_model.update_weights("C:\io__path\...", is_package=True)

and I am trying to load the model in a different Python process with

mymodel =...

@<1523701083040387072:profile|UnevenDolphin73> : I do not get this impression, because during update_weights I get the message

2023-02-21 13:54:49,185 - clearml.model - INFO - No output storage destination defined, registering local model C:\Users..._Demodaten_FF_2023-02-21_13-53-51.624362.model

CostlyOstrich36 any ideas?

Do you mean "exactly" as in "you finally got it" or in the sense of "yes, that was easy to miss"?

But still, in the web app the task is considered to be still "running". I am not sure what to do, so that the task is considered to be "completed".

@<1523701083040387072:profile|UnevenDolphin73> : I see. I did not make the connection that output_uri=True is what I was missing. I thought this was the default. But the default is actually "None", which is different than "True".

@<1523701205467926528:profile|AgitatedDove14> : Wait, so, if a task is initialized in process A and I call mark_completed in a process B, which process is terminated? A or B?