Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Profile picture
VivaciousBadger56
Moderator
14 Questions, 84 Answers
  Active since 10 January 2023
  Last activity 11 months ago

Reputation

0

Badges 1

84 × Eureka!
0 Votes
1 Answers
512 Views
0 Votes 1 Answers 512 Views
Hi everyone, I am very new to ClearML. I would like to do a tutorial to gain practical experience with ClearML. I have seen that https://clear.ml/docs/latest...
one year ago
0 Votes
4 Answers
520 Views
0 Votes 4 Answers 520 Views
How can one combine ClearML with other tools. The documentation does not provide much information on integration. I am particularily interested in combining ...
one year ago
0 Votes
12 Answers
578 Views
0 Votes 12 Answers 578 Views
I am trying to run the urbandsounds8k example, but when I run "preprocessing" I get the error in the line self.metadata = Task.get_task(task_id=self.original...
one year ago
0 Votes
4 Answers
618 Views
0 Votes 4 Answers 618 Views
Is there an larger example on how to use task.connect other than in the doc string? For example None says "Once objects are connected to a task, all object e...
one year ago
0 Votes
0 Answers
570 Views
0 Votes 0 Answers 570 Views
How do I properly complete a task in a Python console? If I run the code from clearml import Task, TaskTypes task = Task.init(project_name='FirstTrial', task...
one year ago
0 Votes
9 Answers
576 Views
0 Votes 9 Answers 576 Views
What is the difference between Model and InputModel?
one year ago
0 Votes
1 Answers
670 Views
0 Votes 1 Answers 670 Views
In None it is said that code level variables are logged. I do not see where in the ClearML web app and I am not able to find a way to do this programmatically.
one year ago
0 Votes
2 Answers
503 Views
0 Votes 2 Answers 503 Views
one year ago
0 Votes
22 Answers
532 Views
0 Votes 22 Answers 532 Views
In
In None is said that the python environment is logged. I do not see where in the ClearML web app and I am not able to find a way to do this programmatically.
one year ago
0 Votes
12 Answers
663 Views
0 Votes 12 Answers 663 Views
I cannot get the configuration from a task: I run from clearml import Task, TaskTypes, Model, OutputModel, InputModel from clearml.task_parameters import Tas...
one year ago
0 Votes
8 Answers
654 Views
0 Votes 8 Answers 654 Views
one year ago
0 Votes
4 Answers
482 Views
0 Votes 4 Answers 482 Views
Hello everyone, I would like to know what your projects are in terms of the usage of ClearML pipelines? What are your most elaborate pipelines? So far, I am ...
11 months ago
0 Votes
45 Answers
16K Views
0 Votes 45 Answers 16K Views
one year ago
0 Votes
31 Answers
18K Views
0 Votes 31 Answers 18K Views
ClearML pipelines can be build from tasks, functions, and decorated functions, according to the examples in None . I am guessing there was a certain way of d...
one year ago
0 What Is The Difference Between Model And Inputmodel?

@<1523701070390366208:profile|CostlyOstrich36> , I am build a PoC, evaluating if we should use ClearML for our entire ML team and go Scale or Enterprise pricing. For that I need to know all/most capabilities and concepts of ClearML to see if ClearML is future-proof.

TL;DR: difficult to narrow it down, but we (amongst other things), we need a model store

one year ago
0 Is There A Way To Save The Models Completely On The Clearml Server? It Seems That Clearml Server Does Not Store The Models Or Artifacts Itself, But They Are Stored Somewhere Else (E.G., Aws S3-Bucket) Or On My Local Machine And Clearml Server Is Only Sto

@<1523701087100473344:profile|SuccessfulKoala55> Also, I think that - in this case, but also in other cases - the issue is not just the documentation, but also the design of the SDK.

one year ago
0 What Is The Difference Between Model And Inputmodel?

Also, I could not find any larger examples on github about Model, InputModel, or OutputModel. It's kind of difficult to build a PoC this way... 😅

one year ago
0 What Is The Difference Between Model And Inputmodel?

@<1523701070390366208:profile|CostlyOstrich36> : After more playing around, it seems that ClearML Server does not store the models or artifacts itself. These are stored somewhere else (e.g., AWS S3-bucket) or on my local machine and ClearML Server is only storing configuration parameters and previews (e.g., when the artifact is a pandas dataframe). Is that right? Is there a way to save the models completely on the ClearML server?

one year ago
0 Clearml Pipelines Can Be Build From Tasks, Functions, And Decorated Functions, According To The Examples In

What is helper_functions in

  • None
  • Nonegood for?
    I do not find any example. The descriptions says

By default the pipeline step function has no access to any of the other functions, by specifying additional functions here, the remote pipel...

one year ago
0 What Is The Difference Between Model And Inputmodel?

As far as I understand, the workflow is like this. I define some model. Then I register it as an OutputModel. Then I train it. During training I save snapshots (not idea how, though) and then I save the final model when training is finished. This way the Model is a) connected to the task and b) available in the model store of ClearML.

Later, in a different task, I can load an already trained model with InputModel. This InputModel is read-only (regarding the ClearML model store), but I can ma...

one year ago
0 What Is The Difference Between Model And Inputmodel?

@<1523701070390366208:profile|CostlyOstrich36> : Thanks, where can I find more information on ClearML's model repository. I hardly find any in the documentation.

Also, that leaves the question open, what Model is for. I described how I understand, the workflow should look like, but my question remains open...

one year ago
0 Clearml Pipelines Can Be Build From Tasks, Functions, And Decorated Functions, According To The Examples In

@<1523701083040387072:profile|UnevenDolphin73> , you wrote

Well, I would say then that in the second scenario it’s just rerunning the pipeline, and in the third it’s not running it at all

(I mean, in both the code may have changed, the only difference is that in the second case you’re interested in measuring it, and in the third,
you’re not, so it sounds like a user-specific decision).

Well, I would hope that in the second scenario step A is not rerun. Yes, in the third scen...

one year ago
0 Clearml Pipelines Can Be Build From Tasks, Functions, And Decorated Functions, According To The Examples In

@<1523701205467926528:profile|AgitatedDove14> , you wrote

  • Components anyway need to be available when you define the pipeline controller/decorator, i.e. same codebase

No you an specify a different code base, see here:

Is the code in this "other" repo downloaded to the agent's machine? Or is the component's code pushed to the machine on which the repository is?
If the second case is true: How is the other machine (on which the other repo is lying on) turned into an agent?

one year ago
0 Is There A Way To Save The Models Completely On The Clearml Server? It Seems That Clearml Server Does Not Store The Models Or Artifacts Itself, But They Are Stored Somewhere Else (E.G., Aws S3-Bucket) Or On My Local Machine And Clearml Server Is Only Sto

I have already been trying to contribute (have three pull requests), but honestly I feel it is a bit weird, that I need to update a documentation about something I do not understand, while I actually try to evaluate if ClearML is the right tool for our company...

one year ago
0 Clearml Pipelines Can Be Build From Tasks, Functions, And Decorated Functions, According To The Examples In

@<1523701083040387072:profile|UnevenDolphin73> : A big point for me is to reuse/cache those artifacts/datasets/models that need to be passed between the steps, but have been produced by colleagues' executions at some earlier point. So for example, let the pipeline be A(a) -> B(b) -> C(c), where A,B,C are steps and their code, excluding configurations/parameters, and a,b,c are the configurations/parameters. Then I might have the situation, that my colleague ran the pipeline A(a) -> B(b) -> C(c...

one year ago
0 Is There A Way To Save The Models Completely On The Clearml Server? It Seems That Clearml Server Does Not Store The Models Or Artifacts Itself, But They Are Stored Somewhere Else (E.G., Aws S3-Bucket) Or On My Local Machine And Clearml Server Is Only Sto

@<1523701087100473344:profile|SuccessfulKoala55> I think I might have made a mistake earlier - but not in the code I posted before. Now, I have the following situation:

  • In my training Python process on my notebook I train the custom made model and put it on my harddrive as a zip file. Then I run the code
output_model = OutputModel(task=task, config_dict={...}, name=f"...")
output_model.update_weights(weights_filename=r"C:\path\to\mymodel.zip", is_package=True)
  1. I delete the "...
one year ago
0 Is There A Way To Save The Models Completely On The Clearml Server? It Seems That Clearml Server Does Not Store The Models Or Artifacts Itself, But They Are Stored Somewhere Else (E.G., Aws S3-Bucket) Or On My Local Machine And Clearml Server Is Only Sto

@<1523701070390366208:profile|CostlyOstrich36>

My training outputs a model as a zip file. The way I save and load the zip file to make up my model is custom made (no library is directly used), because we invented the entire modelling ourselves. What I did so far:

output_model = OutputModel(task=..., config_dict={...}, name=f"...")
output_model.update_weights("C:\io__path\...", is_package=True)

and I am trying to load the model in a different Python process with

mymodel =...
one year ago
0 I Cannot Get The Configuration From A Task: I Run

@<1523701205467926528:profile|AgitatedDove14> : "does that make sense ?" Not really.

"you do not need to automatically Add/Log/Track things into the Task in the current process." - I do not need to automatically do [...]? You mean I can do it automatically, but alternatively I can do it manually? Do you mean I use close within a process to prevent automatic logging/adding/tracking? But, as far as I know, after I used close I am not able to log etc. manually either. So...

"Mark...

one year ago
0 In

I just see the website that I linked to. I am not sure what is meant by "python environment". I cannot make a screen shot, because I do not know where to look for this in the first place.

one year ago
0 In

But still, in the web app the task is considered to be still "running". I am not sure what to do, so that the task is considered to be "completed".

one year ago
0 Hello Everyone, I Would Like To Know What Your Projects Are In Terms Of The Usage Of Clearml Pipelines? What Are Your Most Elaborate Pipelines? So Far, I Am Using "Only" A Pipeline That Looks Like This:

"using your method you may not reach the best set of hyperparameters."

Of course you are right. It is an efficiency trade-off of speed vs effectiveness. Whether this is worth it or not depends on the use-case. Here it is worth it, because the performance of the modelling is not sensitive to the parameter we search for first. Being in the ball-park is enough. And, for the second set of parameters, we need to do a full grid search (the parameters are booleans and strings); thus, this wo...

11 months ago
0 Is There A Way To Save The Models Completely On The Clearml Server? It Seems That Clearml Server Does Not Store The Models Or Artifacts Itself, But They Are Stored Somewhere Else (E.G., Aws S3-Bucket) Or On My Local Machine And Clearml Server Is Only Sto

@<1523701083040387072:profile|UnevenDolphin73> : I see. I did not make the connection that output_uri=True is what I was missing. I thought this was the default. But the default is actually "None", which is different than "True".

one year ago
0 Clearml Pipelines Can Be Build From Tasks, Functions, And Decorated Functions, According To The Examples In

No these are 3 different ways of building pipelines.

That is what I meant to say 🙂 , sorry for the confusion, @<1523701205467926528:profile|AgitatedDove14> .

@<1523701083040387072:profile|UnevenDolphin73> , your point is a strong one. What are clear situations in which pipelines can only be build from tasks, and not one of the other ways? An idea would be if the tasks are created from all kinds of - kind of - unrelated projects where the code that describes the pipeline does not ...

one year ago
0 I Cannot Get The Configuration From A Task: I Run

Secondly, I do not understand this:

None says

Manually mark a Task as completed. This will close the running process and will change the Task’s status to Completed (Use this function to close and change status of remotely executed tasks). To simply change the Task’s status to completed, use task.close()

None says

Closes the current Task and cha...

one year ago
0 Hello Everyone, I Would Like To Know What Your Projects Are In Terms Of The Usage Of Clearml Pipelines? What Are Your Most Elaborate Pipelines? So Far, I Am Using "Only" A Pipeline That Looks Like This:

@<1537605940121964544:profile|EnthusiasticShrimp49> : The biggest advantage I see to split your code into pipeline components is caching. A little bit structuring your code, but I was told by the staff this should not one's main aim with ClearML components. What is your main take away for splitting your code into components?

My HPO on top of the pipeline is already working 🙂 I am currently experimenting on using the HPO in a (other) pipeline that creates two HPO steps (from the same funct...

11 months ago
0 In

Ah, wait, when I run the entire script, I do see the respective information in "execution".

one year ago
0 In

It means "The syntax for the file name, folder name or volume label / disk is wrong" somthing along those lines. The [...] is the directory path to my project, which I opened in PyCharm and from which I run the commands in the Python Console.

one year ago
0 In

But then I do not see the "installed packages" in "excecution".

one year ago
0 In

>pip show clearml
WARNING: Ignoring invalid distribution -upyterlab (c:\users\...\lib\site-packages)
WARNING: Ignoring invalid distribution -illow (c:\users\...\lib\site-packages)
Name: clearml
Version: 1.6.4
Summary: ClearML - Auto-Magical Experiment Manager, Version Control, and MLOps for AI
Home-page: None
`Auth...

one year ago
0 Clearml Pipelines Can Be Build From Tasks, Functions, And Decorated Functions, According To The Examples In

: What does

  • The component code still needs to be self-composed (or, function component can also be quite complex)

Well it can address the additional repo (it will be automatically added to the PYTHONPATH), and you can add auxilary functions (as long as they are part of the initial pipeline script), by passing them to

helper_functions

mean? Is it not possible that I call code that is somewhere else on my local computer and/or in my code base? That makes thi...

one year ago
Show more results compactanswers