Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Profile picture
EnthusiasticCow4
Moderator
32 Questions, 118 Answers
  Active since 11 March 2023
  Last activity one month ago

Reputation

0

Badges 1

98 × Eureka!
0 Votes
0 Answers
455 Views
0 Votes 0 Answers 455 Views
Currently, things run in a pipeline are automatically moved to a /.pipelines/ subproject. Is there a way to allow the tasks done via pipelines to show up as ...
9 months ago
0 Votes
7 Answers
586 Views
0 Votes 7 Answers 586 Views
10 months ago
0 Votes
2 Answers
618 Views
0 Votes 2 Answers 618 Views
11 months ago
0 Votes
6 Answers
235 Views
0 Votes 6 Answers 235 Views
I've noticed a change from ClearML 1.13.1 to 1.13.2 relating to the Hydra integration. I'm not yet convinced it's a bug but it could be but it also might jus...
4 months ago
0 Votes
12 Answers
123 Views
0 Votes 12 Answers 123 Views
Hello all, I want to clarify something. In the ClearML Task Scheduler .add_tast() method there's a parameter for schedule_function . I think I had some assum...
one month ago
0 Votes
6 Answers
136 Views
0 Votes 6 Answers 136 Views
2 months ago
0 Votes
2 Answers
606 Views
0 Votes 2 Answers 606 Views
Hello all. I'm generating an OutputModel in one task and using it as an InputModel for another task. Since there's already a timestamp on the model creation ...
10 months ago
0 Votes
29 Answers
193 Views
0 Votes 29 Answers 193 Views
3 months ago
0 Votes
1 Answers
641 Views
0 Votes 1 Answers 641 Views
If you're paying for the premium features would those be available to a self hosted server or only on the web client?
11 months ago
0 Votes
1 Answers
570 Views
0 Votes 1 Answers 570 Views
For model monitoring, is there a good feature in ClearML? If not is there a go-to tool that works well with this framework?
12 months ago
0 Votes
9 Answers
678 Views
0 Votes 9 Answers 678 Views
Hello all. I'm experimenting with ClearML and I've run into a strange issue. I used Task.init on a project, it logs to the app.clear.ml but it doesn't seem t...
one year ago
0 Votes
14 Answers
168 Views
0 Votes 14 Answers 168 Views
2 months ago
0 Votes
6 Answers
139 Views
0 Votes 6 Answers 139 Views
2 months ago
0 Votes
5 Answers
591 Views
0 Votes 5 Answers 591 Views
11 months ago
0 Votes
3 Answers
680 Views
0 Votes 3 Answers 680 Views
one year ago
0 Votes
0 Answers
112 Views
0 Votes 0 Answers 112 Views
2 months ago
0 Votes
7 Answers
654 Views
0 Votes 7 Answers 654 Views
one year ago
0 Votes
6 Answers
627 Views
0 Votes 6 Answers 627 Views
Hi all, I've been experimenting around with automating the data sync. This is related to this thread: None Here is my expectation, so correct me if I'm wrong...
one year ago
0 Votes
17 Answers
654 Views
0 Votes 17 Answers 654 Views
Hello again, Is there any way to get a list of the datasets from ClearML that excludes archived datasets?
10 months ago
0 Votes
16 Answers
489 Views
0 Votes 16 Answers 489 Views
I'm trying to spin up a task on an agent and inside the task I have two packages that I've created custom versions of and specified a git repo for in the req...
7 months ago
0 Votes
1 Answers
571 Views
0 Votes 1 Answers 571 Views
Hello all. I'm trying to figure out how to get the task from pipelines listed under the experiments tab or at least in a place that's easier to find. I'm run...
9 months ago
0 Votes
2 Answers
102 Views
0 Votes 2 Answers 102 Views
Why is one of the ghosts from Packman in the corner of my workspace folders? πŸ‘»
2 months ago
0 Votes
5 Answers
178 Views
0 Votes 5 Answers 178 Views
Hello all, I'm trying to queue a task in python but I'd like to reuse the prior task ID. In the webapp you can Reset and Enqueue a task and it'll reuse the t...
3 months ago
0 Votes
1 Answers
678 Views
0 Votes 1 Answers 678 Views
one year ago
0 Votes
3 Answers
673 Views
0 Votes 3 Answers 673 Views
I ran into something that I'd describe and an error but I want to verify this to be the case first. The error seems to be produced if I call output_model.upd...
12 months ago
0 Votes
5 Answers
576 Views
0 Votes 5 Answers 576 Views
Is there any way to exclude archived datasets from Dataset.list_datasets()?
9 months ago
0 Votes
2 Answers
516 Views
0 Votes 2 Answers 516 Views
9 months ago
0 Votes
5 Answers
554 Views
0 Votes 5 Answers 554 Views
9 months ago
0 Votes
1 Answers
536 Views
0 Votes 1 Answers 536 Views
I found an interesting error. If I run the following: # Load the model from remote storage if model_url is provided if cfg.clearml.get("model_url", None): # ...
10 months ago
0 Votes
7 Answers
544 Views
0 Votes 7 Answers 544 Views
AWS Autoscaler question: what is best practices for providing the autoscaled instances with access to credentials for things like S3? With self-hosted agents...
9 months ago
Show more results questions
0 Hi All, I'Ve Been Experimenting Around With Automating The Data Sync. This Is Related To This Thread:

Hi again @<1523701435869433856:profile|SmugDolphin23> ,

The approach you suggested seems to be working albeit with one issue. It does correctly identify the different versions of the dataset when new data is added, but I get an error when I try and finalize the dataset:

Code:

        if self.task:
            # get the parent dataset from the project
            parent = self.clearml_dataset = Dataset.get(
                dataset_name="[LTV] Dataset",
                dataset_project=...
one year ago
0 Hope Everyone'S Having A Nice Holiday Period. I'Ve Been Debating Between Cron And The Clearml Taskscheduler Cron Is The Solution I'M Currently Using But I Wanted To Understand The Advantages To Using The Taskscheduler. Right Now I'M Using The Classic Cro

Hi @<1523701205467926528:profile|AgitatedDove14> . I think I'm misunderstanding something here. I have the scheduler service running. Now that it's running how does one add a new task or remove an existing task from the scheduler? I get that I can add them before starting the scheduler service but once the service is running is there any way to connect to it and change the schedule?

I thought the advantage of this service would be we could schedule tasks just by connecting to the existing t...

3 months ago
0 I'M A Bit Confused. It Seems Like Something Has Changed With How Clearml Handles Recording Datasets In Tasks. It Used To Be The Case That When I Would Create A Dataset Under A Task, Clearml Would Record The Id Of The Dataset In The Hyperparameters/Datase

I see. Thanks for the insight. That seems to be the case. I'm struggling a bit with datasets. For example, if I wanted to trace the genealogy of a dataset that's used by traditional tasks and pipelines. I'll try and write something up about the challenges around that when I get the chance. But your comment revealed another issue:

It appears that the partial name matching isn't going well. I'm unclear why this wouldn't be matching. In the attached photo you can see the input for `partial_nam...

9 months ago
0 I'M A Bit Confused. It Seems Like Something Has Changed With How Clearml Handles Recording Datasets In Tasks. It Used To Be The Case That When I Would Create A Dataset Under A Task, Clearml Would Record The Id Of The Dataset In The Hyperparameters/Datase

The plot thickens. It seems like there's something odd going on with the interaction between [LTV] and additional text. If I just search [LTV] it works, if I just search Dataset Test it works, but if I put them together it breaks the search. Now that I think about it, there's other oddities that seem to happen in the web interface that might be explained by some bugs around using brackets in names.

9 months ago
0 I Know At Least One Other Person Has Posted About This Previously, But When I Interact With

I found I was having this issue as well. I don't have an alias defined in the pipeline but in a task and I get the same error. I'm not hosting my own server but using the free web service at the moment.

12 months ago
0 Hi All, Details: Both Projects Are Using Clearml V1.14.2Rc0 (But I'Ve Tested It With Other Versions). I'M Using The Web-App, So We'Re Not Hosting Our Own Cleaml-Server We Do Have A Server With Several Cleaml-Agents V1.7.0 I'M Running Into A Seemingly Co

I'm not sure why the logs were incomplete. I think part of the reason it wasn't pulling from the repo was that it was pulling from cache. I cleared the clearml cache for that project and reran it. This should be the full log.

2 months ago
0 Hi All, Details: Both Projects Are Using Clearml V1.14.2Rc0 (But I'Ve Tested It With Other Versions). I'M Using The Web-App, So We'Re Not Hosting Our Own Cleaml-Server We Do Have A Server With Several Cleaml-Agents V1.7.0 I'M Running Into A Seemingly Co

Hi Jake πŸ‘ ,

Maybe the content is cached? The repo isn't big. I didn't realize the log was missing content. I believe I copied everything but I'll double check in a moment.

2 months ago
0 I'M Trying To Spin Up A Task On An Agent And Inside The Task I Have Two Packages That I'Ve Created Custom Versions Of And Specified A Git Repo For In The Requirements.Txt. Example With Hydra-Core And Omegaconf:

Sorry I disappeared (went on a well deserved vacation). The problem is happening because of the ordering of the install. If I install using pip install -r ./requirements.txt then pip installs the packages in the order of the requirements file. However, during the installation process from ClearML, it installs the packages in order UNLESS there's a custom path provided, then it's saved for last. The reason this breaks my code is I have later packages that depend on the custom packages, as ...

6 months ago
0 I'M Trying To Spin Up A Task On An Agent And Inside The Task I Have Two Packages That I'Ve Created Custom Versions Of And Specified A Git Repo For In The Requirements.Txt. Example With Hydra-Core And Omegaconf:

Why? That's not how I authenticate. Also, if it was simply an issue with authentication wouldn't there be some error message in the log?

7 months ago
0 I'M Trying To Spin Up A Task On An Agent And Inside The Task I Have Two Packages That I'Ve Created Custom Versions Of And Specified A Git Repo For In The Requirements.Txt. Example With Hydra-Core And Omegaconf:

It's even attempting to install omegaconf but not from the repo, likely because it's a dependency of hydra-colorlog.

Collecting omegaconf<2.4,>=2.2
  Using cached omegaconf-2.2.3-py3-none-any.whl (79 kB)
  Using cached omegaconf-2.2.2-py3-none-any.whl (79 kB)
  Using cached omegaconf-2.2.1-py3-none-any.whl (78 kB)
7 months ago
0 Hello Again, Is There Any Way To Get A List Of The Datasets From Clearml That Excludes Archived Datasets?

@<1539780284646428672:profile|PoisedElephant79> Are you sure you're not simply referring to the get operation? That seems to exclude archived datasets. But I don't see anything like that for the list_datasets operation.

10 months ago
0 Hello All, I’M An Ml Engineer Looking To Transition Our Company To A New Mlops System. Many Of Our Projects Are Currently Built Around Hydra And I’M Attempting To See What I Would Need To Do To Integrate Clearml Into Our Workflow. I’M Fully Aware That You

The answer is simple but also not completely obvious to someone new to the platform. So you can inject new command line args that hydra will recognize. This is what the Hydra section of args is for. However, if you enable _allow_omegaconf_edit_: True , I think ClearML will β€œinject” the OmegaConf saved under the configuration object of the prior run, overwriting the overrides. I’ll experiment with this behavior a bit more to be sure.

one year ago
0 Hello Again, Is There Any Way To Get A List Of The Datasets From Clearml That Excludes Archived Datasets?

I had 2 datasets on archive and 0 unarchived. When I ran the following command:

Dataset.list_datasets(dataset_project=self.task.get_project_name(), only_completed=True)

It returned two entrees for the two datasets I had on archive.

10 months ago
0 I'M Running Into A Perplexing Issue. I Have Several Agents Running On A Workstation, I Also Am Directly Running Code From The Same Workstation. There Are Several Projects On The Workstation But One Of The Projects Is Struggling With Authentication To Git

Results:

I first tried uncommenting enable_git_ask_pass: false but it didn't resolve the issue.

I then cleared the cache in the vcs-cache folder, and that did fix the issue. This is the second time the cache seemed to have been the root cause of the problem. At some point I did move from token-based auth to ssh keys. Would this require clearing the cache for any project that was cached prior to the auth change?

2 months ago
2 months ago
0 I'M Trying To Understand The Difference Between Tasks And Pipelines. I Get The Big Picture But The Thing I Struggle To Understand Is The Interplay In The Code Itself. If You Set Up A Pipeline You Wouldn'T Include A Task Init? Or Is There A Reason To Use B

Thanks for your reply @<1523701070390366208:profile|CostlyOstrich36> Is there an example where a pipeline is built from existing tasks? I'd like to experiment with it and I don' t see any examples of what you describe with my (clearly lacking) google-fu. What happens if you wrap a function with a task.init() with a pipeline decorator or is that the process you're speaking of?

one year ago
0 Hey There, I Am A New User Of Clearml And Really Enjoying It So Far! I Noticed That My Model Checkpoints Are Saved After Each Epoch. Instead I Would Like To Only Save The Best And Last Model Checkpoint. Is That Possible? I Could Not Find Something Regardi

Depending on the framework you're using it'll just hook into the save model operation. Every time you save a model, which will probably happen every epoch for some subset of the training. If you want to do it with the existing framework you could change the checkpoint so that it only clones the best model in memory and saves the write operation for last. The risk with this is if the training crashes, you'll lose your best model.

Optionally, you could also disable the ClearML integration with...

one year ago
9 months ago
Show more results compactanswers