Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Profile picture
EnthusiasticCow4
Moderator
32 Questions, 118 Answers
  Active since 11 March 2023
  Last activity 7 months ago

Reputation

0

Badges 1

98 × Eureka!
0 Votes
2 Answers
430 Views
0 Votes 2 Answers 430 Views
Why is one of the ghosts from Packman in the corner of my workspace folders? 👻
8 months ago
0 Votes
2 Answers
1K Views
0 Votes 2 Answers 1K Views
Hello all. I'm generating an OutputModel in one task and using it as an InputModel for another task. Since there's already a timestamp on the model creation ...
one year ago
0 Votes
3 Answers
1K Views
0 Votes 3 Answers 1K Views
I ran into something that I'd describe and an error but I want to verify this to be the case first. The error seems to be produced if I call output_model.upd...
one year ago
0 Votes
4 Answers
1K Views
0 Votes 4 Answers 1K Views
one year ago
0 Votes
6 Answers
566 Views
0 Votes 6 Answers 566 Views
8 months ago
0 Votes
7 Answers
1K Views
0 Votes 7 Answers 1K Views
one year ago
0 Votes
6 Answers
1K Views
0 Votes 6 Answers 1K Views
Hi all, I've been experimenting around with automating the data sync. This is related to this thread: None Here is my expectation, so correct me if I'm wrong...
one year ago
0 Votes
5 Answers
996 Views
0 Votes 5 Answers 996 Views
one year ago
0 Votes
1 Answers
984 Views
0 Votes 1 Answers 984 Views
Hello all. I'm trying to figure out how to get the task from pipelines listed under the experiments tab or at least in a place that's easier to find. I'm run...
one year ago
0 Votes
2 Answers
1K Views
0 Votes 2 Answers 1K Views
one year ago
0 Votes
5 Answers
621 Views
0 Votes 5 Answers 621 Views
Hello all, I'm trying to queue a task in python but I'd like to reuse the prior task ID. In the webapp you can Reset and Enqueue a task and it'll reuse the t...
8 months ago
0 Votes
1 Answers
1K Views
0 Votes 1 Answers 1K Views
For model monitoring, is there a good feature in ClearML? If not is there a go-to tool that works well with this framework?
one year ago
0 Votes
5 Answers
990 Views
0 Votes 5 Answers 990 Views
Is there any way to exclude archived datasets from Dataset.list_datasets()?
one year ago
0 Votes
12 Answers
525 Views
0 Votes 12 Answers 525 Views
Hello all, I want to clarify something. In the ClearML Task Scheduler .add_tast() method there's a parameter for schedule_function . I think I had some assum...
7 months ago
0 Votes
14 Answers
587 Views
0 Votes 14 Answers 587 Views
8 months ago
0 Votes
2 Answers
911 Views
0 Votes 2 Answers 911 Views
one year ago
0 Votes
0 Answers
817 Views
0 Votes 0 Answers 817 Views
Currently, things run in a pipeline are automatically moved to a /.pipelines/ subproject. Is there a way to allow the tasks done via pipelines to show up as ...
one year ago
0 Votes
7 Answers
1K Views
0 Votes 7 Answers 1K Views
AWS Autoscaler question: what is best practices for providing the autoscaled instances with access to credentials for things like S3? With self-hosted agents...
one year ago
0 Votes
29 Answers
613 Views
0 Votes 29 Answers 613 Views
9 months ago
0 Votes
6 Answers
555 Views
0 Votes 6 Answers 555 Views
8 months ago
0 Votes
17 Answers
1K Views
0 Votes 17 Answers 1K Views
Hello again, Is there any way to get a list of the datasets from ClearML that excludes archived datasets?
one year ago
0 Votes
0 Answers
509 Views
0 Votes 0 Answers 509 Views
7 months ago
0 Votes
1 Answers
1K Views
0 Votes 1 Answers 1K Views
one year ago
0 Votes
6 Answers
635 Views
0 Votes 6 Answers 635 Views
I've noticed a change from ClearML 1.13.1 to 1.13.2 relating to the Hydra integration. I'm not yet convinced it's a bug but it could be but it also might jus...
10 months ago
0 Votes
7 Answers
1K Views
0 Votes 7 Answers 1K Views
one year ago
0 Votes
7 Answers
1K Views
0 Votes 7 Answers 1K Views
one year ago
0 Votes
1 Answers
895 Views
0 Votes 1 Answers 895 Views
I found an interesting error. If I run the following: # Load the model from remote storage if model_url is provided if cfg.clearml.get("model_url", None): # ...
one year ago
0 Votes
16 Answers
1K Views
0 Votes 16 Answers 1K Views
I'm trying to spin up a task on an agent and inside the task I have two packages that I've created custom versions of and specified a git repo for in the req...
one year ago
0 Votes
1 Answers
1K Views
0 Votes 1 Answers 1K Views
If you're paying for the premium features would those be available to a self hosted server or only on the web client?
one year ago
0 Votes
9 Answers
1K Views
0 Votes 9 Answers 1K Views
Hello all. I'm experimenting with ClearML and I've run into a strange issue. I used Task.init on a project, it logs to the app.clear.ml but it doesn't seem t...
one year ago
Show more results questions
0 Hello All, I Want To Clarify Something. In The

Should I post this in dev?

7 months ago
0 Hi All, Details: Both Projects Are Using Clearml V1.14.2Rc0 (But I'Ve Tested It With Other Versions). I'M Using The Web-App, So We'Re Not Hosting Our Own Cleaml-Server We Do Have A Server With Several Cleaml-Agents V1.7.0 I'M Running Into A Seemingly Co

Project 2:

2024-01-22 17:21:56
task 6518c3cd13394aa4abbc8f0dc34eb763 pulled from 8a69a982f5824762aeac7b000fbf2161 by worker bigbrother:10
2024-01-22 17:22:03
Current configuration (clearml_agent v1.7.0, location: /tmp/.clearml_agent.bojpliyx.cfg):
----------------------
agent.worker_id = bigbrother:10
agent.worker_name = bigbrother
agent.force_git_ssh_protocol = true
agent.python_binary = /home/natephysics/anaconda3/bin/python
agent.package_manager.type = pip
agent.package_manager.pip_v...
8 months ago
0 Hope Everyone'S Having A Nice Holiday Period. I'Ve Been Debating Between Cron And The Clearml Taskscheduler Cron Is The Solution I'M Currently Using But I Wanted To Understand The Advantages To Using The Taskscheduler. Right Now I'M Using The Classic Cro

Alright, I fixed the issue with the scheduler eating itself. But now I'm still getting the same bug as two days ago. So the Scheduler process starts fine and doesn't "crash." But I don't get the config object in the web-app again. It seems to work if I run it locally.

To answer your earlier question, I'm using the app.clear.ml portal so

  • WebApp: 3.20.1-1525
  • Server: 3.20.1-1299
  • API: 2.28
  • And my Python ClearML version: 1.14
8 months ago
0 I'M Trying To Spin Up A Task On An Agent And Inside The Task I Have Two Packages That I'Ve Created Custom Versions Of And Specified A Git Repo For In The Requirements.Txt. Example With Hydra-Core And Omegaconf:

Sorry I disappeared (went on a well deserved vacation). The problem is happening because of the ordering of the install. If I install using pip install -r ./requirements.txt then pip installs the packages in the order of the requirements file. However, during the installation process from ClearML, it installs the packages in order UNLESS there's a custom path provided, then it's saved for last. The reason this breaks my code is I have later packages that depend on the custom packages, as ...

one year ago
0 I'M Running Into A Perplexing Issue. I Have Several Agents Running On A Workstation, I Also Am Directly Running Code From The Same Workstation. There Are Several Projects On The Workstation But One Of The Projects Is Struggling With Authentication To Git

I think this error occurred for me because when I first authenticated with the project I was using username/password and later I transitioned to using ssh keys. That's why clearing the cache worked.

Did you validate that branch exists on remote?

7 months ago
0 Hello Again, Is There Any Way To Get A List Of The Datasets From Clearml That Excludes Archived Datasets?

I had 2 datasets on archive and 0 unarchived. When I ran the following command:

Dataset.list_datasets(dataset_project=self.task.get_project_name(), only_completed=True)

It returned two entrees for the two datasets I had on archive.

one year ago
0 Hello Again, Is There Any Way To Get A List Of The Datasets From Clearml That Excludes Archived Datasets?

@<1539780284646428672:profile|PoisedElephant79> Are you sure you're not simply referring to the get operation? That seems to exclude archived datasets. But I don't see anything like that for the list_datasets operation.

one year ago
0 Hi

That behavior seems strange. In the pipeline in the clearML pagem if you click on one of the steps and select full details (see attached) you can see the commit ID and the branch. Can you validate that the branch is correct but the commit ID is incorrect?
image

10 months ago
0 In My Current Project I Generate The Data From An Sql Query. Is The Only Way To Register The Dataset With Clearml To Write The Files To Disk First Or Is There Another Method? This Leads Into The Second Issue I Have, Which Is What Happens When I Store The

This is odd, the ordering of the files is different and there appears to be some missing from the preview. But as far as I can tell the files aren't different. What am I missing here?
image

one year ago
0 Hope Everyone'S Having A Nice Holiday Period. I'Ve Been Debating Between Cron And The Clearml Taskscheduler Cron Is The Solution I'M Currently Using But I Wanted To Understand The Advantages To Using The Taskscheduler. Right Now I'M Using The Classic Cro

Hi @<1523701205467926528:profile|AgitatedDove14> . I think I'm misunderstanding something here. I have the scheduler service running. Now that it's running how does one add a new task or remove an existing task from the scheduler? I get that I can add them before starting the scheduler service but once the service is running is there any way to connect to it and change the schedule?

I thought the advantage of this service would be we could schedule tasks just by connecting to the existing t...

9 months ago
0 I'M Running Into A Perplexing Issue. I Have Several Agents Running On A Workstation, I Also Am Directly Running Code From The Same Workstation. There Are Several Projects On The Workstation But One Of The Projects Is Struggling With Authentication To Git

Is it possible the cached repository was cloned before you changed your agent settings?

Which settings are you referring to? I can't remember if I was using https auth when the project would have been first cached. Would that make a difference?

Also, did you set

agent.enable_git_ask_pass: true

?

The only instance of it in the config is commented out.

    # if set, use GIT_ASKPASS to pass user/pass when cloning / fetch repositories
    # it solves pas...
8 months ago
0 Hope Everyone'S Having A Nice Holiday Period. I'Ve Been Debating Between Cron And The Clearml Taskscheduler Cron Is The Solution I'M Currently Using But I Wanted To Understand The Advantages To Using The Taskscheduler. Right Now I'M Using The Classic Cro

Oh, I get what's happening. That segment of the code is rerun when the task is enqueued remotely. So it's deleting itself. This also explains why it works fine locally. It's an ouroboros, the task is deleting itself.

8 months ago
0 Hello Everybody, Is It Possible To Download My Python Code From Clearml Server?

This doesn't really make a lot of sense. ClearML would be better served for tracking which version of the code you used for a corresponding task and you'd use something like github or gitlab to track code and host your code. You could use ClearML to help you reconstruct the environment and code from a task given it's being tracked by git and hosted somewhere you can access.

one year ago
0 Hi All, I'Ve Been Experimenting Around With Automating The Data Sync. This Is Related To This Thread:

Interesting approach. I'll give that a try. Thanks for the reply!

one year ago
0 I'M Trying To Spin Up A Task On An Agent And Inside The Task I Have Two Packages That I'Ve Created Custom Versions Of And Specified A Git Repo For In The Requirements.Txt. Example With Hydra-Core And Omegaconf:

Actually this is not how it works, pip will install in any way it sees fit, and it is not consistent between versions (it has to do with dependency resolving)

Oh I see. What a pain. 🤣

You can configure the agent to first install specific packages, and only then others, just add the package names here:

That's an interesting solution. I'll keep that in mind as I work more with ClearML.

Thanks for your help Martin!

one year ago
0 Hope Everyone'S Having A Nice Holiday Period. I'Ve Been Debating Between Cron And The Clearml Taskscheduler Cron Is The Solution I'M Currently Using But I Wanted To Understand The Advantages To Using The Taskscheduler. Right Now I'M Using The Classic Cro

@<1523701205467926528:profile|AgitatedDove14>

And the Task is still running? What's he clearml python version and webui version ?

No, the task stops (it's running remote, I haven't tested it running local).

8 months ago
0 In My Current Project I Generate The Data From An Sql Query. Is The Only Way To Register The Dataset With Clearml To Write The Files To Disk First Or Is There Another Method? This Leads Into The Second Issue I Have, Which Is What Happens When I Store The

I have manually verified that the line-by-line content of the csv files is identical using hashlib.sha256(). Why would it be that the file content is the same, they are generated by the same process (literally just rerunning the same code twice) but ClearML treats them differently.

one year ago
0 In My Current Project I Generate The Data From An Sql Query. Is The Only Way To Register The Dataset With Clearml To Write The Files To Disk First Or Is There Another Method? This Leads Into The Second Issue I Have, Which Is What Happens When I Store The

Alright, I tried testing it out by commenting out the code for generating new csv's, so for successive runs the CSVs are identical. However, when I use dataset.add_files() it still generated a new version of the dataset.

# log the data to ClearML if a task is passed
        if self.task:
            self.clearml_dataset = Dataset.create(dataset_name="[LTV] Dataset")
            self.clearml_dataset.add_files(path=save_path, verbose=True)
            if self.tags is not None:
          ...
one year ago
0 I'M A Bit Confused. It Seems Like Something Has Changed With How Clearml Handles Recording Datasets In Tasks. It Used To Be The Case That When I Would Create A Dataset Under A Task, Clearml Would Record The Id Of The Dataset In The Hyperparameters/Datase

I see. Thanks for the insight. That seems to be the case. I'm struggling a bit with datasets. For example, if I wanted to trace the genealogy of a dataset that's used by traditional tasks and pipelines. I'll try and write something up about the challenges around that when I get the chance. But your comment revealed another issue:

It appears that the partial name matching isn't going well. I'm unclear why this wouldn't be matching. In the attached photo you can see the input for `partial_nam...

one year ago
0 Hi All, Details: Both Projects Are Using Clearml V1.14.2Rc0 (But I'Ve Tested It With Other Versions). I'M Using The Web-App, So We'Re Not Hosting Our Own Cleaml-Server We Do Have A Server With Several Cleaml-Agents V1.7.0 I'M Running Into A Seemingly Co

I'm not sure why the logs were incomplete. I think part of the reason it wasn't pulling from the repo was that it was pulling from cache. I cleared the clearml cache for that project and reran it. This should be the full log.

8 months ago
0 Hello All. I'M Generating An Outputmodel In One Task And Using It As An Inputmodel For Another Task. Since There'S Already A Timestamp On The Model Creation Date, Is There A Way To Get The Date From The Inputmodel?

Thanks Martin. I read this method as "getting the data associated with the model training" not "getting metadata for the model". This is what I'm looking for.

one year ago
Show more results compactanswers