Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Profile picture
EnthusiasticCow4
Moderator
34 Questions, 129 Answers
  Active since 11 March 2023
  Last activity 2 months ago

Reputation

0

Badges 1

108 × Eureka!
0 Votes
3 Answers
1K Views
0 Votes 3 Answers 1K Views
I ran into something that I'd describe and an error but I want to verify this to be the case first. The error seems to be produced if I call output_model.upd...
one year ago
0 Votes
1 Answers
1K Views
0 Votes 1 Answers 1K Views
one year ago
0 Votes
4 Answers
1K Views
0 Votes 4 Answers 1K Views
one year ago
0 Votes
1 Answers
1K Views
0 Votes 1 Answers 1K Views
For model monitoring, is there a good feature in ClearML? If not is there a go-to tool that works well with this framework?
one year ago
0 Votes
19 Answers
189 Views
0 Votes 19 Answers 189 Views
Hi all 😄 The hyperparameter tuner functionality has just stopped working. When I try and launch an instance of the tuner it gets stuck at "This instance has...
2 months ago
0 Votes
6 Answers
947 Views
0 Votes 6 Answers 947 Views
one year ago
0 Votes
7 Answers
2K Views
0 Votes 7 Answers 2K Views
one year ago
0 Votes
7 Answers
1K Views
0 Votes 7 Answers 1K Views
one year ago
0 Votes
16 Answers
2K Views
0 Votes 16 Answers 2K Views
I'm trying to spin up a task on an agent and inside the task I have two packages that I've created custom versions of and specified a git repo for in the req...
one year ago
0 Votes
7 Answers
1K Views
0 Votes 7 Answers 1K Views
AWS Autoscaler question: what is best practices for providing the autoscaled instances with access to credentials for things like S3? With self-hosted agents...
one year ago
0 Votes
3 Answers
1K Views
0 Votes 3 Answers 1K Views
2 years ago
0 Votes
1 Answers
1K Views
0 Votes 1 Answers 1K Views
I found an interesting error. If I run the following: # Load the model from remote storage if model_url is provided if cfg.clearml.get("model_url", None): # ...
one year ago
0 Votes
2 Answers
734 Views
0 Votes 2 Answers 734 Views
Why is one of the ghosts from Packman in the corner of my workspace folders? 👻
one year ago
0 Votes
12 Answers
934 Views
0 Votes 12 Answers 934 Views
Hello all, I want to clarify something. In the ClearML Task Scheduler .add_tast() method there's a parameter for schedule_function . I think I had some assum...
one year ago
0 Votes
5 Answers
1K Views
0 Votes 5 Answers 1K Views
one year ago
0 Votes
6 Answers
1K Views
0 Votes 6 Answers 1K Views
Hi all, I've been experimenting around with automating the data sync. This is related to this thread: None Here is my expectation, so correct me if I'm wrong...
one year ago
0 Votes
7 Answers
1K Views
0 Votes 7 Answers 1K Views
one year ago
0 Votes
0 Answers
826 Views
0 Votes 0 Answers 826 Views
one year ago
0 Votes
1 Answers
1K Views
0 Votes 1 Answers 1K Views
If you're paying for the premium features would those be available to a self hosted server or only on the web client?
one year ago
0 Votes
29 Answers
1K Views
0 Votes 29 Answers 1K Views
one year ago
0 Votes
6 Answers
1K Views
0 Votes 6 Answers 1K Views
I've noticed a change from ClearML 1.13.1 to 1.13.2 relating to the Hydra integration. I'm not yet convinced it's a bug but it could be but it also might jus...
one year ago
0 Votes
5 Answers
1K Views
0 Votes 5 Answers 1K Views
Is there any way to exclude archived datasets from Dataset.list_datasets()?
one year ago
0 Votes
2 Answers
1K Views
0 Votes 2 Answers 1K Views
one year ago
0 Votes
0 Answers
1K Views
0 Votes 0 Answers 1K Views
Currently, things run in a pipeline are automatically moved to a /.pipelines/ subproject. Is there a way to allow the tasks done via pipelines to show up as ...
one year ago
0 Votes
6 Answers
950 Views
0 Votes 6 Answers 950 Views
one year ago
0 Votes
14 Answers
1K Views
0 Votes 14 Answers 1K Views
one year ago
0 Votes
9 Answers
1K Views
0 Votes 9 Answers 1K Views
Hello all. I'm experimenting with ClearML and I've run into a strange issue. I used Task.init on a project, it logs to the app.clear.ml but it doesn't seem t...
2 years ago
0 Votes
2 Answers
1K Views
0 Votes 2 Answers 1K Views
one year ago
0 Votes
5 Answers
1K Views
0 Votes 5 Answers 1K Views
Hello all, I'm trying to queue a task in python but I'd like to reuse the prior task ID. In the webapp you can Reset and Enqueue a task and it'll reuse the t...
one year ago
0 Votes
3 Answers
391 Views
0 Votes 3 Answers 391 Views
4 months ago
Show more results questions
0 Hello All, I Want To Clarify Something. In The

Sounds good. Lmk if there's some changes that are required.

one year ago
0 I'Ve Noticed A Change From Clearml

@<1523701435869433856:profile|SmugDolphin23> Yes. I'll try it in about 14 hours when I'm back at work and let you know how it goes. 😂

one year ago
0 Hi

That behavior seems strange. In the pipeline in the clearML pagem if you click on one of the steps and select full details (see attached) you can see the commit ID and the branch. Can you validate that the branch is correct but the commit ID is incorrect?
image

one year ago
0 Hey All, I Made A Few Small Changes To

Awesome! Did you managed to solve the tailscale issue with ClearML sessions? Sorry I wasn't active with that. I don't use sessions often and I found a suitable alternative in the short time. Any hopes of the changes making their way to a PR for the official release?

one year ago
0 I'M Running Into A Perplexing Issue. I Have Several Agents Running On A Workstation, I Also Am Directly Running Code From The Same Workstation. There Are Several Projects On The Workstation But One Of The Projects Is Struggling With Authentication To Git
1707128614082 bigbrother:gpu0 INFO task 59d23c5919b04fd6947c1e463fa8c78c pulled from 9890a035b8f84872ab18d7ff207c26c6 by worker bigbrother:gpu0

Current configuration (clearml_agent v1.7.0, location: /tmp/.clearml_agent.vo_oc47r.cfg):
----------------------
agent.worker_id = bigbrother:gpu0
agent.worker_name = bigbrother
agent.force_git_ssh_protocol = true
agent.python_binary = /home/natephysics/anaconda3/bin/python
agent.package_manager.type = pip
agent.package_manager.pip_version.0 = ...
one year ago
one year ago
0 Hello All, I Want To Clarify Something. In The

@<1523701435869433856:profile|SmugDolphin23> Yeah, I just wanted to validate it was worth spending the time. Since there is already a parameter that takes callable (i.e. schedule_function ) it might make sense that we reuse the parameter. If it returns a str we validate that it's a task and if it does we can run the task as if we originally passed it as the task_id in .add_task() . This would only be a breaking change if the callable that was passed happened to return a task_id ...

one year ago
0 I'M Running Into A Perplexing Issue. I Have Several Agents Running On A Workstation, I Also Am Directly Running Code From The Same Workstation. There Are Several Projects On The Workstation But One Of The Projects Is Struggling With Authentication To Git

Is it possible the cached repository was cloned before you changed your agent settings?

Which settings are you referring to? I can't remember if I was using https auth when the project would have been first cached. Would that make a difference?

Also, did you set

agent.enable_git_ask_pass: true

?

The only instance of it in the config is commented out.

    # if set, use GIT_ASKPASS to pass user/pass when cloning / fetch repositories
    # it solves pas...
one year ago
0 I'M Running Into A Perplexing Issue. I Have Several Agents Running On A Workstation, I Also Am Directly Running Code From The Same Workstation. There Are Several Projects On The Workstation But One Of The Projects Is Struggling With Authentication To Git

Results:

I first tried uncommenting enable_git_ask_pass: false but it didn't resolve the issue.

I then cleared the cache in the vcs-cache folder, and that did fix the issue. This is the second time the cache seemed to have been the root cause of the problem. At some point I did move from token-based auth to ssh keys. Would this require clearing the cache for any project that was cached prior to the auth change?

one year ago
0 Hello Again, Is There Any Way To Get A List Of The Datasets From Clearml That Excludes Archived Datasets?

I had 2 datasets on archive and 0 unarchived. When I ran the following command:

Dataset.list_datasets(dataset_project=self.task.get_project_name(), only_completed=True)

It returned two entrees for the two datasets I had on archive.

one year ago
0 Hi All, I'Ve Been Experimenting Around With Automating The Data Sync. This Is Related To This Thread:

Interesting approach. I'll give that a try. Thanks for the reply!

one year ago
0 I'M A Bit Confused. It Seems Like Something Has Changed With How Clearml Handles Recording Datasets In Tasks. It Used To Be The Case That When I Would Create A Dataset Under A Task, Clearml Would Record The Id Of The Dataset In The Hyperparameters/Datase

I see. Thanks for the insight. That seems to be the case. I'm struggling a bit with datasets. For example, if I wanted to trace the genealogy of a dataset that's used by traditional tasks and pipelines. I'll try and write something up about the challenges around that when I get the chance. But your comment revealed another issue:

It appears that the partial name matching isn't going well. I'm unclear why this wouldn't be matching. In the attached photo you can see the input for `partial_nam...

one year ago
0 I'M Trying To Spin Up A Task On An Agent And Inside The Task I Have Two Packages That I'Ve Created Custom Versions Of And Specified A Git Repo For In The Requirements.Txt. Example With Hydra-Core And Omegaconf:

Actually this is not how it works, pip will install in any way it sees fit, and it is not consistent between versions (it has to do with dependency resolving)

Oh I see. What a pain. 🤣

You can configure the agent to first install specific packages, and only then others, just add the package names here:

That's an interesting solution. I'll keep that in mind as I work more with ClearML.

Thanks for your help Martin!

one year ago
one year ago
0 Hope Everyone'S Having A Nice Holiday Period. I'Ve Been Debating Between Cron And The Clearml Taskscheduler Cron Is The Solution I'M Currently Using But I Wanted To Understand The Advantages To Using The Taskscheduler. Right Now I'M Using The Classic Cro

Well, if I stop the cron service and start it back up I don't have to re-register each schedule. If, for instance, I start the TaskScheduler, register a task, and stop the scheduler, how do I restart the TaskScheduler in a way that re-register the tasks? Because, in theory, they could be registered from several users and I might be unaware of tasks that were previously scheduled. What is the best practices to preserve state?

one year ago
0 Hope Everyone'S Having A Nice Holiday Period. I'Ve Been Debating Between Cron And The Clearml Taskscheduler Cron Is The Solution I'M Currently Using But I Wanted To Understand The Advantages To Using The Taskscheduler. Right Now I'M Using The Classic Cro

Hi @<1523701205467926528:profile|AgitatedDove14> . I think I'm misunderstanding something here. I have the scheduler service running. Now that it's running how does one add a new task or remove an existing task from the scheduler? I get that I can add them before starting the scheduler service but once the service is running is there any way to connect to it and change the schedule?

I thought the advantage of this service would be we could schedule tasks just by connecting to the existing t...

one year ago
0 Hey There, I Am A New User Of Clearml And Really Enjoying It So Far! I Noticed That My Model Checkpoints Are Saved After Each Epoch. Instead I Would Like To Only Save The Best And Last Model Checkpoint. Is That Possible? I Could Not Find Something Regardi

Depending on the framework you're using it'll just hook into the save model operation. Every time you save a model, which will probably happen every epoch for some subset of the training. If you want to do it with the existing framework you could change the checkpoint so that it only clones the best model in memory and saves the write operation for last. The risk with this is if the training crashes, you'll lose your best model.

Optionally, you could also disable the ClearML integration with...

2 years ago
0 In My Current Project I Generate The Data From An Sql Query. Is The Only Way To Register The Dataset With Clearml To Write The Files To Disk First Or Is There Another Method? This Leads Into The Second Issue I Have, Which Is What Happens When I Store The

Thanks for the reply @<1523701070390366208:profile|CostlyOstrich36> !

It says in the documentation that:
Add a folder into the current dataset. calculate file hash, and compare against parent, mark files to be uploaded

It seems to recognize the dataset as another version of the data but doesn't seem to be validating the hashes on a per file basis. Also, if you look at the photo, it seems like some of the data does get recognized as the same as the prior data. It seems like it's the correct...

one year ago
0 Hey All, I'Ve Been Doing Some Hyperparameter Tuning Today Using A Clearml Managed Dataset. However, I Kept Running Into The Following Error:

We have a server that has many agents running on it because there are many instances where training can be run over several agents as a single agent doesn't take up all the resources available to the server.

2 months ago
0 In My Current Project I Generate The Data From An Sql Query. Is The Only Way To Register The Dataset With Clearml To Write The Files To Disk First Or Is There Another Method? This Leads Into The Second Issue I Have, Which Is What Happens When I Store The

I have manually verified that the line-by-line content of the csv files is identical using hashlib.sha256(). Why would it be that the file content is the same, they are generated by the same process (literally just rerunning the same code twice) but ClearML treats them differently.

one year ago
0 I'M Trying To Spin Up A Task On An Agent And Inside The Task I Have Two Packages That I'Ve Created Custom Versions Of And Specified A Git Repo For In The Requirements.Txt. Example With Hydra-Core And Omegaconf:

It's even attempting to install omegaconf but not from the repo, likely because it's a dependency of hydra-colorlog.

Collecting omegaconf<2.4,>=2.2
  Using cached omegaconf-2.2.3-py3-none-any.whl (79 kB)
  Using cached omegaconf-2.2.2-py3-none-any.whl (79 kB)
  Using cached omegaconf-2.2.1-py3-none-any.whl (78 kB)
one year ago
Show more results compactanswers