Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Profile picture
PricklyRaven28
Moderator
18 Questions, 109 Answers
  Active since 10 January 2023
  Last activity 3 months ago

Reputation

0

Badges 1

107 × Eureka!
0 Votes
6 Answers
1K Views
0 Votes 6 Answers 1K Views
Hey
Hey 🙂 Working with ClearML pipelines, is there a way to change default directory artifacts to archive with tar instead of zip ? Speaking of the default retu...
2 years ago
0 Votes
2 Answers
1K Views
0 Votes 2 Answers 1K Views
Hey
Hey 🙂 I'm using pipelines with decorators and trying to set a custom docker image to the pipeline itself which doesn't seem to accept a "docker=" argument l...
one year ago
0 Votes
0 Answers
893 Views
0 Votes 0 Answers 893 Views
Hi 🙂 we have a clearml pipeline that has a step that runs a multi gpu training (with hugginface), we need to invoke it with accelerate launch so we use subp...
11 months ago
0 Votes
8 Answers
1K Views
0 Votes 8 Answers 1K Views
There is a problem starting from clearml 1.7.0 with python-fire from clearml import Task import fire def check(first): print(first) if __name__ == '__main__'...
2 years ago
0 Votes
4 Answers
322 Views
0 Votes 4 Answers 322 Views
Hi
Hi 🙂 I'm working on slack alerts based on the open source example, we added slack mentions like this def get_username_tag(self, task:Task): res = Task._get_...
6 months ago
0 Votes
18 Answers
1K Views
0 Votes 18 Answers 1K Views
https://clearml.slack.com/archives/CTK20V944/p1713357955958089 Any idea about this?
11 months ago
0 Votes
12 Answers
1K Views
0 Votes 12 Answers 1K Views
Hey, Is there a way to temporarily turn off clearml logging? I’m using pipeline and when developing i don’t wand them to be added to the UI and spam it. in w...
2 years ago
0 Votes
30 Answers
1K Views
0 Votes 30 Answers 1K Views
Hey everyone 🙂 I’m trying to use a ClearML on prem for experiment visualization only, having some issues with multi GPU. It seems that clearml is creating a...
3 years ago
0 Votes
2 Answers
1K Views
0 Votes 2 Answers 1K Views
Hi
Hi 🙂 I'm trying to figure out if i have a way to report pipeline-step artifact paths in the main pipeline task. (So i don't need to dig into steps to find t...
2 years ago
0 Votes
30 Answers
1K Views
0 Votes 30 Answers 1K Views
If possible, i would like all together prevent the fileserver and write everything to S3 (without needing every user to change their config)
2 years ago
0 Votes
1 Answers
1K Views
0 Votes 1 Answers 1K Views
Also it’s weird to me that when running pipelines in debugging_pipeline that it will even show in the UI
2 years ago
0 Votes
7 Answers
1K Views
0 Votes 7 Answers 1K Views
i think there is some bug with clearml==1.7.1. I’m working with pipelines and after updating to 1.7.1 all the kwargs are passed as None to the steps, when do...
2 years ago
0 Votes
4 Answers
358 Views
0 Votes 4 Answers 358 Views
Hi
Hi 🙂 I have an AWS autoscaler (from script) running with EC2 instances, is there a way to configure a queue for a multi-gpu instance where each task gets 1 ...
6 months ago
0 Votes
2 Answers
370 Views
0 Votes 2 Answers 370 Views
Hi
Hi 🙂 I have a git folder that i'm running with a local agent in services mode (for slack alerts), but i don't want the agent to clone the repo (for security...
5 months ago
0 Votes
0 Answers
1K Views
0 Votes 0 Answers 1K Views
Hey 🙂 I’m also using the AWS autoscaler with pipelines, and i can’t seem to make the intermediate steps output their results to S3 instead of the fileserver...
2 years ago
0 Votes
1 Answers
733 Views
0 Votes 1 Answers 733 Views
Hi
Hi 🙂 How can i disable requirements install when an agent is starting a task? we use docker, so everything should already be installed
11 months ago
0 Votes
62 Answers
85K Views
0 Votes 62 Answers 85K Views
Hey, We are using clearml 1.9.0 with transformers 4.25.1… and we started getting errors that do not reproduce in earlier versions (only works in 1.7.2 all 1....
2 years ago
0 Votes
0 Answers
1K Views
0 Votes 0 Answers 1K Views
(using a local clearml server)
2 years ago
0 Hey, We Are Using Clearml 1.9.0 With Transformers 4.25.1… And We Started Getting Errors That Do Not Reproduce In Earlier Versions (Only Works In 1.7.2 All 1.8.X Don’T Work):

` from clearml.automation import PipelineDecorator
from clearml import TaskTypes

@PipelineDecorator.component(task_type=TaskTypes.data_processing, cache=True)
def run_demo():
from transformers import AutoTokenizer, DataCollatorForTokenClassification, AutoModelForTokenClassification, TrainingArguments, Trainer
from datasets import load_dataset

dataset = load_dataset("conllpp")

model_checkpoint = 'bert-base-cased'
lr = 2e-5
num_train_epochs  = 5
weight_decay =...
2 years ago
0 Hey

Hi, yes it's running with autoscaler so it's for sure in docker mode

Are you saying that it should've worked? I got 'docker' attribute doesn't exist error. Maybe it's the version of the clearml server?

one year ago
0 Hey, We Are Using Clearml 1.9.0 With Transformers 4.25.1… And We Started Getting Errors That Do Not Reproduce In Earlier Versions (Only Works In 1.7.2 All 1.8.X Don’T Work):

Yes, and the old version only works without the patch.
I see the model on the artifacts tab, but it's not actually uploaded.

2 years ago
0 Hey Everyone

any idea? 🙏

3 years ago
0 Hey Everyone

they also appear to be relying on the tensorboard callback which seems not to work on distributed training

3 years ago
0 Hey Everyone

maybe the fastai bindings in clearml package are outdated

3 years ago
0 Hey Everyone

when u say use Task.current_task() you for logging? which i’m guessing that the fastai binding should do right?

3 years ago
0 Hey Everyone

and use that instead of the -m fastai.launch part ofcourse

3 years ago
0 Hi

@<1523701205467926528:profile|AgitatedDove14> i probably missed it, I'll try tomorrow thanks!

2 years ago
0 Hey

It's models not datasets in our case...

But we can also just tar the folder and return that... Was just hoping to avoid doing that

2 years ago
0 Hey Everyone

reduced to a small snippet
` from fastai.vision.all import *
from fastai.distributed import *
from clearml import Task
from fastai.callback.tensorboard import TensorBoardCallback
from wwf.vision.timm import timm_learner

task = Task.init(project_name='LIOR_TEST', auto_connect_arg_parser={'rank': False})
path = untar_data(URLs.PETS)

size = 460
batch_size = 32

dblock = DataBlock(blocks=(ImageBlock, CategoryBlock),
get_items=get_image_files,
get_y=lambda x: ...

3 years ago
0 Hey Everyone

i’ll check if it’s already initialized

3 years ago
0 Hey, Is There A Way To Temporarily Turn Off Clearml Logging? I’M Using Pipeline And When Developing I Don’T Wand Them To Be Added To The Ui And Spam It. In Wandb There Is A Way To Do

Yes, but it’s more complex because i’m using a pipeline… where i don’t explicitly call Task.init()

2 years ago
0 I Think There Is Some Bug With Clearml==1.7.1. I’M Working With Pipelines And After Updating To

SmugSnake6 yep, that’s exactly it.

Hope the team is aware and will fix it

2 years ago
0 Hey Everyone

Noting one difference i do is using TensorBoardCallback , because i believe the clearml docs use an outdated fastai 1 version…

3 years ago
0 Https://Clearml.Slack.Com/Archives/Ctk20V944/P1713357955958089

Glad to hear you were able to reproduce it! Waiting for your reply 🙏

10 months ago
0 Hi

The task is given by the monitor(like in the example) the problem is the user

6 months ago
0 Hi

what i'm doing is getting

parent = Task.get_task(task.parent)

and then checking
parent.data.user

but the user is some unknown id that doesn't exist in the all_users list

6 months ago
0 Hi

@<1523701070390366208:profile|CostlyOstrich36>

Sorry for the (very) late response.

We use the open source version which isn't part of the ClearML setup.

Anyway, we are using a standalone script but we have it source controlled in git... clearml picks this up and tries to clone the entire repo in the agent. i want to prevent this an just use the script.

3 months ago
0 Https://Clearml.Slack.Com/Archives/Ctk20V944/P1713357955958089

to make it very reproducible, i created a docker file for it, so make sure to run build_docker.sh and then run.sh

10 months ago
Show more results compactanswers