Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Profile picture
VexedCat68
Moderator
60 Questions, 381 Answers
  Active since 10 January 2023
  Last activity 7 months ago

Reputation

0

Badges 1

371 × Eureka!
0 Votes
1 Answers
1K Views
0 Votes 1 Answers 1K Views
In the configuration section of an experiment, I can see the args I passed via argparse. Is there any way to create other sections other than args and tf_define
3 years ago
0 Votes
3 Answers
1K Views
0 Votes 3 Answers 1K Views
2 years ago
0 Votes
2 Answers
1K Views
0 Votes 2 Answers 1K Views
Given this pipeline step, is there any way to get the return value outside of the pipeline? Like put split_dataset_id in a variable in the main pipeline file.
2 years ago
0 Votes
13 Answers
1K Views
0 Votes 13 Answers 1K Views
Another simple query guys. I've installed clearml on ubuntu. However, it says command not found when i run any command with clearml . I feel like it might be...
3 years ago
0 Votes
15 Answers
1K Views
0 Votes 15 Answers 1K Views
I'm publishing the model artifact after the tensorflow model is saved. But in the UI, the download button is grayed out. And get_local_copy() doesn't seem to...
2 years ago
0 Votes
12 Answers
1K Views
0 Votes 12 Answers 1K Views
I'm training a tensorflow model and saving it in the end. I looked at the OutputModel class. How do I connect the model I'm saving to the OutputModel?
2 years ago
0 Votes
19 Answers
1K Views
0 Votes 19 Answers 1K Views
Is this the write way to add a tag to an output model artifact of a task? torch.save(model, ' http://best.pt ') output_model = task.models['output'][-1] outp...
2 years ago
0 Votes
7 Answers
926 Views
0 Votes 7 Answers 926 Views
Hi guys, needed a bit of clarification. Every 15 minutes, the scheduler prints, "Syncing scheduler, sleeping for 15 minutes until next sync". Can you guide m...
2 years ago
0 Votes
25 Answers
944 Views
0 Votes 25 Answers 944 Views
Um, is there a way to delete an artifact from a task that is running?
2 years ago
0 Votes
10 Answers
1K Views
0 Votes 10 Answers 1K Views
2 years ago
0 Votes
13 Answers
1K Views
0 Votes 13 Answers 1K Views
3 years ago
0 Votes
7 Answers
1K Views
0 Votes 7 Answers 1K Views
When i create clearml-dataset from the cli, I get an id. The same doesn't happen when I use the Python API. is there any way to get the ID in python?
3 years ago
0 Votes
5 Answers
1K Views
0 Votes 5 Answers 1K Views
3 years ago
0 Votes
29 Answers
866 Views
0 Votes 29 Answers 866 Views
2 years ago
0 Votes
13 Answers
958 Views
0 Votes 13 Answers 958 Views
3 years ago
0 Votes
16 Answers
1K Views
0 Votes 16 Answers 1K Views
2 years ago
0 Votes
12 Answers
1K Views
0 Votes 12 Answers 1K Views
I'm on the machine with ClearML Server hosted. Is there any way to see datasets uploaded to ClearML Data without downloading them using ClearML Data?
3 years ago
0 Votes
5 Answers
916 Views
0 Votes 5 Answers 916 Views
Is it possible to add just a string or some other object as an artifact? if yes, then how?
2 years ago
0 Votes
10 Answers
1K Views
0 Votes 10 Answers 1K Views
3 years ago
0 Votes
2 Answers
997 Views
0 Votes 2 Answers 997 Views
Is there a ClearML way to write a scheduler to watch a folder and publish a dataset when there are X number of files in that folder, or do I have to write a ...
3 years ago
0 Votes
3 Answers
1K Views
0 Votes 3 Answers 1K Views
In the case where I'm passing a schedule_fn to add_task in a TaskScheduler, how do I pass the function arguments?
3 years ago
0 Votes
10 Answers
1K Views
0 Votes 10 Answers 1K Views
3 years ago
0 Votes
27 Answers
1K Views
0 Votes 27 Answers 1K Views
I wanted to ask, how to run pipeline steps conditionally? E.g if step returns a specific value, exit the pipeline or run another step instead of the sequenti...
2 years ago
0 Votes
7 Answers
1K Views
0 Votes 7 Answers 1K Views
2 years ago
0 Votes
14 Answers
1K Views
0 Votes 14 Answers 1K Views
2 years ago
0 Votes
3 Answers
955 Views
0 Votes 3 Answers 955 Views
In your docs for Dataset at https://clear.ml/docs/latest/docs/references/sdk/dataset#class-dataset , I think you might have duplicate explanations for list_m...
2 years ago
0 Votes
9 Answers
974 Views
0 Votes 9 Answers 974 Views
Trying to create a data pipeline on my own. Wanted to ask, for each batch of data, do I have to create a new Dataset Object or do I just create one Dataset O...
3 years ago
0 Votes
12 Answers
914 Views
0 Votes 12 Answers 914 Views
Is there a quicker way to abort all running experiments in a project? I have over a thousand running anonymous data tasks in a specific project and I want to...
2 years ago
0 Votes
5 Answers
982 Views
0 Votes 5 Answers 982 Views
3 years ago
0 Votes
5 Answers
1K Views
0 Votes 5 Answers 1K Views
2 years ago
Show more results questions
0 When It Comes To Continuous Training, I Wanted To Know How You Train Or Would Train If You Have Annotated Data Incoming? Do You Train Completely Online Where You Train As Soon As You Have A Training Example Available? Do You Instead Train When You Have A

AgitatedDove14 Sorry for pinging you on this old thread. I had an additional query. If you've worked on a process similar to the one mentioned above, how do you set the learning rate? And what was the learning strategy? ADAM? RMSProp?

2 years ago
0 When It Comes To Continuous Training, I Wanted To Know How You Train Or Would Train If You Have Annotated Data Incoming? Do You Train Completely Online Where You Train As Soon As You Have A Training Example Available? Do You Instead Train When You Have A

With online learning, my two main concerns are that the training would be completely stochastic in nature, I would not be able to split the data into train test splits, and that it would be very expensive and inefficient to train online.

3 years ago
0 When It Comes To Continuous Training, I Wanted To Know How You Train Or Would Train If You Have Annotated Data Incoming? Do You Train Completely Online Where You Train As Soon As You Have A Training Example Available? Do You Instead Train When You Have A

I get what you're saying. I was considering training on just the new data to see how it works. To me it felt like that was the fastest way to deal with data drift. I understand that it may introduce instability however. I was curious how other developers who have successfully managed to set up continuous training deal with it. 100% new data, or a ratio between new and old data. And if it is the latter, what should be the case, which should be the majority, old data or new data?

3 years ago
0 When It Comes To Continuous Training, I Wanted To Know How You Train Or Would Train If You Have Annotated Data Incoming? Do You Train Completely Online Where You Train As Soon As You Have A Training Example Available? Do You Instead Train When You Have A

Understandable. I mainly have regular image data, not video sequences so I can do the train test splits like you mentioned normally. What about the epochs though? Is there a recommended number of epochs when you train on that new batch?

3 years ago
0 When It Comes To Continuous Training, I Wanted To Know How You Train Or Would Train If You Have Annotated Data Incoming? Do You Train Completely Online Where You Train As Soon As You Have A Training Example Available? Do You Instead Train When You Have A

Would you know what the pros would be to learning online other than the fact that the incoming data is as close to the current distribution of data based on time as possible for us. Also would those benefits worth it to train online?

3 years ago
0 When It Comes To Continuous Training, I Wanted To Know How You Train Or Would Train If You Have Annotated Data Incoming? Do You Train Completely Online Where You Train As Soon As You Have A Training Example Available? Do You Instead Train When You Have A

Lastly, I have asked this question multiple times, but since the MLOps process is so new, I want to learn from others experience regarding evaluation strategies. What would be a good evaluation strategy? Splitting the batch into train test? that would mean less data for training but we can test it asap. Another idea I had was training on the current batch, then evaluating it on incoming batches. Any other ideas?

3 years ago
0 When Using Dataset.Get_Local_Copy(), Once I Get The Location, Can I Add Another Folder Inside Location Add Some Files In It, Create A New Dataset Object, And Then Do Dataset.Upload(Location)? Should This Work? Or Since Its Get_Local_Copy, I Won'T Be Able

Sorry for the late response. Agreed, that can work, although I would prefer a way to access the data by M number of batches added instead of a certain range, since these cases aren't interchangeable. Also a simple thing that can be done is that you can create an empty Dataset in the start, and then make it the parent of every dataset you add.

2 years ago
0 Hey Guys, Sorry For The Rapid Fire Questions In The Past Few Days. I Have Another Issue Though. I Initially Ran A Task, Directly From A Repo. It Succesfully Installed The Requirements From The Requirements File In The Repo And Ran The Task Without Any Iss

Another issue I'm having is I ran a task using clearml-task and did it using a repo. It runs fine, when I clone said task however and run it on the same queue again, it throws an error from the code. I can't seem to figure out why its happening.

2 years ago
0 Um, Is There A Way To Delete An Artifact From A Task That Is Running?

I plan to append the checkpoint to a list, when the len(list) > N, I'll just pop out the one with the highest loss, and delete that file from clearml and storage. That's how I plan to work with it.

2 years ago
0 Um, Is There A Way To Delete An Artifact From A Task That Is Running?

I ran a training code from a github repo. It saves checkpoints every 2000 iterations. Only problem is I'm training it for 3200 epochs and there's more than 37000 iterations in each epoch. So the checkpoints just added up. I've stopped the training for now. I need to delete all of those checkpoints before I start training again.

2 years ago
0 Once I Set Up A Clear Ml Server On A Machine. I Understand I Need An Agent Listening On A Queue To Run Code. Do I Absolutely Have To Have An Agent And The Gpu On The Same Machine As The Server? Also Do The Gpu And Agent Have To Be On The Same Machine?

Basically since I want to train AI Models right. I'm trying to set up the architecture where I can automate the process from data fetching to model training, and need GPU for training.

3 years ago
0 When Using Dataset.Get_Local_Copy(), Once I Get The Location, Can I Add Another Folder Inside Location Add Some Files In It, Create A New Dataset Object, And Then Do Dataset.Upload(Location)? Should This Work? Or Since Its Get_Local_Copy, I Won'T Be Able

Well I'm still researching how it'll work. I'm expecting it to not be very good and will make the model learning very stochastic in nature.

I plan to instead at the training stage, instead of just getting this model, use Dataset.squash, to get previous M datasets merged together.

This should introduce stability in the dataset.

Also this way, our model is trained on a batch of data multiple times but only for a few times before that batch is discarded. We keep the training data fresh for co...

2 years ago
Show more results compactanswers