Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Profile picture
ShallowGoldfish8
Moderator
8 Questions, 41 Answers
  Active since 10 January 2023
  Last activity 2 years ago

Reputation

0

Badges 1

33 × Eureka!
0 Votes
2 Answers
2K Views
0 Votes 2 Answers 2K Views
Is there a way to load only selected files and selected columns from a dataset (saved as multiple .parquet files) without having to download all of it?
3 years ago
0 Votes
14 Answers
2K Views
0 Votes 14 Answers 2K Views
Is there any simple way to orchestrate a batch to train a model with different features (in order to do feature selection, for example) through a single .py ...
3 years ago
0 Votes
3 Answers
2K Views
0 Votes 3 Answers 2K Views
3 years ago
0 Votes
5 Answers
2K Views
0 Votes 5 Answers 2K Views
3 years ago
0 Votes
2 Answers
2K Views
0 Votes 2 Answers 2K Views
When trying to run the server from the docker image ( docker-compose -f /opt/clearml/docker-compose.yml up -d as instructed in None ), I am getting an error ...
2 years ago
0 Votes
16 Answers
2K Views
0 Votes 16 Answers 2K Views
Hi guys, I am having some trouble running some training scripts with the agent functionality: https://stackoverflow.com/questions/73279794/catboostclearml-er...
3 years ago
0 Votes
15 Answers
2K Views
0 Votes 15 Answers 2K Views
3 years ago
0 Votes
5 Answers
2K Views
0 Votes 5 Answers 2K Views
Is there a way to upload an artifact I forgot to upload during the task duration to that task after it is already complete?
2 years ago
0 Is There Any Simple Way To Orchestrate A Batch To Train A Model With Different Features (In Order To Do Feature Selection, For Example) Through A Single .Py File? I Saw The Following Example

Looks quite good indeed! Thanks! Is there in the repository the experiment template used in this example? Just not fully sure how the parameters are used/connected in it. Could I just build it and log these parameters using task.set_parameters() so that I call task.get_parameters() later?

3 years ago
0 Hi Guys, I Am Having Some Trouble Running Some Training Scripts With The Agent Functionality:

` from importlib.machinery import EXTENSION_SUFFIXES
import catboost
from clearml import Task, Logger, Dataset

import lightgbm as lgb
import numpy as np
import pandas as pd
import dask.dataframe as dd
import matplotlib.pyplot as plt

MODELS = {
'catboost': {
'model_class': catboost.CatBoostClassifier,
'file_extension': 'cbm'
},
'lgbm': {
'model_class': lgb.LGBMClassifier,
'file_extension': 'txt'
}
}

class ModelTrainer():
def init(sel...

3 years ago
0 Hi Guys, I Am Having Some Trouble Running Some Training Scripts With The Agent Functionality:

Simplified a little bit and removed private parameters, but thats pretty much the code. We did not try with toy examples, since that was already done with the example pipelines when we implemented and the model training itself is quite simple basic there already (only few hyperparameters set)

3 years ago
0 Task Struck At

My code pretty much createas a dataset, uploads it, trains a model (thats where the current task starts), evaluates it and upload all the artifacts and metrics. The artifacts and configurations are upload alright, but the metrics and plots are not. As with Lavi, my code hangs on the task.close(), where it seems to be waiting for the metrics, etc but never finishes. No retry message is shown as well.
After a print I added for debug right before task.close() the only message I get in the consol...

2 years ago
0 Task Struck At

Also, I was using tensorboard

2 years ago
0 Task Struck At

Hi Martin, I updated clearml but the problem persists

2 years ago
0 Hi Guys, I Am Having Some Trouble Running Some Training Scripts With The Agent Functionality:

oooohhh.. you mean the key of the nested dict, that would make a lot of sense

3 years ago
0 Is There Any Simple Way To Orchestrate A Batch To Train A Model With Different Features (In Order To Do Feature Selection, For Example) Through A Single .Py File? I Saw The Following Example

I was checking here, and apparently if I use a parameter as suggested, together with a Task.init(task_name=f'{task name in this loop}') for each of the loops it should work, right? Creating different tasks in the server

3 years ago
0 Hi Guys, I Am Having Some Trouble Running Some Training Scripts With The Agent Functionality:

That would make sense, although clearml, at least on UI, shows the deeper level of the nested dict as a int, as one would expect

3 years ago
0 Is There Any Simple Way To Orchestrate A Batch To Train A Model With Different Features (In Order To Do Feature Selection, For Example) Through A Single .Py File? I Saw The Following Example

yes, but is there a way to generate multiple tasks like I mentioned using task.init in different points of a .py and and run each of them as a separate remote exercution? Didn you just say that once I trigger the task.execute_remotely it will ignore the task.init?

3 years ago
3 years ago
0 Hi Guys, I Am Having Some Trouble Running Some Training Scripts With The Agent Functionality:

Martin, if you want, feel free to add your answer in the stackoverflow so that I can mark it as a solution.

3 years ago
0 Task Struck At

PanickyMoth78 AgitatedDove14 any news on this? I also got a similar issue

2 years ago
0 Hi There, I Am Intending To Work More Often With The Datasets, But Not Sure If There Is A Way To Retrieve Specific Files From A Uploaded Dataset. I Saw I Can Retrieve Chunks Of Data, But Not Sure How That Would Work With A Dataset Of Parquet Files. If I H

Apparently found out a solution:
dataset_zip = dataset._task.artifacts['data'].get() will return the path to the zip file containing all the files (that will be downloaded to the local machine)
after that:
import zipfile zip_file = zipfile.ZipFile(d, 'r') files = zip_file.namelist()retrieving the names of the files
unzip using
import os os.system(f'unzip {dataset_zip}') # in this case to your script directoryand using the files list one can them open them selectively

3 years ago
0 Hi There, I Am Intending To Work More Often With The Datasets, But Not Sure If There Is A Way To Retrieve Specific Files From A Uploaded Dataset. I Saw I Can Retrieve Chunks Of Data, But Not Sure How That Would Work With A Dataset Of Parquet Files. If I H

Could you supply any reference of this dataset containing other datasets? I might have skipped that when reading the documentation, but I do not recall seeing this functionality.

3 years ago
0 Task Struck At

Yes, seems indeed it was waiting for the uploads, which weren't happening ( I did give it quite a while to try to finish the process in my tests). I thought it was a problem with metrics, but apprently it was more like the artifacts before them. The artifacts were shown in the webui dashboard, but were not on S3

2 years ago
0 Task Struck At

` all done
ClearML Monitor: Could not detect iteration reporting, falling back to iterations as seconds-from-start
^CTraceback (most recent call last):
File "/home/zanini/repo/RecSys/src/cli/retraining_script.py", line 710, in <module>
mr.retrain()
File "/home/zanini/repo/RecSys/src/cli/retraining_script.py", line 701, in retrain
self.task.close()
File "/home/zanini/repo/RecSys/.venv/lib/python3.9/site-packages/clearml/task.py", line 1783, in close
self.__shutdown()
File "...

2 years ago
0 Task Struck At

After commenting all the metric/plot reporting, we noticed the model was not uploading the artifacts to S3. A solution was to add wait_for_upload in task.upload_artifact()

2 years ago
0 When Trying To Run The Server From The Docker Image ( `Docker-Compose -F /Opt/Clearml/Docker-Compose.Yml Up -D` As Instructed In

sorted by using command below before docker-compose call
export DOCKER_DEFAULT_PLATFORM=linux/amd64

2 years ago
2 years ago
Show more results compactanswers