Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Profile picture
GrievingTurkey78
Moderator
34 Questions, 125 Answers
  Active since 10 January 2023
  Last activity 2 years ago

Reputation

0

Badges 1

119 × Eureka!
0 Votes
3 Answers
2K Views
0 Votes 3 Answers 2K Views
Hi! I have some ClearML agents on GCP and sometimes the instance seems to reboot making the experiment fail and all the progress is lost. What is the best wa...
3 years ago
0 Votes
13 Answers
2K Views
0 Votes 13 Answers 2K Views
5 years ago
0 Votes
2 Answers
2K Views
0 Votes 2 Answers 2K Views
Hi! I changed from trains to clearml and ran some experiments using keras but it seems the metrics are not being tracked automagically, has anyone ran into t...
4 years ago
0 Votes
10 Answers
2K Views
0 Votes 10 Answers 2K Views
Hi! I am getting the following error on an agent: /usr/local/bin/python3.8: No module named virtualenv clearml_agent: ERROR: Command '['python3.8', '-m', 'vi...
3 years ago
0 Votes
6 Answers
2K Views
0 Votes 6 Answers 2K Views
Hi! I am saving some intermediate .pt files on the experiments and clearml automatically detects them as models, this makes the clearml.model - INFO message ...
4 years ago
0 Votes
2 Answers
2K Views
0 Votes 2 Answers 2K Views
Hi all! Currently I am trying to create a tool that can perform certain operations on dataset ids, this is a skeleton of what I have in mind (based on the ex...
4 years ago
0 Votes
1 Answers
2K Views
0 Votes 1 Answers 2K Views
Quick question on the clearml-data package, Can I add files to a dataset from google storage instead of having to download them?
4 years ago
0 Votes
7 Answers
2K Views
0 Votes 7 Answers 2K Views
Hi! I am currently using Hydra+ClearML and wanted to know if there are still some updates coming. At the moment, if I change the defaults hydra uses from the...
4 years ago
0 Votes
4 Answers
2K Views
0 Votes 4 Answers 2K Views
Hi, is there a way to force the requirements.txt? I have a package I installed directly from github but the version is always wrong. Any other way to do this?
4 years ago
0 Votes
17 Answers
2K Views
0 Votes 17 Answers 2K Views
4 years ago
0 Votes
3 Answers
2K Views
0 Votes 3 Answers 2K Views
Hi! I recently updated my server and my clearml version, now when I set a task to be executed remotely its default state is aborted hence I have to reset and...
4 years ago
0 Votes
6 Answers
2K Views
0 Votes 6 Answers 2K Views
Hi! I have some agents on GCP. Lately I have been getting some experiments that simply stop running (no signs that the experiment crashed). Here is a plot th...
4 years ago
0 Votes
2 Answers
2K Views
0 Votes 2 Answers 2K Views
I am trying to upgrade from clearml server 0.16 to the newest version but I am getting some errors when spinning up the new containers: WiredTiger error (-31...
4 years ago
0 Votes
30 Answers
2K Views
0 Votes 30 Answers 2K Views
Hi! Is there something happening with the ModelCheckpoint callback on tensorflow==2.4.0 ? Using 2.2.0 gave me an input model on the artifacts tab in the GUI 😢
4 years ago
0 Votes
5 Answers
2K Views
0 Votes 5 Answers 2K Views
Hi! I was taking a look at the https://pytorch-lightning.readthedocs.io/en/latest/common/lightning_cli.html and wanted to know if anyone has used clearml wit...
4 years ago
0 Votes
9 Answers
2K Views
0 Votes 9 Answers 2K Views
Hi! Does ClearML have a way to turn on/off virtual machines depending if there are experiments on queue?
4 years ago
0 Votes
4 Answers
2K Views
0 Votes 4 Answers 2K Views
Hi! If I have a folder with multiple ckpt files would the manual way to upload them be the following: output_model = OutputModel(task) output_model.update_we...
3 years ago
0 Votes
2 Answers
2K Views
0 Votes 2 Answers 2K Views
Hi ! While restarting the server I got ERROR: for agent-services removal of container 8f1d8539340d6d073eb5b51294f5f5d802048a3614d459b5c4fb1d38a05ce538 is alr...
4 years ago
0 Votes
3 Answers
2K Views
0 Votes 3 Answers 2K Views
Hi! I am trying to run some experiments on an agent I have configured to use the requirements.txt the problem is it only shows Cython on the list of installe...
4 years ago
0 Votes
2 Answers
2K Views
0 Votes 2 Answers 2K Views
Hi
Hi AgitatedDove14 ! Regarding the Hydra integration, which pattern should be used? Call the task inside the decorated function? Will this store the parameter...
4 years ago
0 Votes
2 Answers
2K Views
0 Votes 2 Answers 2K Views
Hi! Regarding the artifact.get_local_copy() method, since there is no way to specify the path where the artifact will be downloaded, I wanted to confirm that...
5 years ago
0 Votes
5 Answers
2K Views
0 Votes 5 Answers 2K Views
Hi
Hi 👋 I am logging some figures on pytorch lightning using the example here. The figures are correctly saved on Tensorboard's images tab but unfortunately ar...
3 years ago
0 Votes
7 Answers
2K Views
0 Votes 7 Answers 2K Views
Hi! I am trying to download data from GS using StorageManager.get_local_copy() . It works fine when I point it to a file i.e gs://bucket/dataset/image.png bu...
5 years ago
0 Votes
7 Answers
2K Views
0 Votes 7 Answers 2K Views
Hi! If I have a pipeline on gitlab that uses ClearML for some tests is there some way to setup the credentials so that it doesn’t fail?
4 years ago
0 Votes
21 Answers
2K Views
0 Votes 21 Answers 2K Views
Hi! Any idea why clearml fails to detect iteration reporting? ClearML Monitor: Could not detect iteration reporting, falling back to iterations as seconds-fr...
4 years ago
0 Votes
5 Answers
2K Views
0 Votes 5 Answers 2K Views
Hi, with the upcoming version of Hydra it seems the binding breaks. Specifically in the run_job function the argument order changed from https://github.com/f...
4 years ago
0 Votes
15 Answers
2K Views
0 Votes 15 Answers 2K Views
Hi
Hi 👋 I am trying to set up a trains server on GCP. I followed all the steps listed here https://allegro.ai/docs/deploying_trains/trains_server_gcp/ . I also...
5 years ago
0 Votes
8 Answers
2K Views
0 Votes 8 Answers 2K Views
Hello 👋 I am using a self hosted clearml setup using the requirments file of the project. When I run the task it is failing and I get: Collecting torch==2.0...
2 years ago
0 Votes
10 Answers
2K Views
0 Votes 10 Answers 2K Views
I am also experiencing a weird behaviour when running a script using the module flag. For example I run: python -m module.script arg1 arg 2And after the scri...
5 years ago
0 Votes
2 Answers
2K Views
0 Votes 2 Answers 2K Views
Hi! I have the previous trains server configured with multiple experiments; I created it using the gcloud images provided. If I want to update the server to ...
4 years ago
Show more results questions
5 years ago
0 Hi! Any Idea Why Clearml Fails To Detect Iteration Reporting?

Sure! Could you point me out how its done

4 years ago
0 Hi! Any Idea Why Clearml Fails To Detect Iteration Reporting?

Hey CostlyOstrich36 I am doing a lot of things before the first plot is reported! Is the seconds_from_start parameter unbounded? What should I do if it takes a lot of time to report the first plot?

4 years ago
0 Hi

SuccessfulKoala55 on both 8080 and 8008 I get: Safari can’t open the page http://<External IP>:80XX because Safari can’t establish a secure connection to the server http://<External IP>:80XX .

5 years ago
0 Hi! I Was Taking A Look At The

Yes AgitatedDove14 , I am not sure what they use by default. Here is a simple working example:
` from typing import Optional

import torch
from clearml import Task
from pytorch_lightning import LightningDataModule, LightningModule
from pytorch_lightning.utilities.cli import LightningCLI
from torch.utils.data import DataLoader, Dataset, Subset

class RandomDataset(Dataset):
def init(self, size, length):
self.len = length
self.data = torch.randn(length, size)

def ...
4 years ago
0 Hi! Is There Something Happening With The

If you try:
ModelCheckpoint('best_model.hdf5', save_best_only=True)does it work too?

4 years ago
0 Hi! If I Have A Pipeline On Gitlab That Uses Clearml For Some Tests Is There Some Way To Setup The Credentials So That It Doesn’T Fail?

I need to fetch a dataset for some simple tests but since it doesn’t have credentials to the self-hosted server it wont find the dataset

4 years ago
0 Hello

Managed to get:

clearml_agent: ERROR: Command '['/home/ramon/.clearml/venvs-builds/3.9/bin/python', '-m', 'pip', '--disable-pip-version-check', 'install', '-r', '/var/tmp/requirements_tb0x2i3j.txt', '--extra-index-url', '
 died with <Signals.SIGKILL: 9>.

while building the task with the id on the agent

2 years ago
0 Hi! Is There Something Happening With The

Basically one points to an hdf5 and the other one has no extensiion

4 years ago
0 Hello! There Is Great Alternative For Argparse Developed By Facebook For Ml Named

Just to make sure get everything right AgitatedDove14 :
We have to define the Task inside the function decorated with the @hydra.main We can modify the parameters that are overridden on UI on : configuration tab -> Args -> overrides -> modify the listAdditional question:
Will the sweep functionality work?

5 years ago
0 Hi! Is There Something Happening With The

This works:
filepath = self.log_dir + os.sep + "checkpoint" self.callbacks.append( ModelCheckpoint( filepath, monitor="val_loss", mode="min", save_best_only=True, save_weights_only=True, ) )And this doesn’t:
` filepath = self.log_dir + os.sep + "checkpoint.hdf5"
self.callbacks.append(
ModelCheckpoint(
filepath,
...

4 years ago
0 Hi! Is There A Way To Run A Task Without Reporting To The Server? For Example If I Want To Debug A Script By Running It Locally Without It Appearing On The Server

I feel it’s easier not to report than cleaning after but please correct me if I am overthinking it. I’ll check if I could wrap the code in something that calls the Task.delete if debugging

4 years ago
0 Hi! Is There Something Happening With The

Any idea why this could happen?

4 years ago
0 Hi! Is There Something Happening With The

I changed it to point to a folder and it shows up

4 years ago
0 Hello! There Is Great Alternative For Argparse Developed By Facebook For Ml Named

AgitatedDove14 from this thread I understand hydra is not supported and therefore overriding the parameters from the UI wont work, but is there still a way to track and add the parameters to the experiment? Will task.connect_configuration work with the yaml files?

5 years ago
0 Hi! I Am Getting The Following Error On An Agent:

Give me a couple of minutes 🙌

3 years ago
0 Hi, I Was Getting A Really Weird Error Due To Mismatch On The Versions Between The Installed Libraries In My Environment And The Ones Ran In The Node (I Manually Changed The Installed Packages And Everything Worked). How Can I Force Trains To Use Exactly

TimelyPenguin76 I found out its just one package that is causing the error ( cloudpickle breaks everything). Is there a way to use Pigar but force a single package to have a version?

5 years ago
0 Hi! Is There Something Happening With The

Thanks AgitatedDove14 ! seems to be subclassed model + extension

4 years ago
0 Hi! Any Idea Why Clearml Fails To Detect Iteration Reporting?

Oh I think I am wrong! Then it must be the clearml monitoring. Still it fails way before the timer ends.

4 years ago
0 Hi! I Am Saving Some Intermediate

Hi CostlyOstrich36 ! The message is the following:
clearml.model - INFO - Selected model id: 27c1a1700b0b4e25a4344dc4ef9868faThey are not models, those are intermediate tensors I am caching to make training faster. I don't need to log them.

4 years ago
0 Hi All! Is There A Way For Trains To Recognize The Cli Arguments When Using

Yes AgitatedDove14 , I added git user name and password on the trains.conf file. On the results tab of the UI the logs clone command shows the SSH command instead of the HTTPS :
Repository cloning failed: Command ['clone', mailto:'git@gitlab.com : ...

5 years ago
0 Hi All! Is There A Way For Trains To Recognize The Cli Arguments When Using

Yes, it’s similar; somewhat more automatic since it detects the classes of functions arguments and generates the CLI. What do you mean by that AgitatedDove14 get all the parameters and use task.connect ?

5 years ago
0 Hi All! Is There A Way For Trains To Recognize The Cli Arguments When Using

Yes, exactly! Unfortunately I am not so familiar with the internals of the library but I could take a look and figure that out.

5 years ago
0 Hi All! Is There A Way For Trains To Recognize The Cli Arguments When Using

I am about to try everything AgitatedDove14 but ran into a gitlab error from the agent, I added the username and password to the configuration file but still get a Host key verification failed . Is it common that the cloning message shows the SSH link instead of the HTTPS when username and password are provided?

5 years ago
0 Hi All! Is There A Way For Trains To Recognize The Cli Arguments When Using

Sure, I’ll share It through a private message!

5 years ago
Show more results compactanswers