Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Profile picture
GrievingTurkey78
Moderator
34 Questions, 125 Answers
  Active since 10 January 2023
  Last activity 8 months ago

Reputation

0

Badges 1

119 × Eureka!
0 Votes
12 Answers
569 Views
0 Votes 12 Answers 569 Views
Hi all! Is there a way for trains to recognize the CLI arguments when using https://github.com/google/python-fire instead of argparse?
3 years ago
0 Votes
6 Answers
534 Views
0 Votes 6 Answers 534 Views
Hi! I have some agents on GCP. Lately I have been getting some experiments that simply stop running (no signs that the experiment crashed). Here is a plot th...
2 years ago
0 Votes
4 Answers
570 Views
0 Votes 4 Answers 570 Views
Hi! I am having some problems with a loss after a good amount of training, what would be the best way to log a value to have a better idea of what is happening?
2 years ago
0 Votes
8 Answers
434 Views
0 Votes 8 Answers 434 Views
Hello 👋 I am using a self hosted clearml setup using the requirments file of the project. When I run the task it is failing and I get: Collecting torch==2.0...
8 months ago
0 Votes
3 Answers
568 Views
0 Votes 3 Answers 568 Views
Hi! I recently updated my server and my clearml version, now when I set a task to be executed remotely its default state is aborted hence I have to reset and...
2 years ago
0 Votes
17 Answers
554 Views
0 Votes 17 Answers 554 Views
3 years ago
0 Votes
5 Answers
562 Views
0 Votes 5 Answers 562 Views
Hi, with the upcoming version of Hydra it seems the binding breaks. Specifically in the run_job function the argument order changed from https://github.com/f...
2 years ago
0 Votes
2 Answers
530 Views
0 Votes 2 Answers 530 Views
Hi! I have the previous trains server configured with multiple experiments; I created it using the gcloud images provided. If I want to update the server to ...
3 years ago
0 Votes
1 Answers
553 Views
0 Votes 1 Answers 553 Views
Quick question on the clearml-data package, Can I add files to a dataset from google storage instead of having to download them?
3 years ago
0 Votes
7 Answers
672 Views
0 Votes 7 Answers 672 Views
Hi! If I have a pipeline on gitlab that uses ClearML for some tests is there some way to setup the credentials so that it doesn’t fail?
2 years ago
0 Votes
2 Answers
530 Views
0 Votes 2 Answers 530 Views
Hi! I changed from trains to clearml and ran some experiments using keras but it seems the metrics are not being tracked automagically, has anyone ran into t...
3 years ago
0 Votes
15 Answers
538 Views
0 Votes 15 Answers 538 Views
Hi
Hi 👋 I am trying to set up a trains server on GCP. I followed all the steps listed here https://allegro.ai/docs/deploying_trains/trains_server_gcp/ . I also...
3 years ago
0 Votes
4 Answers
563 Views
0 Votes 4 Answers 563 Views
Hi! If I have a folder with multiple ckpt files would the manual way to upload them be the following: output_model = OutputModel(task) output_model.update_we...
2 years ago
0 Votes
5 Answers
630 Views
0 Votes 5 Answers 630 Views
Hi
Hi 👋 I am logging some figures on pytorch lightning using the example here. The figures are correctly saved on Tensorboard's images tab but unfortunately ar...
2 years ago
0 Votes
9 Answers
640 Views
0 Votes 9 Answers 640 Views
Hi! Does ClearML have a way to turn on/off virtual machines depending if there are experiments on queue?
3 years ago
0 Votes
7 Answers
612 Views
0 Votes 7 Answers 612 Views
Hi! I am currently using Hydra+ClearML and wanted to know if there are still some updates coming. At the moment, if I change the defaults hydra uses from the...
3 years ago
0 Votes
2 Answers
549 Views
0 Votes 2 Answers 549 Views
Hi
Hi AgitatedDove14 ! Regarding the Hydra integration, which pattern should be used? Call the task inside the decorated function? Will this store the parameter...
3 years ago
0 Votes
6 Answers
589 Views
0 Votes 6 Answers 589 Views
Hi! I am saving some intermediate .pt files on the experiments and clearml automatically detects them as models, this makes the clearml.model - INFO message ...
2 years ago
0 Votes
11 Answers
612 Views
0 Votes 11 Answers 612 Views
Hi! Is there a way to run a task without reporting to the server? For example if I want to debug a script by running it locally without it appearing on the s...
2 years ago
0 Votes
13 Answers
530 Views
0 Votes 13 Answers 530 Views
3 years ago
0 Votes
2 Answers
579 Views
0 Votes 2 Answers 579 Views
I am trying to upgrade from clearml server 0.16 to the newest version but I am getting some errors when spinning up the new containers: WiredTiger error (-31...
2 years ago
0 Votes
5 Answers
562 Views
0 Votes 5 Answers 562 Views
Hi! I was taking a look at the https://pytorch-lightning.readthedocs.io/en/latest/common/lightning_cli.html and wanted to know if anyone has used clearml wit...
2 years ago
0 Votes
3 Answers
578 Views
0 Votes 3 Answers 578 Views
Hi! I am trying to run some experiments on an agent I have configured to use the requirements.txt the problem is it only shows Cython on the list of installe...
2 years ago
0 Votes
2 Answers
561 Views
0 Votes 2 Answers 561 Views
Hi! What would be the way for manually uploading a model? I have intermediate .pt files which I don't want to upload. Is there a way to turn off clearml capt...
2 years ago
0 Votes
10 Answers
536 Views
0 Votes 10 Answers 536 Views
I am also experiencing a weird behaviour when running a script using the module flag. For example I run: python -m module.script arg1 arg 2And after the scri...
3 years ago
0 Votes
10 Answers
560 Views
0 Votes 10 Answers 560 Views
Hi! I am getting the following error on an agent: /usr/local/bin/python3.8: No module named virtualenv clearml_agent: ERROR: Command '['python3.8', '-m', 'vi...
2 years ago
0 Votes
7 Answers
565 Views
0 Votes 7 Answers 565 Views
Hi! I am trying to download data from GS using StorageManager.get_local_copy() . It works fine when I point it to a file i.e gs://bucket/dataset/image.png bu...
3 years ago
0 Votes
3 Answers
578 Views
0 Votes 3 Answers 578 Views
Hi! I have some ClearML agents on GCP and sometimes the instance seems to reboot making the experiment fail and all the progress is lost. What is the best wa...
2 years ago
0 Votes
21 Answers
570 Views
0 Votes 21 Answers 570 Views
Hi! Any idea why clearml fails to detect iteration reporting? ClearML Monitor: Could not detect iteration reporting, falling back to iterations as seconds-fr...
2 years ago
0 Votes
4 Answers
666 Views
0 Votes 4 Answers 666 Views
Hi, is there a way to force the requirements.txt? I have a package I installed directly from github but the version is always wrong. Any other way to do this?
2 years ago
Show more results questions
0 Hey Everyone- I Have An Issue Started Today With Trains-Agent Which I’M Getting This Error On Startup:

I am still getting the error even with the v0.16.3 agent, is there something else we have to do other than updating it?

3 years ago
0 Hello! There Is Great Alternative For Argparse Developed By Facebook For Ml Named

AgitatedDove14 from this thread I understand hydra is not supported and therefore overriding the parameters from the UI wont work, but is there still a way to track and add the parameters to the experiment? Will task.connect_configuration work with the yaml files?

3 years ago
0 Hello! There Is Great Alternative For Argparse Developed By Facebook For Ml Named

Just to make sure get everything right AgitatedDove14 :
We have to define the Task inside the function decorated with the @hydra.main We can modify the parameters that are overridden on UI on : configuration tab -> Args -> overrides -> modify the listAdditional question:
Will the sweep functionality work?

3 years ago
3 years ago
0 Hi

The plot is generated and added to tensorboard but seems clearml is not catching it.

2 years ago
0 Hi! If I Have A Pipeline On Gitlab That Uses Clearml For Some Tests Is There Some Way To Setup The Credentials So That It Doesn’T Fail?

I need to fetch a dataset for some simple tests but since it doesn’t have credentials to the self-hosted server it wont find the dataset

2 years ago
0 Hi! Any Idea Why Clearml Fails To Detect Iteration Reporting?

I'll give that a try! Thanks CostlyOstrich36

2 years ago
0 Hi! I Am Trying To Run Some Experiments On An Agent I Have Configured To Use The Requirements.Txt The Problem Is It Only Shows Cython On The List Of Installed Packages. It Crashes Due To Missing Packages.

` [package_manager.force_repo_requirements_txt=true] Skipping requirements, using repository "requirements.txt"
Using base prefix '/opt/conda'
New python executable in /home/ramon/.clearml/venvs-builds/3.7/bin/python3.7
Also creating executable in /home/ramon/.clearml/venvs-builds/3.7/bin/python
Installing setuptools, pip, wheel...
2021-06-10 09:57:56
done.
Collecting pip<20.2
Using cached pip-20.1.1-py2.py3-none-any.whl (1.5 MB)
Installing collected packages: pip
Attempting uninstall: p...

2 years ago
0 Hi! I Am Getting The Following Error On An Agent:

I have the agent configured to force install requirements.txt

2 years ago
0 Hi

Hi AgitatedDove14 thanks for your reply, with the dashboard I meant the Web-App (UI) . I am trying to access http://<External IP>:8080 but unfortunately nothing shows up.

3 years ago
0 Hi

I configured a firewall rule that opened the ports for the instance (not 100% sure if this is the right way) using network tags. Yes, the whole screen is black and no trains logo show up: Safari can’t open the page because the server where this page is located isn’t responding.

3 years ago
0 Hi

I enabled both https and http

3 years ago
0 Hi

SuccessfulKoala55 on both 8080 and 8008 I get: Safari can’t open the page http://<External IP>:80XX because Safari can’t establish a secure connection to the server http://<External IP>:80XX .

3 years ago
0 Hi

SuccessfulKoala55 just to let you know: since I opened the link straight from the GCP console it was using https on the address instead of http hence the error. Thanks a lot for your help!

3 years ago
0 Hi

Also, should I allow 8080 , 8008 , and 8081 on ingress and egress on GCP or is only egress enough?

3 years ago
0 Hi

Thanks SuccessfulKoala55 I’ll give it a try!

3 years ago
0 Hi

I am using the code inside the on_train_epoch_end inside a metric. So the important part is:
` fig = plt.figure()

my plot

logger.experiment.add_figure("fig", fig)
plt.close() `

2 years ago
0 Hi

Let me work on it 👌

2 years ago
0 Hi! Any Idea Why Clearml Fails To Detect Iteration Reporting?

Sure! Could you point me out how its done

2 years ago
0 I Am Also Experiencing A Weird Behaviour When Running A Script Using The Module Flag. For Example I Run:

So should I set them all with a default value? The working dir is the project one, the one that contains the module package

3 years ago
0 I Am Also Experiencing A Weird Behaviour When Running A Script Using The Module Flag. For Example I Run:

Yes, everything is that way (work dir and args are ok) except the script path . It shows -m module arg1 arg2 .

3 years ago
3 years ago
0 Hi All! Is There A Way For Trains To Recognize The Cli Arguments When Using

Yes, it’s similar; somewhat more automatic since it detects the classes of functions arguments and generates the CLI. What do you mean by that AgitatedDove14 get all the parameters and use task.connect ?

3 years ago
0 Hi All! Is There A Way For Trains To Recognize The Cli Arguments When Using

Yes AgitatedDove14 , I added git user name and password on the trains.conf file. On the results tab of the UI the logs clone command shows the SSH command instead of the HTTPS :
Repository cloning failed: Command ['clone', mailto:'git@gitlab.com : ...

3 years ago
0 Hi All! Is There A Way For Trains To Recognize The Cli Arguments When Using

AgitatedDove14 I filed an issue of fire for them to point us to the argument parsing method https://github.com/google/python-fire/issues/291

3 years ago
0 Hi All! Is There A Way For Trains To Recognize The Cli Arguments When Using

Yes, exactly! Unfortunately I am not so familiar with the internals of the library but I could take a look and figure that out.

3 years ago
0 Hi All! Is There A Way For Trains To Recognize The Cli Arguments When Using

Sure, I’ll share It through a private message!

3 years ago
Show more results compactanswers