Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Profile picture
ZanySealion18
Moderator
6 Questions, 23 Answers
  Active since 03 November 2023
  Last activity 4 months ago

Reputation

0

Badges 1

22 × Eureka!
0 Votes
11 Answers
490 Views
0 Votes 11 Answers 490 Views
ClearML doesn't pick up model checkpoints automatically. Any idea what might be wrong? (code attached in the thread). Thanks
6 months ago
0 Votes
3 Answers
217 Views
0 Votes 3 Answers 217 Views
4 months ago
0 Votes
6 Answers
308 Views
0 Votes 6 Answers 308 Views
My agent (running on GCP in docker mode) is having trouble with git fetch --all. I'm using SSH for authentication, however, known_hosts doesn't seem to be pa...
5 months ago
0 Votes
11 Answers
292 Views
0 Votes 11 Answers 292 Views
I've been using ClearML on on-premise machines and would now like to deploy everything in GCP (deploy everything new from scratch, don't care about migrating...
5 months ago
0 Votes
8 Answers
280 Views
0 Votes 8 Answers 280 Views
5 months ago
0 Votes
0 Answers
295 Views
0 Votes 0 Answers 295 Views
Hey folks, could anyone please take a look at this thread? Thanks! None
6 months ago
0 Clearml Doesn'T Pick Up Model Checkpoints Automatically. Any Idea What Might Be Wrong? (Code Attached In The Thread). Thanks

clearml-1.13.1

Task.add_requirements("requirements.txt")
task = Task.init(project_name="My project", task_name="My task")
task.execute_remotely(queue_name="default")
...
6 months ago
0 Clearml Doesn'T Pick Up Model Checkpoints Automatically. Any Idea What Might Be Wrong? (Code Attached In The Thread). Thanks

@<1523701087100473344:profile|SuccessfulKoala55> Kind reminder again, thanks and sorry!

6 months ago
0 Clearml Doesn'T Pick Up Model Checkpoints Automatically. Any Idea What Might Be Wrong? (Code Attached In The Thread). Thanks

No worries, sorry for pinging, was just making sure you (or anyone else who might help) doesn't miss it 🙂
I use Task.add_requirements("requirements.txt") right before the Task.init.
In main, I parse arguments command-line, add_requirements, initialize Task and call execute_remotely. After that it's all pretty much the usual workflow. Initialize the model, setup dataloaders, optimizer and run the training. I'm using pytorch-ignite and have model checkpoint made on validation evaluator COMPL...

6 months ago
0 I'Ve Been Using Clearml On On-Premise Machines And Would Now Like To Deploy Everything In Gcp (Deploy Everything New From Scratch, Don'T Care About Migrating The Data). I'Ve Followed Tutorial

Probably not, I'm trying to access it via external IP. Could you point me to instructions for that in the docs, I don't remember seeing it anywhere? Thanks!

5 months ago
0 My Agent (Running On Gcp In Docker Mode) Is Having Trouble With Git Fetch --All. I'M Using Ssh For Authentication, However, Known_Hosts Doesn'T Seem To Be Passed To The Docker So It Prompts For Authentification/Fingerprint. Any Ideas?

Tried but it didn't help. I suspect the issue is here: "'docker', 'run', '-t', '--gpus', '"device=0"', '-v', '/tmp/ssh-krPvUxRks5/agent.1949:/tmp/ssh-krPvUxRks5/agent.1949', '-e', 'SSH_AUTH_SOCK=/tmp/ssh-krPvUxRks5/agent.1949'"

It passes SSH socket instead of .ssh directory (not sure why, an agent I have running on my own machine behaves differently)? Do you happen to know how to fix this? Thanks!

5 months ago
0 I Have The Following Setup: Instance-1 With Clearml Server Hosted In Gcp, Visible Only Through Internal Network, Instance-2 With Clearml Agent Hosted In Gcp, Visible Only Through Internal Network, Reverse-Ngnix-Proxy Hosted In Gcp Acting As Interface Betw

Additional info:
-Public URL uses HTTPS, internal traffic doesn't.
-clearml.storage fails while trying to fetch None ...

Meaning it just replaced the internal IP with the URL at some point for some reason, it doesn't exist in that form anywhere in any configs (http and public URL).

5 months ago
0 I Have The Following Setup: Instance-1 With Clearml Server Hosted In Gcp, Visible Only Through Internal Network, Instance-2 With Clearml Agent Hosted In Gcp, Visible Only Through Internal Network, Reverse-Ngnix-Proxy Hosted In Gcp Acting As Interface Betw

I hacked around the solution by setting api.files_server for the agent to the public URL, but ideally I'd avoid going through reverse-proxy if there's some path_substitution equivalent for this. Thanks

4 months ago
0 Clearml Doesn'T Pick Up Model Checkpoints Automatically. Any Idea What Might Be Wrong? (Code Attached In The Thread). Thanks

@<1523701087100473344:profile|SuccessfulKoala55> kind reminder not to miss this when you catch time, thanks!

6 months ago
0 I'Ve Been Using Clearml On On-Premise Machines And Would Now Like To Deploy Everything In Gcp (Deploy Everything New From Scratch, Don'T Care About Migrating The Data). I'Ve Followed Tutorial

To make sure I understand, I need to setup a domain with a cert and it should work, no additional ClearML config is required?

5 months ago
0 Clearml Doesn'T Pick Up Model Checkpoints Automatically. Any Idea What Might Be Wrong? (Code Attached In The Thread). Thanks

Oh, I misunderstood then docs/examples, sorry. I'm using pytorch-ignite.

Thanks for the tip!

6 months ago
0 I'Ve Been Using Clearml On On-Premise Machines And Would Now Like To Deploy Everything In Gcp (Deploy Everything New From Scratch, Don'T Care About Migrating The Data). I'Ve Followed Tutorial

Having a bit of trouble with this one (sorry for possibly dumb questions).
Are there any docs on how to add certs to the docker image? I see this ( None ) which is where letsencrypt points me to, but I'm not sure what's the proper way to do this on the webapp docker (I'd assume there's a non-hacky way to do it as others are using the same setup I'm trying to make work I guess)

5 months ago
0 I Have The Following Setup: Instance-1 With Clearml Server Hosted In Gcp, Visible Only Through Internal Network, Instance-2 With Clearml Agent Hosted In Gcp, Visible Only Through Internal Network, Reverse-Ngnix-Proxy Hosted In Gcp Acting As Interface Betw

One more related question (I hope there's a similar solution), when I log images, they appear in the UI with http://<my-ip> so they are inaccessible (they should be translated to None . Is there any path_substitution variant for this scenario in the config? I can't seem to find it in the docs. Thanks!

5 months ago
0 Clearml Doesn'T Pick Up Model Checkpoints Automatically. Any Idea What Might Be Wrong? (Code Attached In The Thread). Thanks
    model_checkpoint = ModelCheckpoint(
        "checkpoint",
        n_saved=2,
        filename_prefix="best",
        score_function=score_function,
        score_name="accuracy",
        global_step_transform=global_step_from_engine(trainer),
    )

    # Save the model after every epoch of val_evaluator is completed
    val_evaluator.add_event_handler(
        Events.COMPLETED, model_checkpoint, {"model": model}
    )
6 months ago