Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hey All, I'M Having An Issue Using Hydra And Tensorboardx, Where Clearml Isn'T Resetting The Iterations Across Different Multiruns Although It Looks As Expected In Tensorboard Itself:

Hey all, I'm having an issue using hydra and tensorboardX, where clearml isn't resetting the iterations across different multiruns although it looks as expected in tensorboard itself:

  
  
Posted 3 years ago
Votes Newest

Answers 10


HighOtter69
By default if you are continuing an experiment it will start from the last iteration of the previous run. you can reset it with:
task.set_initial_iteration(0)

  
  
Posted 3 years ago

Hi HighOtter69 ,

Can you share the ClearML version you use?

  
  
Posted 3 years ago

I'm using 0.17.4

  
  
Posted 3 years ago

So what's weird is that I'm explicitly closing the task but the separate experiments are not starting back at iteration 0

  
  
Posted 3 years ago

the separate experiments are not starting back at iteration 0

What do you mean by that?

  
  
Posted 3 years ago

pip install clearml==0.17.5rc5

  
  
Posted 3 years ago

When I launch multiple experiments across a parameter grid using hydra multirun, the scalars that were auto connected from tensorflowx act as if they were continuing from the same experiment in serial rather than different experiments. This makes it difficult to compare scalars across different experiments.

  
  
Posted 3 years ago

The newest rc does indeed fix the issue, thank you!

  
  
Posted 3 years ago

I've tried explicitly resetting the iteration account back to 0 across runs using set_initial_iteration with no luck

  
  
Posted 3 years ago

HighOtter69
Could you test with the latest RC? I think this fixed it:
https://github.com/allegroai/clearml/issues/306

  
  
Posted 3 years ago
826 Views
10 Answers
3 years ago
one year ago
Tags