Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hello Channel, Two Other Related Questions:

Hello channel,

Two other related questions:

  • ClearML is supposed to automatically detect GIT repo directly. It works when I run a python script but it does not when I run a flask command inside my codebase. Is it an expected behavior? Do you have some workarounds for this?
  • I have a strange behavior ( I am new to ClearML). I setup multiple AWS autoscalers for testing purposes. Each Autoscaler has created workers at some point in the day. Then I deleted those workers, but I can still see them in the Workers and Queues tab, and they still pick tasks for me. I am thus annoyed because I would like to test with new autoscalers parameters and formats, but my previous workers (supposed to be deleted) are still alive... I send a screenshot for better explanation!
    Thanks a lot,
    Have a nice evening all!
    image
  
  
Posted one year ago
Votes Newest

Answers 11


Hi @<1556812486840160256:profile|SuccessfulRaven86>

it does not when I run a flask command inside my codebase. Is it an expected behavior? Do you have some workarounds for this?

Hmm where do you have your Task.init ?
(btw: what's the use case of a flask app tracking?)

Then I deleted those workers,

How did you delete those workers? the autoscaler is supposed to spin the ec2 instances down when they are idle, in theory there is no need for manual spin down.

  
  
Posted one year ago

The flask command is ran inside the git project, which is the strange behavior. It is executed in ~/code/repo/ as flask train ...

  
  
Posted one year ago

@<1556812486840160256:profile|SuccessfulRaven86> is the issue with flask reproducible ? if so could you open a github issue, so we do not forget to look into it?

  
  
Posted one year ago

No problem. I guess this might be a small visualisation bug, but I really have the impression that these workers still pick up tasks, which is strange. I should test again to be sure.

  
  
Posted one year ago

@<1523701205467926528:profile|AgitatedDove14> If you have any other insights, pls do not hesitate! Thanks a lot

  
  
Posted one year ago

try these values:

os.environ.update({
    'CLEARML_VCS_COMMIT_ID': '<commit_id>',
    'CLEARML_VCS_BRANCH': 'origin/master',
    'CLEARML_VCS_DIFF': '',
    'CLEARML_VCS_STATUS': '',
    'CLEARML_VCS_ROOT': '.',
    'CLEARML_VCS_REPO_URL': '
',
})

task = Task.init(...)
  
  
Posted one year ago

I have my Task.init inside a train() function inside the flask command. We basically have flask commands allowing to trigger specific behaviors. When running it locally, everything works properly except the repository information. The use case is linked to the way our codebase works. For example, I am going to do flask train {arguments} and it will trigger the training of a model (that I want to track).

I stopped the autoscaler and deleted it manually. I did it because I want to test and debug multiple configurations of the autoscaler and I cannot modify an autoscaler currently running (maybe because I am not the manager of the workspace).

  
  
Posted one year ago

I tried playing with those, but I do not succeed to have a role on the source code detection. I can modify the env variables, nothing happen on CLearML server unfortunately.

  
  
Posted one year ago

No Task.create is for creating an external Task not logging your own process,
That said you can probably override the git repo with env vars:
None

  
  
Posted one year ago

I will check that. Do you think we could bypass it using Task.create ? And passing all the needed params?

  
  
Posted one year ago

I cannot modify an autoscaler currently running

Yes this is a known limitation, and I know they are working on fixing it for the next version

We basically have flask commands allowing to trigger specific behaviors. ...

Oh I see now, I suspect the issue is that the flask command is not executed from within the git project?!

  
  
Posted one year ago