Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hello, I Have The Same Issue As This Github Issue:

Hello,

I have the same issue as this github issue: None

I tried setting up my AWS autoscaler conf file with the following params:

sdk.development.store_uncommitted_code_diff = true
sdk.development.store_code_diff_from_remote = true
sdk.development.store_uncommitted_code_diff_on_train = true

I also tried with all to False.

However, nothing changes, and my code still fails with the following error: ERROR! Failed applying git diff, see diff above.

Do you have any insights? Thank you.

  
  
Posted one year ago
Votes Newest

Answers 5


Sure, here is the updated clearml.conf file of the AWS autoscaler instance:

agent {
    vcs_cache.enabled: false

    package_manager: {
          type: poetry,
          poetry_version: "1.4.2",
     }  
}

sdk {
    development {
         store_code_diff_from_remote: false,
    }

}

I see uncommited changes, where as I would like to have nothing.

  
  
Posted one year ago

These changes reflect the modifications I have in my staging area (not commited, not put in staging area with git add ) But I would like to remove this uncommited section from clearml and not be blocked by it

  
  
Posted one year ago

If I may ask as well for another issue in that thread that is taking me a big amount of time:

Poetry Enabled: Ignoring requested python packages, using repository poetry lock file!
Creating virtualenv alfred-Rp77Shgw-py3.9 in /root/.cache/pypoetry/virtualenvs
Installing dependencies from lock file
2023-04-17 10:17:57
Package operations: 351 installs, 1 update, 1 removal
failed installing poetry requirements: Command '['poetry', 'install', '-n']' returned non-zero exit status 1.
Ignoring pip: markers 'python_version >= "3.10"' don't match your environment
Collecting pip<20.2
  Using cached pip-20.1.1-py2.py3-none-any.whl (1.5 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 23.0.1
    Uninstalling pip-23.0.1:
      Successfully uninstalled pip-23.0.1
Successfully installed pip-20.1.1

I am supposed to use poetry package manager, and the snippet above proves it. However, it directly fails and uses pip manager. Do you have any ideas why? I tried adding -vvv flag but I have no more logs about the issue. Very surprisely, when I run the same code on a google colab clearm agent with same configuration, I do not have this poetry fail

I am spending ages on this issue, but until now, no fixes have been found 😞
I was telling myself that I could switch to Pip (which is the default behavior when poetry fails), but not all packages are correclty retrieved from clearml, so it cannot work.

  
  
Posted one year ago

Ok. I spinned up three AWS autoscalers, each with different conf. I also fixed a submodule issue in my repo (which I was believing was the problem of the git diff) and every run now passes and fails after (not this problem). So I think store_code_diff_from_remote is of no help from me but my problem is gone...

  
  
Posted one year ago

Hi @<1556812486840160256:profile|SuccessfulRaven86> , just to make things easier, can you comment out these 3 lines in the config file? This will cause the sdk to have default behavior. Afterwards try with store_code_diff_from_remote: false

What do you see in the uncommitted changes section of the experiment?

  
  
Posted one year ago
822 Views
5 Answers
one year ago
one year ago
Tags