Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hello There, I Am Trying To Organize The Dl Code Into A Monorepo, The Repo Will Have A Section Of Shared Packages That Will Be Used By Other Packages That Are The Actual Training Projects. Let'S Say That I Install The Shared Libs With Pip In Editable Mod

Hello there,

I am trying to organize the DL code into a monorepo, the repo will have a section of shared packages that will be used by other packages that are the actual training projects. Let's say that I install the shared libs with pip in editable mode on my development evironment, how does the clearml-agent will handle those libraries if I submit a job? Will it detect that the libs are in the same repository and install then? What would be the best approach?

  
  
Posted one year ago
Votes Newest

Answers 5


Then as you suggested, I would just use sys.path it is probably the easiest and actually very safe (because the subfolders are Always next to the "main" source code)

  
  
Posted one year ago

They are on the same repo on git, something like:
my-repo train project1 project2 libs lib1 ...

  
  
Posted one year ago

But I am starting to wonder whether It would be easier just changing sys,path on the scripts that use the sibling libs.

that depends, how would the sibling packages get to a remote machine ?

  
  
Posted one year ago

Hi SkinnyPanda43

Let's say that I install the shared libs with pip in editable mode on my development evironment, how does the clearml-agent will handle those libraries if I submit a job

So installing packages from local folders with "-e" is in general ill-advised.
But using a full git path should work out of the box. for example if you install pip install https://github.com/user/repo/repo.git then the agent will be able to install it on the remote machine. The main challenge in your setup is that the agent is installing the requirements will have a full path to the folder and not a relative one...
I would try the following and play around with the path (I think the root is the main repo path)
Task.add_requirements("my_nested_package", "-e ./relative/path/to/package") Task.init(...)wdyt?

  
  
Posted one year ago

Thank you, sorry for the daly, I have been playing around with this setup, right now I am generating a requirements.txt with git links to the sibling packages, I had to set on the agent to force the use of the reuqirements.txt. But I am starting to wonder whether It would be easier just changing sys,path on the scripts that use the sibling libs.

  
  
Posted one year ago
779 Views
5 Answers
one year ago
one year ago
Tags