Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi All! I Have Methods Inside Notebooks That I Made Available To Clis Using Nbdev

Hi all!

I have methods inside notebooks that I made available to CLIs using nbdev export and fastai’s @call_parse , and also a functional EKS cluster with self-hosted clearml for remote execution of tasks.
When running these cli commands with underlying python code, everything reports correctly to the clearml server. However no useful template is created for down stream executions - the source code template is all messed up, just mentioning the cli entrypoint. Also uncommitted changes show that try to load my own package (I suspect fastai causes that).
My question is - what is a best practice in this case to be able to run exported scripts (python code not made available thourgh a .py file but through a bash script).

Any help is appreciated

  
  
Posted one year ago
Votes Newest

Answers 3


Reproducing is simple:

  • In a notebook, create a method and decorate it by fastai.script’s @call_parse .
  • Call Task init inside it
  • Expose the method with nbdev using settings.ini
  • run via bash
    The created task contains no repo/branch and has uncommitted changes:
#!/home/ubuntu/miniconda3/bin/python
# EASY-INSTALL-ENTRY-SCRIPT: 'my-package','console_scripts','my-script'
import re
import sys
# for compatibility with easy_install; see #2198
__requires__ = 'my-package'
try:
    from importlib.metadata import distribution
except ImportError:
    try:
        from importlib_metadata import distribution
    except ImportError:
        from pkg_resources import load_entry_point
def importlib_load_entry_point(spec, group, name):
    dist_name, _, _ = spec.partition('==')
    matches = (
        entry_point
        for entry_point in distribution(dist_name).entry_points
        if entry_point.group == group and entry_point.name == name
    )
    return next(matches).load()
globals().setdefault('load_entry_point', importlib_load_entry_point)
if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(load_entry_point('my-package', 'console_scripts', 'my-script')())

These changes are not in our code base, I suspect they are the result of exporting with bash.

Since this is a key workflow (executing py code via exported script with bash) I wanted to ask if there’s a way to do so without an explicit python entrypoint (what does work is overriding attributes in the task object created so that future tasks in this template will run an actual file. But this still requires a file entrypoint to exist :/)

  
  
Posted one year ago

  • In a notebook, create a method and decorate it by fastai.script’s @call_parse .Any chance you have a very simple code/notebook to reference (this will really help in fixing the issue)?
  
  
Posted one year ago

Hi @<1528908687685455872:profile|MassiveBat21>

However

no useful

template

is created for down stream executions - the source code template is all messed up,

Interesting, could you provide the code that is "created", or even better some way to reproduce it ? It sounds like sort of a bug? or maybe a feature support that is missing.

My question is - what is a best practice in this case to be able to run exported scripts (python code not made available thourgh a .py file but through a bash script).

If you are exporting the code manually, then I would use clearml-task to launch it (or create an experiment).
None

BTW: if in your jupyter notebook you add the Task.init call, it will automatically export it into clearml so you can clone/enqueue it right from the UI, without the need to manually export the notebook

  
  
Posted one year ago
631 Views
3 Answers
one year ago
one year ago
Tags