Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
I Have A Question About How To Load A Local Directory To Clearml'S

I have a question about how to load a local directory to clearml's venv-builds

Assume the following file layout:
` tree
.
├── demo.py
└── local_dir
├── hello.py
└── init.py

1 directory, 3 files `

where demo.py

` from clearml import PipelineController

def step_one_task(short_name, code_root):
import json
import os, sys

HERE = os.path.dirname(os.path.abspath(__file__))
print(HERE)
sys.path.append(HERE)

from local_dir.hello import pi_arr

print(pi_arr())

if name == "main":

# create the pipeline controller
pipe = PipelineController(
    project="rudolf",
    name="Demo Pipeline",
    version="0.1",
    add_pipeline_tags=False,
    target_project=True,
)

# set the default execution queue to be used (per step we can override the execution)
pipe.set_default_execution_queue("rudolf")

step_one = "step_one"

pipe.add_function_step(name=step_one, function=step_one_task)

pipe.start(queue="rudolf")

print("pipeline completed") `

and local_dir/hello.py

has the following content:
import numpy as np def pi_arr(): return np.array([3.14])

I am getting this error: when running python demo.py

Traceback (most recent call last): File "/yossi/.clearml/venvs-builds/3.7/code/step_one_task.py", line 30, in <module> results = step_one_task(**kwargs) File "/yossi/.clearml/venvs-builds/3.7/code/step_one_task.py", line 11, in step_one_task from local_dir.hello import pi_arr File "/yossi/.clearml/venvs-builds/3.7/lib/python3.7/site-packages/clearml/binding/import_bind.py", line 59, in __patched_import3 level=level) ModuleNotFoundError: No module named 'local_dir'
How can I make ClearML to see the local_dir library?

  
  
Posted 2 years ago
Votes Newest

Answers 7


TimelyPenguin76

thanks again... I have tried what you have suggested. It does inject the dir, but not cloning the whole content.

os.system("tree") from within demo.py pipeline step function:

└── test-demo-clearml.git ├── demo.py ├── __init__.py ├── local_dir └── step_one_task.pyvs.
from the actual tree (under ~/clearml_using_local_lib )
` tree
.
├── demo.py
├── init.py
└── local_dir
├── hello.py
├── init.py
└── pycache
└── init.cpython-37.pyc

2 directories, 5 files `

in the python

pipe.add_function_step( name=step_one, function=step_one_task, repo="~/clearml_using_local_lib", )

  
  
Posted 2 years ago

How did you solve it?

  
  
Posted 2 years ago

solved with Alon's help 1x1

  
  
Posted 2 years ago

would love to hear your thoughts

  
  
Posted 2 years ago

Will make it as a local repo. Thank you so much!

  
  
Posted 2 years ago

Hi AverageRabbit65 ,

Is this part of a repository? if so, you can specify the repo in the add_function_step

  
  
Posted 2 years ago

there was an issue with the layout of my simple git example. So if you do the above it should work

  
  
Posted 2 years ago