Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
I Have A Question About How To Load A Local Directory To Clearml'S

I have a question about how to load a local directory to clearml's venv-builds

Assume the following file layout:
` tree
.
├── demo.py
└── local_dir
├── hello.py
└── init.py

1 directory, 3 files `

where demo.py

` from clearml import PipelineController

def step_one_task(short_name, code_root):
import json
import os, sys

HERE = os.path.dirname(os.path.abspath(__file__))
print(HERE)
sys.path.append(HERE)

from local_dir.hello import pi_arr

print(pi_arr())

if name == "main":

# create the pipeline controller
pipe = PipelineController(
    project="rudolf",
    name="Demo Pipeline",
    version="0.1",
    add_pipeline_tags=False,
    target_project=True,
)

# set the default execution queue to be used (per step we can override the execution)
pipe.set_default_execution_queue("rudolf")

step_one = "step_one"

pipe.add_function_step(name=step_one, function=step_one_task)

pipe.start(queue="rudolf")

print("pipeline completed") `

and local_dir/hello.py

has the following content:
import numpy as np def pi_arr(): return np.array([3.14])

I am getting this error: when running python demo.py

Traceback (most recent call last): File "/yossi/.clearml/venvs-builds/3.7/code/step_one_task.py", line 30, in <module> results = step_one_task(**kwargs) File "/yossi/.clearml/venvs-builds/3.7/code/step_one_task.py", line 11, in step_one_task from local_dir.hello import pi_arr File "/yossi/.clearml/venvs-builds/3.7/lib/python3.7/site-packages/clearml/binding/import_bind.py", line 59, in __patched_import3 level=level) ModuleNotFoundError: No module named 'local_dir'
How can I make ClearML to see the local_dir library?

  
  
Posted one year ago
Votes Newest

Answers 7


Hi AverageRabbit65 ,

Is this part of a repository? if so, you can specify the repo in the add_function_step

  
  
Posted one year ago

Will make it as a local repo. Thank you so much!

  
  
Posted one year ago

TimelyPenguin76

thanks again... I have tried what you have suggested. It does inject the dir, but not cloning the whole content.

os.system("tree") from within demo.py pipeline step function:

└── test-demo-clearml.git ├── demo.py ├── __init__.py ├── local_dir └── step_one_task.pyvs.
from the actual tree (under ~/clearml_using_local_lib )
` tree
.
├── demo.py
├── init.py
└── local_dir
├── hello.py
├── init.py
└── pycache
└── init.cpython-37.pyc

2 directories, 5 files `

in the python

pipe.add_function_step( name=step_one, function=step_one_task, repo="~/clearml_using_local_lib", )

  
  
Posted one year ago

would love to hear your thoughts

  
  
Posted one year ago

solved with Alon's help 1x1

  
  
Posted one year ago

How did you solve it?

  
  
Posted one year ago

there was an issue with the layout of my simple git example. So if you do the above it should work

  
  
Posted one year ago