Unanswered
Hey All,
We Are Trying To Clone A Task That Uses Custom Pip Installed Packages And Run It Via An Agent.
When Running Locally, We Simply “
We are cloning an existing task (pipeline). Adding Task.add_requirements("./path/to/package")
before .Task.clone(...)
gives:
2023-02-22 14:08:31,508 - clearml.task - WARNING - Requirement ignored, Task.add_requirements() must be called before Task.init()
Followed by this further down:
with Path(package_name).open() as requirements_txt:
File "/home/ec2-user/miniconda3/envs/ml/lib/python3.10/site-packages/pathlib2/__init__.py", line 1548, in open
return io.open(
IsADirectoryError: [Errno 21] Is a directory: 'package'
Because we're cloning a task, I've tried specifying the package path in the pipeline decorator we're using and the steps which require the package. E.g.:
@PipelineDecorator.component(
cache=True,
task_type=TaskTypes.data_processing,
packages='./ppackage',
)
def step(...
@PipelineDecorator.pipeline(
packages='./package',
)
def pipe(...
When running this pipeline, I get the following which has made me think that packages cannot be specified via paths?
with Path(package_name).open() as requirements_txt:
File "/home/ec2-user/miniconda3/envs/ml/lib/python3.10/site-packages/pathlib2/__init__.py", line 1548, in open
return io.open(
IsADirectoryError: [Errno 21] Is a directory: 'package'
166 Views
0
Answers
one year ago
one year ago