ScrawnyCrocodile51 how are you calling Tasl.init()?
function's signature in the source code
task = Task.init(
project_name=PROJECT_NAME,
task_name=TASK_NAME,
task_type=Task.TaskTypes.data_processing,
)
task.set_repo(
repo="git@xxx.git",
branch=branch
)
task.set_base_docker(
docker_image="docker-image",
)
task.execute_remotely(queue_name=QUEUE_NAME)
This is how I use the task init
I mean that task.init
will automatically detect the repo without specifying it
Hi ScrawnyCrocodile51 , You can set the repository using Task.set_repo
- None
Although if you use Task.init
it will automatically detect the repository from the script. If you don't want to execute the code on your machine you can use Task.execute_remotely
- None
And finally, you can use Task.set_base_docker
to set the docker image you want to use - None
I would advise using Task.init
and it should make your life much easier 🙂
Would it cause problem to manually set repo when using task.init()?
Thanks for the tips! I will take them for a spin
that is just to compare the same functionality used in task.create() can be achieved when using task.init workflow. Not technically required
Executing task id [7605f1e5ce6b45e99e9302d93bc3bac6]:
repository = git@xxx
branch = xxx
version_num = 9dca88fa23ff93d446eb2ff7d615d7ade213c8aa
tag =
docker_cmd = iocr.io/xxx
entry_point = clearml_init.py
working_dir = dev
Based on the logging,working_dir = dev
is the problem. I need to have a way to overwrite the working_dir.
Can you add a screenshot of the execution section of the experiment when you add task.set_repo and when you don't (Just add some sample script in the same folder and run task.init and print hello world)?
I can try taking it out see if it fix the issue. But i feel it is not the root cause
Confirmed that without task.set_repo, it come down to the same error:ModuleNotFoundError: No module named 'src'
I gave this it a try to switch from Task.create() to Task.init(). I think I am pretty close to switch to using init(). But still have issue of ModuleNotFoundError: No module named 'src'
when using task.init().
My project setup look like this:
project_root/
|--src/
|--runbooks/
|--run_task.py
So if I use Task.create(repo=xx, script="runbooks/run_task.py"), it works but if I switch to using Task.init() with the same repo setup (task.set_repo, and then follow by task.execute_remotely), it will complain about ModuleNotFoundError: No module named 'src'`
I suspect that it is about how the working_directory setup, from what I see in the signature of task.create and task.set_repo, they both state that Default: repository root folder
(task.create signature), execution work directory will be the repository root folder
(task.set_repo signature). But there must be something different about this two workflow since one work another doesn't
Any ideas?
from clearml import Task
task = Task.init(,...)
print("hello world!")