Hello. I Have A Question Regarding Jupyter Notebook Execution With Clearml Worker. Long Story Short: I Turn A Remote Machine Into Worker With Clearml-Agent. Then I Made A Task With The Local Machine By Executing Notebook Cells, Which Imports A Training M

Answered

Hello. I have a question regarding jupyter notebook execution with clearml worker.
Long story short: I turn a remote machine into worker with clearml-agent. Then i made a task with the local machine by executing notebook cells, which imports a training modules, containing clearml task initialization and artefact streaming. This task successfully finished locally. Then i enqueued it and it started to executing on a worker machine. But the i got an error:

`Environment setup completed successfully
Starting Task Execution:
C:\Users\sparkhonyuk\AppData\Local\Programs\Python\Python310\python.exe: can't open file 'C:\Users\sparkhonyuk\.clearml\venvs-builds\3.10\task_repository\seismic-sr-ernest.git\notebooks\03_train_unet_2D.py':
[Errno 2] No such file or directory

So it turned out, that worker changed the extension of03_train_unet_2D.ipynbon.py, tried to find this file in cloned repository on a worker machine, and output an error, because there is no such file. My question is: how to make clearml worker understand, that it should run.ipynbfile, not.py` script?

  				
Posted 
	one year ago

					More
				  		
  Report
		
					AdventurousElephant3
				
					0
					 × 1

Votes Newest

Answers 2

Hi @<1555362936292118528:profile|AdventurousElephant3>
I think your issue is that Task supports two types of code,

single script/jupyter notebook
git repo + git diffIn your example (If I understand correctly) you have a notebook calling another notebook, which means the first notebook will be stored on the Task, but the second notebook (not being part of a repository) will not be stored on the task, and this is why when the agent is running the code it fails to find the second notebook. Could that be the case ?

  				
Posted 
	one year ago

					More
				  		
  Report
		
					AgitatedDove14
				
					0
					 × 1

No, my notebook calling python script while training. This script contains task initialization and artifacts reporting. But when the worker starts to run enqueued task, it considers, that 03_train_unet_2D.ipynb has .py extension. And obviously it can’t find it in cloned repository.

  				
Posted 
	one year ago

					More
				  		
  Report
		
					AdventurousElephant3
				
					0
					 × 1

Write your answer

626 Views

2 Answers

one year ago