Hello, I Am Trying To Modify My Clearml-Agent Running On A Aws Autoscaler (From Clearml Applications). I Want To Be Able To Clone My Repo (Working), And Install My Poetry Dependencies From

Answered

Hello,

I am trying to modify my clearml-agent running on a AWS autoscaler (From ClearML applications).
I want to be able to clone my repo (working), and install my poetry dependencies from poetry.lock .
I also want to use Python 3.9.(16) as requirement, which is not the default Python version.

I read a lot about ClearML docs and found these parameters to modify :

agent {
    python_binary: "/usr/local/bin/python3.9"
    default_python: "3.9"
    package_manager: {
          type: poetry,
          poetry_version: "1.4.1",
     }
    
}

I am thus setting the above snipped as additional clearml.conf inside the AWS autoscaler configuration.
However, none of the above allows to change python default version while poetry is changed (but fails to download...)
Can someone provide me with any help/tips/directions?

Many thanks!

  				
Posted 
	one year ago

					More
				  		
  Report
		
					SuccessfulRaven86
				
					0
					 × 1

Votes Newest

Answers 5

Hi @<1556812486840160256:profile|SuccessfulRaven86> , can you please add an example configuration that reproduces this?

  				
Posted 
	one year ago

					More
				  		
  Report
		
					CostlyOstrich36
				
					0

Do you know whether the agent VM/image has python 3.9 installed ? Also, you emphasised that this happens when setting the package manager to poetry, does it mean this issue doesn’t happen when leaving package manager settings to default values ?

  				
Posted 
	one year ago

					More
				  		
  Report
		
					EnthusiasticShrimp49
				
					0

@<1556812486840160256:profile|SuccessfulRaven86> , what is the base docker image you mention? Did you check that this docker has python 3.9?

  				
Posted 
	one year ago

					More
				  		
  Report
		
					CostlyOstrich36
				
					0

Thank you for the quick replies!

I might do it the wrong way but the above snippet of code is the additional clearml.conf file I add to the AWS autoscaler. Should I add a complete clearml.conf file to it?

That is a good question @<1537605940121964544:profile|EnthusiasticShrimp49> ! I am not sure the image has python 3.9. I tried to check it but did not find the answer. I am using the following AMI: AWS Deep Learning AMI (Ubuntu 18.04) with Support by Terracloudx (Nvidia deep learning AMI). For the docker image, I am using the base one. What parameters should I modify to change the python version?

  				
Posted 
	one year ago

					More
				  		
  Report
		
					SuccessfulRaven86
				
					0
					 × 1

@<1523701070390366208:profile|CostlyOstrich36> The base docker image of the AWS autoscaler is nvidia/cuda:10.2-runtime-ubuntu18.04 . According to me, the python version is not set inside the image, but I am might be wrong and it could be the problem indeed... ?

  				
Posted 
	one year ago

					More
				  		
  Report
		
					SuccessfulRaven86
				
					0
					 × 1

Write your answer

1K Views

5 Answers

one year ago