ClearML FAQ | Hi, I Am Following The Programmatic Orchestration Example Here:

Answered

Hi, I Am Following The Programmatic Orchestration Example Here:

Hi, I am following the programmatic orchestration example here: https://clear.ml/docs/latest/docs/guides/automation/task_piping .

My question is, when I setup the toy base task (eg https://github.com/allegroai/clearml/blob/master/examples/automation/toy_base_task.py ), how can I just "define" it on my local PC, but not actually run it. I want to do this since I want to run it on my EC2 instances using autoscaler.

Currently, as my local machine is windows and EC2 is Linux, I do:
if sys.platform != 'win32': <run task...> else: sys.exit(0)but I assume there are more elegant solutions....

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					CloudySwallow27
				
					0
					 × 1

Votes Newest

Answers 2

Hi CloudySwallow27

how can I just "define" it on my local PC, but not actually run it.

You can use the clearml-task CLI
https://clear.ml/docs/latest/docs/apps/clearml_task#how-does-clearml-task-work
Or you can add the following line in your code, that will cause the execution to stop, and to continue on a remote machine (basically creating the Task and pushing it into an execution queue, or just aborting it)
task = Task.init(...) task.execute_remotely()https://clear.ml/docs/latest/docs/references/sdk/task#execute_remotely

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					AgitatedDove14
				
					0
					 × 1

Thanks this worked!

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					CloudySwallow27
				
					0
					 × 1

Write your answer

992 Views

2 Answers

2 years ago

one year ago

Tags

Similar posts

Hi :wave: I am deploying ClearML on an EC2 instance using the EC2 Community AMI. I am confused as to how to configure the server so that it can recover data if the EC2 instance crashes or is terminated; should I host all my data in S3? If so, how do I rea

Hi, Everyone. If I Am Self-Hosting My Clearml In Ec2, Do I Get To Have The Autoscaler Feature, Or Do I Still Need To Get The Pro Version To Have Access To This Feature ?

Hi There. Quick Question. If I Deployed Clearml Server On One Aws Ec2, Is It Possible To Train My Model On Other Ec2 With Gpu?

Hey All, In Amazon Aws, Using The Auto Scaler, I Want To Mount My Efs On All Ec2 Workers Who Get Spawned By The Auto Scaler. Assuming I Edited My Security Group Rules, And Updated My Iam Role So It Could Access The Efs, How Do I Mount The Efs To The Machi

Hello Everyone! I'M Using Clearml, Yolo V8 And Clearml Gpu Compute (For Orchestration). The Issue Is That I Can'T Find A Compatibility To Use Gpu During Yolo Training (On A Remote Instance, Not Local). My Local Machine Is Macbook On M2 Chip - Maybe It'S

Hi All, I Am Designating An Ec2 Instance To Host My Web-Server And Handle All Auto-Scaling Operations On Aws. I Used The Ami (Ami-094B50Dd004Bdb97D) That Was Given From The Website, When Running The Aws_Autoscaler.Py I'M Getting This Error:

Hi! I Am Setting Up My First Clearml Agent (Ec2) Connecting To My Hosted Clearmlserver. I Am At The Step Where I Am Granting The Agent Access To Github. I Am Using Ssh Authentication, So I:

Hi, When I Run A Pipeline, The Artifacts From My Tasks Are Saved On My Ec2 Server. When I Archive The Pipeline And Go Into The Archive And Delete The Pipeline, The Artifacts Are Not Deleted. It Looks Like They Are Only Deleted If I Delete The Whole Pipel

In Order For A New Worker To Come Online In My K8 Cluster, Do I Need To Have An Ec2 Startup Script Init The Agent/Config, And Then Start The Daemon? Do I Have To Do This Manually Is This A Better Way?

Hi There, Would It Be Possible For The Autoscaler To Support Stopping Instances Instead Of Terminating Them? My Use Case Is The Following: I Am Continuing My Journey With The Clearml-Session Tool, And In Case The Clearml-Session Is Running In A Ec2 Inst