Answered

In The Pipeline Examples, Components Have The Following Note:

In the pipeline examples, components have the following note:

notice all package imports inside the function will be automatically logged as

required packages for the pipeline execution step

Where are these requirements taken from? Is it matched based on name with the executing environment?
I wonder because of our internal repositories 🤔

  				
Posted 
	one year ago

					More
				  		
  Report
		
					UnevenDolphin73
				
					0
					 × 1

Votes Newest

Answers 7

Internal references are resolved only in decorator/function pipelines if I recall correctly

  				
Posted 
	one year ago

					More
				  		
  Report
		
					CostlyOstrich36
				
					0

Hi UnevenDolphin73 , I think this is analyzed in the code

  				
Posted 
	one year ago

					More
				  		
  Report
		
					CostlyOstrich36
				
					0

🤔

  				
Posted 
	one year ago

					More
				  		
  Report
		
					UnevenDolphin73
				
					0
					 × 1

Maybe a single step pipeline?

  				
Posted 
	one year ago

					More
				  		
  Report
		
					CostlyOstrich36
				
					0

CostlyOstrich36 so internal references are not resolved somehow? Or, how should one achieve:

def my_step(): from ..utils import foo foo("bar")

  				
Posted 
	one year ago

					More
				  		
  Report
		
					UnevenDolphin73
				
					0
					 × 1

I'm also getting the following warning, I guess it's some ClearML dependency?
IPython could not be loaded!

  				
Posted 
	one year ago

					More
				  		
  Report
		
					UnevenDolphin73
				
					0
					 × 1

Hm. Is there a simple way to test tasks, one at a time?

  				
Posted 
	one year ago

					More
				  		
  Report
		
					UnevenDolphin73
				
					0
					 × 1

Write your answer

956 Views

7 Answers

one year ago

one year ago

Tags

Similar posts

Hey Guys! I Would Love To Know How To Integrate Hpo Inside Clearml Pipelines. I Have Made A Continuous Learning Pipeline With Data Etl And Model Training And As A Next Step, It Would Be Cool To Add Hpo. Most Of The Examples On The Website Create A New Ta

Hey All, I Have The Following Folder Structure: / |-- Main.Py |-- Pipeline.Py |-- Utils.Py In Some Of The Pipeline Components Defined In Pipeline.Py, I Import Functions Defined In Utils.Py. If I Execute The Pipeline, I Get The Following Error: Modulenotfo

Hi All, How Can I Get The Status Of A Component From Another Component In The Clearml Pipeline (End, Pending, Running)? I Want To Run The Triton Server As A "Daemon" Thread Inside The Component And So That Other Pipeline Components Can Access It (Request)

Hello Community, I Am Trying To Run A Pipeline Using Pipeline Decorater. My Components I.E. Step_One, Step_Two.... I Want To Run This Components On A Queue Default And The Pipeline On Another Queue Services. The Issue Is The Step_One Seems To Abort Automa

[Pipeline] Hey, Is It Possible To Specify The Output Uri For Pipelines And Their Components Using Pipeline Decorators? I Would Like To Store Pipeline Artifacts And Component Artifacts On S3.

Hey, About The Dependency Propagation Of Pipeline Components, If I Call A Vanilla Python Function From A Component Does The Dependencies Specified In The Internal Imports Propagated To This Function Call Too ? And Additionally If That Function Is In Anoth

Hi All! I Noticed When A Pipeline Fails, All Its Components Continue Running. Wouldn'T It Make More Sense For The Pipeline To Send An Abort Signal To All Tasks That Depend On The Pipeline? I'M Using Clearml V1.1.3Rc0 And Clearml-Agent 1.1.0

Hello. I Have Several Questions Regarding The Pipeline Components Of Clearml. I Have Read The Docs, But I Still Don'T Have A Clear Picture Of The Interplay Between Them. As I Know A Little Bit Better Luigi And Kedro, I Will Try To Explain How Are They Rel