ClearML FAQ | Hey, How Do I Use Local Files As Dependencies? I Have Several Tasks That I Want To Run In The Pipeline, So Optimally I Would Use Tasks From Functions. Can I Specify In Task What Local Files Do I Use That Should Be Packaged? Or Do I Have To Pack Everything

Answered

Hey, How Do I Use Local Files As Dependencies? I Have Several Tasks That I Want To Run In The Pipeline, So Optimally I Would Use Tasks From Functions. Can I Specify In Task What Local Files Do I Use That Should Be Packaged? Or Do I Have To Pack Everything

hey, how do I use local files as dependencies?
I have several tasks that I want to run in the pipeline, so optimally I would use tasks from functions.
Can I specify in task what local files do I use that should be packaged?
Or do I have to pack everything in a package and put it somewhere and add to requirements.txt?

  				
Posted 
	one year ago

					More
				  		
  Report
		
					FranticLobster21
				
					0
					 × 1

Votes Newest

Answers

Hi @<1539055479878062080:profile|FranticLobster21>

hey, how do I use local files as dependencies?

You mean like a repository ?

Can I specify in task what local files do I use that should be packaged?

In a git repo?

Basically the agent can do two things, either replicate a single script or clone a git repo + uncommitted changes

  				
Posted 
	one year ago

					More
				  		
  Report
		
					AgitatedDove14
				
					0
					 × 1

Write your answer

1K Views

1 Answer

one year ago

one year ago

Tags

Similar posts

Hello! Is There Any Way To Use Original Files In Cleaml Datasets ? I Have Batch Of Tar Archives And Want To Create Dataset From Them, However Clearml Compresses Them. I Tried To Use

How Do You Disable Auto Upload Pth File ? I Still Want To Have Debug Sample And Manual Upload Artifact, Just Not Uploading All The Pth File. I Think I Saw This In The Chat Somewhere But Could Not Find It Now. Use-Case: We Are Experimenting With New Framew

Hello! Is It Possible To Use Different Credentials For Different S3 Buckets On Different Hosts? When I Try To Do Like So:

Hi Everyone! I’M Trying To Save My Model’S Weights To Storage. And I Can’T Do It. I’M Tried Upload_Artifact(), Both, With Python Objects Or Path To File. There Are Errors In Uploading And Downloading Files. Usually With File Size. But It’S A Small Resnet1

Hello, I Having Been Exhausting The Metrics Quota Way To Fast For The Current Use That I Am Making Of Clearml. Is The Quota Cumulative ? I.E. Do We Get 1G Per Month ? I Am Concerned Because If We Upgrade And Need To Pay

Hi! Is There Any Way To Add Git-Like Ignore File For Versioning Clearml Data? I Saw In Docs A Wildcard Argument When Files Are Added To A Dataset. How Can I Specify Ignoring Of Some File Types? For Example, I Want To Ignore Ipynb Checkpoints. How Can I Do

Hi, I Am Wondering Is There Anyway To Use Clearml Agent Without Setting Up The Environment, For Example, Do Not Enforce The Git Clone And Pip Install But Cd Some Dir And Activate An Environment?

Hi There, I Am Looking For A Way To Use The Hyperparameter Optimization Tool From Clear Ml, For Different Algorithms I Build. The Idea Is, That I Can Use A Command Line To Specify Which Algorithm I Want To Optimize, And Let Clear Ml Do The Rest. The Issue

Hello. How Do I Tell The Clearml Agent To Not Install The Tasks Requirements And Use Existing Local Env Instead? Also What Should I Do If My Task Depends On Multiple Git Repos That Need To Fetched?

Hello, I Want To Use Clearml Model Serving To Host A Llama2 Model. It Has Another Structure As Common Models, E.G. It'S Split Up Into 3 Files. Can Llama2 Be Used With Clearml Model Serving?