Unanswered
Does Anyone Have Any Examples Or Advice On How To Implement A Dag Like This In Clearml Pipelines? Say I Want To Do Crossvalidation (Or In My Case Backtesting On Different Horizons For A Forecasting Model) Where I Have Some Common Pieces And Also A Map/Red
I see, you can manually do that with add steps, i.e.
for elem in map:
pipeline.add_step(..., elem)
or you can do that with full logic:
@PipelineDecorator.component(...)
def square_num(num):
return num**2
@PipelineDecorator.pipeline(...)
def map_flow(nums):
res = []
# This will run in parallel
for num in nums:
res.append(square_num)
# this is where we actually wait for the results
for r in res:
print_nums(r)
map_flow([1,2,3,5,8,13])
92 Views
0
Answers
10 months ago
10 months ago