Unanswered
Does Anyone Have Any Examples Or Advice On How To Implement A Dag Like This In Clearml Pipelines? Say I Want To Do Crossvalidation (Or In My Case Backtesting On Different Horizons For A Forecasting Model) Where I Have Some Common Pieces And Also A Map/Red
I see, you can manually do that with add steps, i.e.
for elem in map:
pipeline.add_step(..., elem)
or you can do that with full logic:
@PipelineDecorator.component(...)
def square_num(num):
return num**2
@PipelineDecorator.pipeline(...)
def map_flow(nums):
res = []
# This will run in parallel
for num in nums:
res.append(square_num)
# this is where we actually wait for the results
for r in res:
print_nums(r)
map_flow([1,2,3,5,8,13])
166 Views
0
Answers
one year ago
one year ago