Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi Everyone, I Get An Error When I Add An Argument Of Type Enum To A Pipeline Component (@Pipelinedecorator.Component). At The Same Time Pipelines (@Pipelinedecorator.Pipeline) And Normal Functions Work Fine With Enums. The Error Message Looks Like This:

Hi everyone,

I get an error when I add an argument of type Enum to a pipeline component (@PipelineDecorator.component). At the same time pipelines (@PipelineDecorator.pipeline) and normal functions work fine with enums. The error message looks like this:

File "...\AppData\Local\Temp\tmploip_b4_.py", line 124
kwargs = {'my_enum': <MyEnum.FALSE: 'FALSE'>}

Here is a little code example

from enum import StrEnum
from clearml.automation.controller import PipelineDecorator

class MyEnum(StrEnum):
    FALSE = "FALSE"
    TRUE = "TRUE"

#this fails
@PipelineDecorator.component(return_values=["foo"], cache=False, task_type=TaskTypes.training)
def my_component(my_enum: MyEnum = MyEnum.FALSE)
    print(my_enum)

#this works
@PipelineDecorator.pipeline(name='name', project='project', version=1)
def my_pipeline(my_enum: MyEnum = MyEnum.FALSE)
    print(my_enum)

#this works
def my_function(my_enum: MyEnum = MyEnum.FALSE)
    print(my_enum)
    

Is it an actual issue or am I misusing something here?

  
  
Posted 2 months ago
Votes Newest

Answers 5


@<1643060801088524288:profile|HarebrainedOstrich43> you are right. we actually attempt to copy the default arguments as well. What happens is that we aggregate these arguments in the kwargs dict, then we dump str(kwargs) in the script of the pipeline step. Problem is, str(dict) actually calls __ repr_ _ on each key/value of the dict, so you end up with repr(MyEnum.FALSE) in your code, which is <MyEnum.FALSE: 'FALSE'> . One way to work around this is to add something like:

class MyEnum(Enum):
    FALSE = "FALSE"
    TRUE = "TRUE"

    def __repr__(self):
        return str(self)  # notice this

to your enum

  
  
Posted 2 months ago

Hi @<1523701435869433856:profile|SmugDolphin23> , thank you for getting back to me. I actually have my Enum in a separate file. I have tried importing it both to the context (file with pipeline component) and to inside of the pipeline component function. Unfortunatelly it does not work. The same scenario for pipeline itself works perfectly fine (when Enum is imported to the context, aka .py file with pipeline code).

Important : it only fails when default value is provided. Simply defining the type works fine for both pipeline and pipeline components.

  
  
Posted 2 months ago

@<1523701435869433856:profile|SmugDolphin23> thank you for such a detailed explanation! I will give it a try now :)

  
  
Posted 2 months ago

or rather than str(self) , something like:

    def __repr__(self):
        return self.__class__.__name__ + "." + self.name

should work better

  
  
Posted 2 months ago

Hi @<1643060801088524288:profile|HarebrainedOstrich43> ! At the moment, we don't support default arguments that are typed via a class implemented in the same module as the function.
The way pipelines work is: we copy the code of the function steps (eventually their decorator's as well if declared in the same file), then we copy all the imports in the module. Problem is, we don't copy classes.
You could have your enum in a separate file, import it and it should work

  
  
Posted 2 months ago
164 Views
5 Answers
2 months ago
2 months ago
Tags