Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
When Clearml Converts A

When ClearML converts a PipelineDecorator.component decorated function to script code, I have noticed that indexing syntax like A[:, 0] is rewritten as A[(:, 0)], something that Python does not allow. Is this intentional or just a bug?

  
  
Posted 3 years ago
Votes Newest

Answers 13


GiganticTurtle0 hi!

Do you have an example of your use case?

  
  
Posted 3 years ago

GiganticTurtle0 I'm not sure I follow, what do you mean by indexing the arguments? Can you post a short usage example ?

  
  
Posted 3 years ago

Sure, I will post a mock example in a while

  
  
Posted 3 years ago

BTW, I would like to mention another problem related to this I have encountered. It seems that arguments of type 'int', 'float' or 'list' (maybe also happens with other types) are transformed to 'str' when passed to a function decorated with PipelineDecorator.component at the time of calling it in the pipeline itself. Again, is this something intentional?

  
  
Posted 3 years ago

... transformed to 'str' when passed to a function decorated with 

PipelineDecorator.component

 at the time of calling it in the pipeline itself. Again, is this something intentional?

Are you sure about that? Notice the example code specifies, int as well...

  
  
Posted 3 years ago

I have also tried with type hints and it still broadcasts to string. Very weird...

  
  
Posted 3 years ago

I have also tried with type hints and it still broadcasts to string. Very weird...

Type hints are ignored, it's the actual value you pass that is important:
` @PipelineDecorator.component(return_values=['data_frame'], cache=True, task_type=TaskTypes.data_processing)
def step_one(pickle_data_url: str, extra: int = 43):
...

@PipelineDecorator.pipeline(name='custom pipeline logic', project='examples', version='0.0.5')
def executing_pipeline(pickle_url, mock_parameter='mock'):
data_frame = step_one(pickle_url, extra=1337) implies extra ` is an integer Because this is the value that was passed by the pipeline controller when calling the function
make sense ?

  
  
Posted 3 years ago

Exactly, when 'extra' has a default value (in this case, 43), the argument preserves its original type. However, when 'extra' is a positional argument then it is transformed to 'str'

  
  
Posted 3 years ago

However, when 'extra' is a positional argument then it is transformed to 'str'

Hmm... okay let me check something

  
  
Posted 3 years ago

Okay the type is inferred from the default value of the function step itself, that means that both:
data_frame = step_one(pickle_url, extra=1337)and
data_frame = step_one(pickle_url, 1337)Will pass extra as int .
That said if the default value of the argument is missing, it will revert to str
In order to use the type hints as casting hint, we actually need to improve the task.connect to support the type casting (they are stored)

  
  
Posted 3 years ago

Nice, in the meantime as a workaround I will implement a temporary parsing code at the beginning of step functions

  
  
Posted 3 years ago

GiganticTurtle0 quick update, a fix will be pushed, so that casting is based on the Actual value passed not even type hints 🙂
(this is only in case there is no default value, otherwise the default value type is used for casting)

  
  
Posted 3 years ago

Glad to hear that! Thanks!

  
  
Posted 3 years ago
919 Views
13 Answers
3 years ago
one year ago
Tags