Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
I Have Code That Does Torch.Load(Path) And Deserializes A Model. I Am Performing This In Package A.B.C, And The Model’S Module Is Available In In A.B.C.Model Unfortunately, The Model Was Serialized With A Different Module Structure - It Was Originally Pla

I have code that does torch.load(path) and deserializes a model.
I am performing this in package a.b.c, and the model’s module is available in in a.b.c.model
unfortunately, the model was serialized with a different module structure - it was originally placed in a (root) module called model
To fix this, I append “model” to sys.path.
This works in pure python.

Unfortunately, this doesn’t work inside clear.ml since there is some mechanism that overrides the import mechanism using import_bind . __patched_import3

Any idea how I can work around this?

  
  
Posted 2 years ago
Votes Newest

Answers 9


I will try and get back to this area of the code soon

  
  
Posted 2 years ago

Hi RoughTiger69

unfortunately, the model was serialized with a different module structure - it was originally placed in a (root) module called

model

....

Is this like a pickle issue?

Unfortunately, this doesn’t work inside clear.ml since there is some mechanism that overrides the import mechanism using

import_bind

.

__patched_import3

What error are you getting? (meaning why isn't it working)

  
  
Posted 2 years ago

What's the python, torch, clearml version?
Any chance this can be reproducible ?
What's the full error trace/stack you are getting?
Can you try to debug it to where exactly it fails here?
https://github.com/allegroai/clearml/blob/86586fbf35d6bdfbf96b6ee3e0068eac3e6c0979/clearml/binding/import_bind.py#L48
RoughTiger69 wdyt?

  
  
Posted 2 years ago

it is a pickle issue
‘package model doesn’t exist’

despite me attempting to add the right path to sys.path right before loading

  
  
Posted 2 years ago

RoughTiger69 can you share the python version and the logs?

  
  
Posted 2 years ago

I think it has something to do with clearml since I can run this code as pure python without clearml, and when I activate clearml, I see that torch.load() hits the

import_bind

.

__patched_import3

when trying to deserialize the saved model

  
  
Posted 2 years ago

python 3.8
I’ve worked around the issue by doing:
sys.modules['model'] = local_model_package

  
  
Posted 2 years ago

it is a pickle issue
‘package model doesn’t exist’

Sounds like it, why do you think clearml has anything there ?
BTW:

import_bind

.

__patched_import3

this is just so when packages that clearml autoconnects with are patched if imported After Task.init was called.

  
  
Posted 2 years ago

Nice workaround!
RoughTiger69 how do I reproduce this behavior? (I'm still unsure on why exactly the clearml binding broke it, and would like to fix that)
(can you also provide the crash trace, maybe that could help as well)

  
  
Posted 2 years ago
1K Views
9 Answers
2 years ago
one year ago
Tags