Reputation
Badges 1
7 × Eureka!I found out that if I work with one model than that model is saved but if I work with different model that model is not saved, everything else being the same. Both models use Functional API
Edit: I have further pinpointed a problem to a ReLU layer. If i uselayers.ReLU()
it does not work, but if I uselayers.Activation("relu")
it works.
I'm sorry for late response. You could probably replicate it by instead of using activation="relu"
you used new ReLU layer after Dense. Or if this does not do, extract part of the model in separate Sequential model, for exampleSequential([ Dense(128), BatchNormalization(), ReLU(), ])
self.d1 = Dense(128, dtype=tf.float32) self.d1_a = ReLU()
i just run it and it saved it, I did nothing more then what was provided in the code. Only difference is in those two lines with comments, one works one does not.
# outputs = TestLayer(a=1, b=2)(inputs) # model uploads properly outputs = TestLayer(a=1, b={"b": 2})(inputs) # model does not upload
I just uncommented first line and commented second line
There is no explicit error. When I call tf.keras.models.save_model(model, model_path)
model is saved to the clearml as an output model when b
is not a dictionary. But if b
is dict than model is not saved, and is not in artifacts under output models
I will try to log model manually, thank you for the info. But still saving keras model should result in clearml artifact according to documentation, so I believe this might be a bug.
I will not be able to do it untill at least tomorrow but rest of the code is quite simple.. It is essentially this;
` task = Task.init(...)
task.output_uri = gs://<artifact_location>
model = build_model_test(None)
tf.keras.models.save_model(model, "/tmp/model")
`
I use both latest stable versions of clearml and tensorflow
AgitatedDove14 Hi, were you able to reproduce issue? Can I be of any assistance? Do you still need toy code?