Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hello, I'M Trying To Save A Keras Model As A Task Artifact, And Then Upload It From Another Task. Does Anyone Know The Syntax For That? What I'Ve Seen Is Not Quite Working.

Hello, I'm trying to save a keras model as a task artifact, and then upload it from another task. Does anyone know the syntax for that? What I've seen is not quite working.

  
  
Posted 3 years ago
Votes Newest

Answers 30


I see, thanks. Let me give it a try.

  
  
Posted 3 years ago

Ok, thanks!

  
  
Posted 3 years ago

ConfusedPig65 could you send the full log (console) of this execution?

  
  
Posted 3 years ago

I see...

  
  
Posted 3 years ago

Then check in the clearml.conf under files_server
And use what you have there (for example http://localhost:8081 )

  
  
Posted 3 years ago

Ah, thanks. What should I use as an upload url? Is
" "OK, or was that just a filler?

  
  
Posted 3 years ago

And I initialized the task with the output_uri

  
  
Posted 3 years ago

It does not upload, the default behavior is to log the artifact (so you know where you stored, but not enforce unnecessary uploads)
If you were to change:
task = Task.init(project_name='examples', task_name='Keras with TensorBoard example')to:
task = Task.init(project_name='examples', task_name='Keras with TensorBoard example', output_uri=" ")It would also upload the model

  
  
Posted 3 years ago

🤞

  
  
Posted 3 years ago

And then I check the artifacts for the task in the UI and don't see anything

  
  
Posted 3 years ago

okay, let me know if it works

  
  
Posted 3 years ago

I'm trying this...

  
  
Posted 3 years ago

ohh sorry, weights_url=path
Basically url can be the local path to the weights file 🙂

  
  
Posted 3 years ago

Hmm, I think it is this line:

WARNING - Model configuration only supports dictionary or string objects
done

Let me check something.

  
  
Posted 3 years ago

So I have a task that just loads a model, but I don't see it as an artifact in the UI

You should see it under Artifacts, Input model if you are calling Keras load function (or similar)

  
  
Posted 3 years ago

Let me check

  
  
Posted 3 years ago

Ok great, otherwise is there another way?

  
  
Posted 3 years ago

You can always log it manually:
from clearml import InputModel input_model = InputModel.import_model(weights_url='/tmp/keras_example/weight.6.hdf5')

  
  
Posted 3 years ago

And what if I have a model but not a weights_url?

  
  
Posted 3 years ago

So I have a task that just loads a model, but I don't see it as an artifact in the UI

  
  
Posted 3 years ago

Where exactly does the artifact upload happen here?

  
  
Posted 3 years ago

OK, and if I wasn't using the latest RC?

  
  
Posted 3 years ago

Okay ConfusedPig65 I found the problem. For some reason the latest TF.keras.load_model . save_model is not tracked.
I'll make sure we push a fix later today

  
  
Posted 3 years ago

As in the context of my code example

  
  
Posted 3 years ago

Hmm, but this is different than you described earlier

  
  
Posted 3 years ago

Sure

  
  
Posted 3 years ago

You can check the keras example, run it twice, on the second time it will continue from the previous checkpoint and you will have input and output model.
https://github.com/allegroai/clearml/blob/master/examples/frameworks/keras/keras_tensorboard.py

  
  
Posted 3 years ago

If you are using the latest RC:
pip install clearml==0.17.5rc5You can pass True it will use the "files_server" as configured in your clearml.conf
I used the http link as a filler to point to the files_server.
Make sense ?

  
  
Posted 3 years ago

Hi ConfusedPig65
Any keras model will be automatically uploaded if you pass an upload url to the Task init:
task = Task.init('examples', 'keras upload test', output_uri=" ")(You can also pass to output_uri s3://buckket/folder or change the default output_uri in the clearml.conf file)
After this line any keras model will be automatically uploaded (you will see it under the Artifacts Tab)
Accessing models from executed tasks:
trains_task = Task.get_task('task_uid_here') last_checkpoint = trains_task.models['output'][-1].get_local_copy()

  
  
Posted 3 years ago

Thank you very much for your time!

  
  
Posted 3 years ago
557 Views
30 Answers
3 years ago
one year ago
Tags