Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi! I Have Local Minio Setup, Via Minio Browser I Can Upload 50-100 Mb Per Second As Its Local. But When I Try To Use Task.Upload_Artifact It Uploads 500 Kb Per Second. Does Anyone Have An Idea About This?

Hi!
I have local Minio setup, via Minio Browser I can upload 50-100 MB per second as its local.
But when I try to use task.upload_artifact it uploads 500 KB per second. Does anyone have an idea about this?

  
  
Posted 3 years ago
Votes Newest

Answers 30


I was thinking to do 2. step manually

So you are saying that step 1 is the problematic one?

  
  
Posted 3 years ago

upload_artifact will actually do two things:
upload the file to the trains-server register it as an artifact on the experiment
What did you mean by "register the artifact manually"? You still need to upload the file to the trains-server (so it is later accessible )

  
  
Posted 3 years ago

There is some overhead, but it should be negligible.

  
  
Posted 3 years ago

` from trains import Task

task = Task.init(project_name="SpeedTest",
task_name="test1",
task_type=Task.TaskTypes.data_processing,
output_uri="s3://<my_minio_ip>:<my_minio_port>/<my_bucket_name>",
reuse_last_task_id=False)

task.upload_artifact(name="test1",
artifact_object="<my_local_file>") `

  
  
Posted 3 years ago

MuddyCrab47 could you post the full sample code you are using?

  
  
Posted 3 years ago

I think I found something, let me dig deeper 🙂

  
  
Posted 3 years ago

How about until weekend? Do you have a temporary solution=

  
  
Posted 3 years ago

Oh, glad we found out a solution.

  
  
Posted 3 years ago

Yes!
This solved the problem.
Thank you AgitatedDove14 !

  
  
Posted 3 years ago

MuddyCrab47 Try this one 🙂

  
  
Posted 3 years ago

Anyhow if the StorageManager.upload was fast, the upload_artifact is calling that exact function. So I don't think we actually have an issue here. What do you think?

  
  
Posted 3 years ago

The code took 0.16 for 33 MB

  
  
Posted 3 years ago

That seems reasonable to me:)

  
  
Posted 3 years ago

okay we are good 🙂

  
  
Posted 3 years ago

So I need to fix Minio then?

  
  
Posted 3 years ago

Yep

  
  
Posted 3 years ago

Does StorageManager.upload and upload_artifact use the same methods?

Yes they both use StorageManager.upload

Is the only difference is task being async?

Two differences:
Upload being async Registering the artifact on the experiment. StorageManager will only upload, where as upload_artifact will make sure the file is registered as an artifact on the experiment, together with all of the artifacts properties.

  
  
Posted 3 years ago

Does StorageManager.upload and upload_artifact use the same methods?

Is the only difference is task being async?

  
  
Posted 3 years ago

Hmm

  
  
Posted 3 years ago

What if I register the artifact manually?

task.upload_artifact('local folder', artifact_object=' ')This one should be quite quick, it's updating the experiment

  
  
Posted 3 years ago

No, I think it might be a glitch in the way the calculate the upload speed, nothing we can do 🙂

  
  
Posted 3 years ago

Whose glitch on calculating the upload speed then? What will I do to fix my problem? 😞

  
  
Posted 3 years ago

When I give my Minio to output_uri argument, it uploads 500 KB /sec as before.

But it worked well when using StorageManager and uploading to the minio directly, is that correct?

.. I give my Minio to output_uri argument

How long did it take to run the demo code I posted?
(The one you mentioned took 0.16s to run locally)

  
  
Posted 3 years ago

What will I do to fix my problem?

What is the problem? we just proved the upload speed is just fine?

  
  
Posted 3 years ago

When I give my Minio to output_uri argument, it uploads 500 KB /sec as before.

  
  
Posted 3 years ago

The latest code you've posted uplods to local.
I need to upload to my local Minio

  
  
Posted 3 years ago

None of them is problematic, this is what I'm trying to say 🙂
I think the minio browser gets confused.
if you want to test the upload time on the client you can try:
task.flush(wait_for_uploads=True) tic = time() task.upload_artifact('test', '/tmp/localfile') task.flush(wait_for_uploads=True) print(time() - tic)

  
  
Posted 3 years ago

artifact_object is local file in my case. I did not understand what you suggest

  
  
Posted 3 years ago

What if I register the artifact manually?

  
  
Posted 3 years ago

Thanks MuddyCrab47 !!!
I found it!
It turns out the artifact upload will always upload from stream (aka no multi-upload). I will make sure we fix it in the next RC (I think the plan is to have it out this weekend)

  
  
Posted 3 years ago
500 Views
30 Answers
3 years ago
one year ago
Tags