Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Can I Upload Files That Are Created On The Remote Machine? So Far It Just Get Stuck (The Files Are Small 10 Kb Or So) I Have A Script That Creates A Folder With Some Files, Some Of Them I Want To Keep, So I Tried Uploading To Storage, No Luck.

Can I upload files that are created on the remote machine? So far it just get stuck (the files are small 10 kb or so)
I have a script that creates a folder with some files, some of them I want to keep, so I tried uploading to storage, no luck.

  
  
Posted 4 years ago
Votes Newest

Answers 30


When this code is running on your machine, does it work?

  
  
Posted 4 years ago

Hi GleamingGiraffe20 ,

Without adding Task.init , i’m getting some OSError: [Errno 9] Bad file descriptor error, do you get those too?

Do you run your script from CLI or IDE (pycharm maybe?)?

  
  
Posted 4 years ago

Can you send me the logs with and without? (you can send the logs in DM if you prefer)

  
  
Posted 4 years ago

The files were crearted, and the one I needed uploaded to storage.

  
  
Posted 4 years ago

I agree - I just want to make sure I understand the exact scenario when it does đŸ™‚

  
  
Posted 4 years ago

Can you point me to a specific example?

  
  
Posted 4 years ago

You can probably replicate this yourself. I'm using simple transformers library for quick benchmarking.
Just run one his examples (I'm using multilabel clasification): https://github.com/ThilinaRajapakse/simpletransformers but change the output_dir to something else. when you don't track this works (no task.init), if you track this get stuck.

  
  
Posted 4 years ago

Hi GleamingGiraffe20 ,

The example in the documentation is missing a filename at the end of remote URL (an error I gotten locally when I tried to upload).

In https://allegro.ai/docs/examples/examples_storagehelper/#uploading-a-file example, the filename is /mnt/data/also_file.ext , did I miss the example you talked about? If so, can you send a link to it?

When using a trains task to track my run AND changing my scripts output directory, I get: ‘TRAINS Monitor: Could not detect iteration reporting, falling back to iterations as seconds-from-start’ - and that’s it, it’s stuck on running and no error. When running the same script without the trains task it works.

changing my scripts output directory can you send a small example of that? did you changed the output_uri ?

  
  
Posted 4 years ago

SuccessfulKoala55 Is this example correct:
https://allegro.ai/docs/examples/examples_storagehelper/#uploading-a-file

manager.upload_file(local_file="/mnt/data/also_file.ext", remote_url="s3://MyBucket/MyFolder")

  
  
Posted 4 years ago

Hi GleamingGiraffe20 , still getting those errors?

  
  
Posted 4 years ago

SuccessfulKoala55 Conclusions:
The example in the documentation is missing a filename at the end of remote URL (an error I gotten locally when I tried to upload). When using a trains task to track my run AND changing my scripts output directory, I get: 'TRAINS Monitor: Could not detect iteration reporting, falling back to iterations as seconds-from-start' - and that's it, it's stuck on running and no error. When running the same script without the trains task it works.

  
  
Posted 4 years ago

The error doesn't appear when not using the storage manager.

  
  
Posted 4 years ago

No error.
I didn't check the contents on the remote machine. However, when you run it locally it creates a bunch of files (text, model etc.)

  
  
Posted 4 years ago

yes, when I comment out the storage manager no error.

  
  
Posted 4 years ago

Remember the script is running on the remote machine as well, and the upload_file function will always try to upload the file. It's meant as a utility function you can use for uploading files, but it does not care if you're running locally or remotely

  
  
Posted 4 years ago

Were you able to replicate the issue with task?

  
  
Posted 4 years ago

manager.upload_file(local_file=str(output_dir/"config.json"), remote_url=remote_url+'bucket/')This is my specific upload. I wanted to make sure the example in the documentation is accurate.

  
  
Posted 4 years ago

How do you load the file? Can you find this file manually?

  
  
Posted 4 years ago

Locally I had no issues finding loading etc., didn't try to load it into memory remotely. The thing is, since this is stuck at training (with msg ‘TRAINS Monitor: Could not detect iteration reporting, falling back to iterations as seconds-from-start’) I will probably have no idea whether it was loaded or not.

  
  
Posted 4 years ago

For 'TRAINS Monitor: Could not detect iteration reporting, falling back to iterations as seconds-from-start' message - the iteration reporting is automatically detected if you are using tensorboard , matplotlib , or explicitly with trains.Logger
Assuming there were no reports, so the monitoring falls back to report every 30 seconds.

Thanks for the examples, will try to reproduce it now.

  
  
Posted 4 years ago

I'm not specifying a filename under remote URL, just like in the example.

  
  
Posted 4 years ago

I'll check.
I do however expect to see an error message when something isn't working, this just got stuck.

  
  
Posted 4 years ago

TimelyPenguin76 good morning,
From the CLI. Yes, I see it.

  
  
Posted 4 years ago

manager.upload_file(local_file="/mnt/data/also_file.ext", remote_url="s3://MyBucket/MyFolder")

  
  
Posted 4 years ago

WackyRabbit7

  
  
Posted 4 years ago

When you are not using the StorageManager you don’t get the OSError: [Errno 9] Bad file descriptor errors?

  
  
Posted 4 years ago

GleamingGiraffe20 when you run on a remote machine, is there a file in /mnt/data/also_file.ext ?

  
  
Posted 4 years ago

This specific issue doesn't concern the output URI (although I did a run both with it and without) as I'm trying to load a config file that's being saved locally using manager.upload_file.

  
  
Posted 4 years ago