SuccessfulKoala55 Is this example correct:
https://allegro.ai/docs/examples/examples_storagehelper/#uploading-a-file
manager.upload_file(local_file="/mnt/data/also_file.ext", remote_url="s3://MyBucket/MyFolder")
No error.
I didn't check the contents on the remote machine. However, when you run it locally it creates a bunch of files (text, model etc.)
manager.upload_file(local_file=str(output_dir/"config.json"), remote_url=remote_url+'bucket/')This is my specific upload. I wanted to make sure the example in the documentation is accurate.
You can probably replicate this yourself. I'm using simple transformers library for quick benchmarking.
Just run one his examples (I'm using multilabel clasification): https://github.com/ThilinaRajapakse/simpletransformers but change the output_dir to something else. when you don't track this works (no task.init), if you track this get stuck.
TimelyPenguin76 good morning,
From the CLI. Yes, I see it.
Were you able to replicate the issue with task?
manager.upload_file(local_file="/mnt/data/also_file.ext", remote_url="s3://MyBucket/MyFolder")
I'm not specifying a filename under remote URL, just like in the example.
The error doesn't appear when not using the storage manager.
This specific issue doesn't concern the output URI (although I did a run both with it and without) as I'm trying to load a config file that's being saved locally using manager.upload_file.
I'll check.
I do however expect to see an error message when something isn't working, this just got stuck.
The files were crearted, and the one I needed uploaded to storage.
yes, when I comment out the storage manager no error.
If it's just a UI issue, than this is still unsolved.
https://github.com/ThilinaRajapakse/simpletransformers#minimal-start-for-multilabel-classification that's what I'm using (just with DistilBERT)
Locally I had no issues finding loading etc., didn't try to load it into memory remotely. The thing is, since this is stuck at training (with msg ‘TRAINS Monitor: Could not detect iteration reporting, falling back to iterations as seconds-from-start’) I will probably have no idea whether it was loaded or not.
SuccessfulKoala55 Conclusions:
The example in the documentation is missing a filename at the end of remote URL (an error I gotten locally when I tried to upload). When using a trains task to track my run AND changing my scripts output directory, I get: 'TRAINS Monitor: Could not detect iteration reporting, falling back to iterations as seconds-from-start' - and that's it, it's stuck on running and no error. When running the same script without the trains task it works.
I guess. maybe I'll just stash.
Not a big fan of notebooks X git so I attempt to push these less frequently.
The execution uses an older version that is committed.
SuccessfulKoala55 when running a changed script without adding committing and pushing the changes are not applied. I do have '# WARNING! git diff too large to store, clear this section to execute without it.' due to unsaved notebook changes.
I guess that's the issue?