Should we also provide credentials for the Storage Account on the Web UI under 'Profile' section?

See https://allegro.ai/docs/webapp/webapp_profile/#cloud-access-credentials
To get the SAS access token, go to the Azure Portal ( https://portal.azure.com ) >> storage accounts >> allegro >> shared access token >> set an expiration date in the far future >> create token >> copy "SAS token"

SmarmyDolphin68 I have an idea - can you check if the AZURE_STORAGE_ACCOUNT and AZURE_STORAGE_KEY environment variables are set in your environment? Trains will use these as overrides, maybe that's the issue

Also, I think this message is not entirely correct and was fixed in the upcoming version - please install
pip install 'azure.storage.blob==2.0.1

Since Trains does not yet support the latest version of azure.storage.blob

See my previous comments 🙂

Oh it worked! I did the pip install multiple times earlier, but to no avail. I think it's because of the env variables? Let me try to unset those and provide it within the trains.conf

2020-12-03 13:31:27,296 - trains.storage - ERROR - Azure blob storage driver not found. Please install driver using "pip install 'azure.storage.blob>=2.0.1'" Traceback (most recent call last): File "test.py", line 3, in <module> task = Task.init(project_name="Test", task_name="debugging") File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/task.py", line 461, in init task.output_uri = cls.__default_output_uri File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/task.py", line 698, in output_uri raise ValueError("Could not get access credentials for '{}' " ValueError: Could not get access credentials for 'azure://<account-name>.blob.core.windows.net/trains' , check configuration file ~/trains.conf

No, those env variables aren't set.

Should we also provide credentials for the Storage Account on the Web UI under 'Profile' section?

This is what I get with the ' https:// ' , this is atleast getting a response from Azure
2020-12-03 13:48:49,667 - trains.Task - INFO - No repository found, storing script code instead TRAINS results page: http://<IP>:8080/projects/<hash>/output/log 2020-12-03 13:48:51,505 - trains.Task - INFO - Waiting for repository detection and full package requirement analysis 2020-12-03 13:48:53,315 - trains.Task - INFO - Finished repository detection and package analysis 2020-12-03 13:48:53,315 - trains.Task - INFO - Waiting to finish uploads 2020-12-03 13:48:57,061 - trains.storage - ERROR - Exception encountered while uploading Failed uploading object <account-name>.blob.core.windows.net/trains/Test/debugging.<hash>/artifacts/Model_PNG/Model_PNG.txt (400): <?xml version="1.0" encoding="utf-8"?><Error><Code>InvalidQueryParameterValue</Code><Message>Value for one of the query parameters specified in the request URI is invalid. RequestId:<hash> Time:2020-12-03T08:18:57.0548988Z</Message><QueryParameterName>comp</QueryParameterName><QueryParameterValue /><Reason /></Error> 2020-12-03 13:48:57,061 - trains.metrics - WARNING - Failed uploading to https://<account-name>.blob.core.windows.net/trains (Failed uploading object myelintrains.blob.core.windows.net/trains/Test/debugging.65025e11f2a64d2a8a84554868d5686a/artifacts/Model_PNG/Model_PNG.txt (400): <?xml version="1.0" encoding="utf-8"?><Error><Code>InvalidQueryParameterValue</Code><Message>Value for one of the query parameters specified in the request URI is invalid. RequestId:b02b48a2-b01e-002b-234c-c9c824000000 Time:2020-12-03T08:18:57.0548988Z</Message><QueryParameterName>comp</QueryParameterName><QueryParameterValue /><Reason /></Error>) 2020-12-03 13:48:57,062 - trains.metrics - ERROR - Not uploading 1/1 events because the data upload failed 2020-12-03 13:48:57,071 - trains.Task - INFO - Finished uploading

Thanks a lot SuccessfulKoala55 🙂

This is the whole error dump:
2020-12-03 13:31:27,296 - trains.storage - ERROR - Azure blob storage driver not found. Please install driver using "pip install 'azure.storage.blob>=2.0.1'" Traceback (most recent call last): File "test.py", line 3, in <module> task = Task.init(project_name="Test", task_name="debugging") File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/task.py", line 461, in init task.output_uri = cls.__default_output_uri File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/task.py", line 698, in output_uri raise ValueError("Could not get access credentials for '{}' " ValueError: Could not get access credentials for 'azure://<account-name>.blob.core.windows.net/trains' , check configuration file ~/trains.confI have installed the azure.storage.blob>=2.0.1' package but it gives the error anyway

Ahh okay, commented out the whole thing and got the same error as earlier ( Could not get access credentials )

I did that, and it works flawlessly. I swear I installed the Azure blob storage package multiple times, anyway thanks a lot again for the detailed debugging O:)

2020-12-03 13:31:27,296 - trains.storage - ERROR - Azure blob storage driver not found. Please install driver using "pip install 'azure.storage.blob>=2.0.1'"

Sure 😄

I'm not sure regarding the env vars - it should have worked with the config file too (that's how it works on my machine)

Can I see the whole error again?

Let's try to circumvent the issue by using the environment variables. Try setting AZURE_STORAGE_ACCOUNT and AZURE_STORAGE_KEY (with your account name and account key, respectively) and commenting out the configuration in trains.conf

I tried setting the variables with export but got this error:
Traceback (most recent call last): File "test.py", line 1, in <module> from trains import Task File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/__init__.py", line 4, in <module> from .task import Task File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/task.py", line 28, in <module> from .backend_interface.metrics import Metrics File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/backend_interface/__init__.py", line 2, in <module> from .task import Task File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/backend_interface/task/__init__.py", line 1, in <module> from .task import Task File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/backend_interface/task/task.py", line 29, in <module> from ...binding.artifacts import Artifacts File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/binding/artifacts.py", line 25, in <module> from ..backend_interface.metrics.events import UploadEvent File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/backend_interface/metrics/__init__.py", line 2, in <module> from .interface import Metrics File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/backend_interface/metrics/interface.py", line 16, in <module> from ...storage.helper import StorageHelper File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/storage/__init__.py", line 2, in <module> from .manager import StorageManager File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/storage/manager.py", line 12, in <module> from .cache import CacheManager File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/storage/cache.py", line 7, in <module> from .helper import StorageHelper File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/storage/helper.py", line 127, in <module> class StorageHelper(object): File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/storage/helper.py", line 219, in StorageHelper _azure_configurations = AzureContainerConfigurations.from_config(config.get('azure.storage', {})) File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/backend_config/bucket_config.py", line 323, in from_config container_configs = [AzureContainerConfig(**entry) for entry in containers] + default_container_configs File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/backend_config/bucket_config.py", line 323, in <listcomp> container_configs = [AzureContainerConfig(**entry) for entry in containers] + default_container_configs TypeError: __init__() missing 2 required positional arguments: 'account_name' and 'account_key'

Sure, it was fun 😄

Note you can't leave an empty {...} entry there... It can be an empty list but not a list with an empty dictionary...

But ValueError: Could not get access credentials for ' azure://trains.blob.core.windows.net/trains ' , check configuration file ~/trains.conf
basically means Trains failed to find the configured credentials in your trains.conf for azure://trains.blob.core.windows.net/trains (or whatever it was before you changed that for security reasons 🙂 )

Should we also provide credentials for the Storage Account on the Web UI under 'Profile' section?

I'll find out - haven't tried that in a long time 🙂

😄

This is very strange, I traced the code and the only thing that might cause this issue is if the configured account name (in sdk.azure.storage.contrainers...account_name is not identical to the the first host name part in the output_uri you're using (i.e. immediately following azure:// ).

You can try removing the env vars and using trains.conf again

It says container name is optional in azure.storage , if I don't provide the container name, and remove it from the end of default_uri , would that work?

Ooops.... we missed the most obvious thing...

Did you comment out your azure.stroage configuration in trains.conf ?