SmarmyDolphin68 I have an idea - can you check if the AZURE_STORAGE_ACCOUNT and AZURE_STORAGE_KEY environment variables are set in your environment? Trains will use these as overrides, maybe that's the issue

Should we also provide credentials for the Storage Account on the Web UI under 'Profile' section?

😄

You can try removing the env vars and using trains.conf again

I tried setting the variables with export but got this error:
Traceback (most recent call last): File "test.py", line 1, in <module> from trains import Task File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/__init__.py", line 4, in <module> from .task import Task File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/task.py", line 28, in <module> from .backend_interface.metrics import Metrics File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/backend_interface/__init__.py", line 2, in <module> from .task import Task File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/backend_interface/task/__init__.py", line 1, in <module> from .task import Task File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/backend_interface/task/task.py", line 29, in <module> from ...binding.artifacts import Artifacts File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/binding/artifacts.py", line 25, in <module> from ..backend_interface.metrics.events import UploadEvent File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/backend_interface/metrics/__init__.py", line 2, in <module> from .interface import Metrics File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/backend_interface/metrics/interface.py", line 16, in <module> from ...storage.helper import StorageHelper File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/storage/__init__.py", line 2, in <module> from .manager import StorageManager File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/storage/manager.py", line 12, in <module> from .cache import CacheManager File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/storage/cache.py", line 7, in <module> from .helper import StorageHelper File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/storage/helper.py", line 127, in <module> class StorageHelper(object): File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/storage/helper.py", line 219, in StorageHelper _azure_configurations = AzureContainerConfigurations.from_config(config.get('azure.storage', {})) File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/backend_config/bucket_config.py", line 323, in from_config container_configs = [AzureContainerConfig(**entry) for entry in containers] + default_container_configs File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/backend_config/bucket_config.py", line 323, in <listcomp> container_configs = [AzureContainerConfig(**entry) for entry in containers] + default_container_configs TypeError: __init__() missing 2 required positional arguments: 'account_name' and 'account_key'

I'm not sure regarding the env vars - it should have worked with the config file too (that's how it works on my machine)

2020-12-03 13:31:27,296 - trains.storage - ERROR - Azure blob storage driver not found. Please install driver using "pip install 'azure.storage.blob>=2.0.1'"

Ooops.... we missed the most obvious thing...

No, those env variables aren't set.

Also, I think this message is not entirely correct and was fixed in the upcoming version - please install
pip install 'azure.storage.blob==2.0.1

Should we also provide credentials for the Storage Account on the Web UI under 'Profile' section?

I'll find out - haven't tried that in a long time 🙂

Note you can't leave an empty {...} entry there... It can be an empty list but not a list with an empty dictionary...

Should we also provide credentials for the Storage Account on the Web UI under 'Profile' section?

See https://allegro.ai/docs/webapp/webapp_profile/#cloud-access-credentials
To get the SAS access token, go to the Azure Portal ( https://portal.azure.com ) >> storage accounts >> allegro >> shared access token >> set an expiration date in the far future >> create token >> copy "SAS token"

This is the whole error dump:
2020-12-03 13:31:27,296 - trains.storage - ERROR - Azure blob storage driver not found. Please install driver using "pip install 'azure.storage.blob>=2.0.1'" Traceback (most recent call last): File "test.py", line 3, in <module> task = Task.init(project_name="Test", task_name="debugging") File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/task.py", line 461, in init task.output_uri = cls.__default_output_uri File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/task.py", line 698, in output_uri raise ValueError("Could not get access credentials for '{}' " ValueError: Could not get access credentials for 'azure://<account-name>.blob.core.windows.net/trains' , check configuration file ~/trains.confI have installed the azure.storage.blob>=2.0.1' package but it gives the error anyway

See my previous comments 🙂

Since Trains does not yet support the latest version of azure.storage.blob

I did that, and it works flawlessly. I swear I installed the Azure blob storage package multiple times, anyway thanks a lot again for the detailed debugging O:)

Ahh okay, commented out the whole thing and got the same error as earlier ( Could not get access credentials )

2020-12-03 13:31:27,296 - trains.storage - ERROR - Azure blob storage driver not found. Please install driver using "pip install 'azure.storage.blob>=2.0.1'" Traceback (most recent call last): File "test.py", line 3, in <module> task = Task.init(project_name="Test", task_name="debugging") File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/task.py", line 461, in init task.output_uri = cls.__default_output_uri File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/task.py", line 698, in output_uri raise ValueError("Could not get access credentials for '{}' " ValueError: Could not get access credentials for 'azure://<account-name>.blob.core.windows.net/trains' , check configuration file ~/trains.conf

This is very strange, I traced the code and the only thing that might cause this issue is if the configured account name (in sdk.azure.storage.contrainers...account_name is not identical to the the first host name part in the output_uri you're using (i.e. immediately following azure:// ).

Can I see the whole error again?

But ValueError: Could not get access credentials for ' azure://trains.blob.core.windows.net/trains ' , check configuration file ~/trains.conf
basically means Trains failed to find the configured credentials in your trains.conf for azure://trains.blob.core.windows.net/trains (or whatever it was before you changed that for security reasons 🙂 )

Oh it worked! I did the pip install multiple times earlier, but to no avail. I think it's because of the env variables? Let me try to unset those and provide it within the trains.conf

Let's try to circumvent the issue by using the environment variables. Try setting AZURE_STORAGE_ACCOUNT and AZURE_STORAGE_KEY (with your account name and account key, respectively) and commenting out the configuration in trains.conf

Sure, it was fun 😄

This is what I get with the ' https:// ' , this is atleast getting a response from Azure
2020-12-03 13:48:49,667 - trains.Task - INFO - No repository found, storing script code instead TRAINS results page: http://<IP>:8080/projects/<hash>/output/log 2020-12-03 13:48:51,505 - trains.Task - INFO - Waiting for repository detection and full package requirement analysis 2020-12-03 13:48:53,315 - trains.Task - INFO - Finished repository detection and package analysis 2020-12-03 13:48:53,315 - trains.Task - INFO - Waiting to finish uploads 2020-12-03 13:48:57,061 - trains.storage - ERROR - Exception encountered while uploading Failed uploading object <account-name>.blob.core.windows.net/trains/Test/debugging.<hash>/artifacts/Model_PNG/Model_PNG.txt (400): <?xml version="1.0" encoding="utf-8"?><Error><Code>InvalidQueryParameterValue</Code><Message>Value for one of the query parameters specified in the request URI is invalid. RequestId:<hash> Time:2020-12-03T08:18:57.0548988Z</Message><QueryParameterName>comp</QueryParameterName><QueryParameterValue /><Reason /></Error> 2020-12-03 13:48:57,061 - trains.metrics - WARNING - Failed uploading to https://<account-name>.blob.core.windows.net/trains (Failed uploading object myelintrains.blob.core.windows.net/trains/Test/debugging.65025e11f2a64d2a8a84554868d5686a/artifacts/Model_PNG/Model_PNG.txt (400): <?xml version="1.0" encoding="utf-8"?><Error><Code>InvalidQueryParameterValue</Code><Message>Value for one of the query parameters specified in the request URI is invalid. RequestId:b02b48a2-b01e-002b-234c-c9c824000000 Time:2020-12-03T08:18:57.0548988Z</Message><QueryParameterName>comp</QueryParameterName><QueryParameterValue /><Reason /></Error>) 2020-12-03 13:48:57,062 - trains.metrics - ERROR - Not uploading 1/1 events because the data upload failed 2020-12-03 13:48:57,071 - trains.Task - INFO - Finished uploading

Sure 😄

Thanks a lot SuccessfulKoala55 🙂

Did you comment out your azure.stroage configuration in trains.conf ?

It says container name is optional in azure.storage , if I don't provide the container name, and remove it from the end of default_uri , would that work?