Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi, I'M Trying To Upload An Artifact To My Connected Azure Storage Container With

Hi, I'm trying to upload an artifact to my connected Azure Storage Container with
` from trains import Task

task = Task.init(project_name="Test", task_name="test")
task.upload_artifact('TestArtifact','test.png') but I face this error: 2020-11-25 22:21:50,130 - trains.storage - ERROR - Exception encountered while uploading Failed uploading object test.blob.core.windows.net/trains/Test/test.74965cb1b53b4663af34b84d3f3d7984/artifacts/TestArtifact/test.png (400): <?xml version="1.0" encoding="utf-8"?><Error><Code>InvalidQueryParameterValue</Code><Message>Value for one of the query parameters specified in the request URI is invalid. I have added the default_uri ` to my azure storage as: https://test.blob.core.windows.net/trains and provided the keys in trains.conf. Can you tell me what am I missing? Thanks

  
  
Posted 3 years ago
Votes Newest

Answers 30


I tried setting the variables with export but got this error:
Traceback (most recent call last): File "test.py", line 1, in <module> from trains import Task File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/__init__.py", line 4, in <module> from .task import Task File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/task.py", line 28, in <module> from .backend_interface.metrics import Metrics File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/backend_interface/__init__.py", line 2, in <module> from .task import Task File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/backend_interface/task/__init__.py", line 1, in <module> from .task import Task File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/backend_interface/task/task.py", line 29, in <module> from ...binding.artifacts import Artifacts File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/binding/artifacts.py", line 25, in <module> from ..backend_interface.metrics.events import UploadEvent File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/backend_interface/metrics/__init__.py", line 2, in <module> from .interface import Metrics File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/backend_interface/metrics/interface.py", line 16, in <module> from ...storage.helper import StorageHelper File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/storage/__init__.py", line 2, in <module> from .manager import StorageManager File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/storage/manager.py", line 12, in <module> from .cache import CacheManager File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/storage/cache.py", line 7, in <module> from .helper import StorageHelper File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/storage/helper.py", line 127, in <module> class StorageHelper(object): File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/storage/helper.py", line 219, in StorageHelper _azure_configurations = AzureContainerConfigurations.from_config(config.get('azure.storage', {})) File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/backend_config/bucket_config.py", line 323, in from_config container_configs = [AzureContainerConfig(**entry) for entry in containers] + default_container_configs File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/backend_config/bucket_config.py", line 323, in <listcomp> container_configs = [AzureContainerConfig(**entry) for entry in containers] + default_container_configs TypeError: __init__() missing 2 required positional arguments: 'account_name' and 'account_key'

  
  
Posted 3 years ago

I did that, and it works flawlessly. I swear I installed the Azure blob storage package multiple times, anyway thanks a lot again for the detailed debugging O:)

  
  
Posted 3 years ago

No, those env variables aren't set.

  
  
Posted 3 years ago

But ValueError: Could not get access credentials for ' azure://trains.blob.core.windows.net/trains ' , check configuration file ~/trains.conf
basically means Trains failed to find the configured credentials in your trains.conf for azure://trains.blob.core.windows.net/trains (or whatever it was before you changed that for security reasons 🙂 )

  
  
Posted 3 years ago

Should we also provide credentials for the Storage Account on the Web UI under 'Profile' section?

  
  
Posted 3 years ago

This is the whole error dump:
2020-12-03 13:31:27,296 - trains.storage - ERROR - Azure blob storage driver not found. Please install driver using "pip install 'azure.storage.blob>=2.0.1'" Traceback (most recent call last): File "test.py", line 3, in <module> task = Task.init(project_name="Test", task_name="debugging") File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/task.py", line 461, in init task.output_uri = cls.__default_output_uri File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/task.py", line 698, in output_uri raise ValueError("Could not get access credentials for '{}' " ValueError: Could not get access credentials for 'azure://<account-name>.blob.core.windows.net/trains' , check configuration file ~/trains.confI have installed the azure.storage.blob>=2.0.1' package but it gives the error anyway

  
  
Posted 3 years ago

It says container name is optional in azure.storage , if I don't provide the container name, and remove it from the end of default_uri , would that work?

  
  
Posted 3 years ago

This is very strange, I traced the code and the only thing that might cause this issue is if the configured account name (in sdk.azure.storage.contrainers...account_name is not identical to the the first host name part in the output_uri you're using (i.e. immediately following azure:// ).

  
  
Posted 3 years ago

Let's try to circumvent the issue by using the environment variables. Try setting AZURE_STORAGE_ACCOUNT and AZURE_STORAGE_KEY (with your account name and account key, respectively) and commenting out the configuration in trains.conf

  
  
Posted 3 years ago

Did you comment out your azure.stroage configuration in trains.conf ?

  
  
Posted 3 years ago

Ahh okay, commented out the whole thing and got the same error as earlier ( Could not get access credentials )

  
  
Posted 3 years ago

2020-12-03 13:31:27,296 - trains.storage - ERROR - Azure blob storage driver not found. Please install driver using "pip install 'azure.storage.blob>=2.0.1'" Traceback (most recent call last): File "test.py", line 3, in <module> task = Task.init(project_name="Test", task_name="debugging") File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/task.py", line 461, in init task.output_uri = cls.__default_output_uri File "/home/sam/VirtualEnvs/test/lib/python3.8/site-packages/trains/task.py", line 698, in output_uri raise ValueError("Could not get access credentials for '{}' " ValueError: Could not get access credentials for 'azure://<account-name>.blob.core.windows.net/trains' , check configuration file ~/trains.conf

  
  
Posted 3 years ago

Can I see the whole error again?

  
  
Posted 3 years ago

Ooops.... we missed the most obvious thing...

  
  
Posted 3 years ago

See my previous comments 🙂

  
  
Posted 3 years ago

Oh it worked! I did the pip install multiple times earlier, but to no avail. I think it's because of the env variables? Let me try to unset those and provide it within the trains.conf

  
  
Posted 3 years ago

Thanks a lot SuccessfulKoala55 🙂

  
  
Posted 3 years ago

Sure 😄

  
  
Posted 3 years ago

You can try removing the env vars and using trains.conf again

  
  
Posted 3 years ago

Sure, it was fun 😄

  
  
Posted 3 years ago

Also, I think this message is not entirely correct and was fixed in the upcoming version - please install
pip install 'azure.storage.blob==2.0.1

  
  
Posted 3 years ago

😄

  
  
Posted 3 years ago

Since Trains does not yet support the latest version of azure.storage.blob

  
  
Posted 3 years ago

Note you can't leave an empty {...} entry there... It can be an empty list but not a list with an empty dictionary...

  
  
Posted 3 years ago

Should we also provide credentials for the Storage Account on the Web UI under 'Profile' section?

I'll find out - haven't tried that in a long time 🙂

  
  
Posted 3 years ago

Should we also provide credentials for the Storage Account on the Web UI under 'Profile' section?

See https://allegro.ai/docs/webapp/webapp_profile/#cloud-access-credentials
To get the SAS access token, go to the Azure Portal ( https://portal.azure.com ) >> storage accounts >> allegro >> shared access token >> set an expiration date in the far future >> create token >> copy "SAS token"

  
  
Posted 3 years ago

SmarmyDolphin68 I have an idea - can you check if the AZURE_STORAGE_ACCOUNT and AZURE_STORAGE_KEY environment variables are set in your environment? Trains will use these as overrides, maybe that's the issue

  
  
Posted 3 years ago

This is what I get with the ' https:// ' , this is atleast getting a response from Azure
2020-12-03 13:48:49,667 - trains.Task - INFO - No repository found, storing script code instead TRAINS results page: http://<IP>:8080/projects/<hash>/output/log 2020-12-03 13:48:51,505 - trains.Task - INFO - Waiting for repository detection and full package requirement analysis 2020-12-03 13:48:53,315 - trains.Task - INFO - Finished repository detection and package analysis 2020-12-03 13:48:53,315 - trains.Task - INFO - Waiting to finish uploads 2020-12-03 13:48:57,061 - trains.storage - ERROR - Exception encountered while uploading Failed uploading object <account-name>.blob.core.windows.net/trains/Test/debugging.<hash>/artifacts/Model_PNG/Model_PNG.txt (400): <?xml version="1.0" encoding="utf-8"?><Error><Code>InvalidQueryParameterValue</Code><Message>Value for one of the query parameters specified in the request URI is invalid. RequestId:<hash> Time:2020-12-03T08:18:57.0548988Z</Message><QueryParameterName>comp</QueryParameterName><QueryParameterValue /><Reason /></Error> 2020-12-03 13:48:57,061 - trains.metrics - WARNING - Failed uploading to https://<account-name>.blob.core.windows.net/trains (Failed uploading object myelintrains.blob.core.windows.net/trains/Test/debugging.65025e11f2a64d2a8a84554868d5686a/artifacts/Model_PNG/Model_PNG.txt (400): <?xml version="1.0" encoding="utf-8"?><Error><Code>InvalidQueryParameterValue</Code><Message>Value for one of the query parameters specified in the request URI is invalid. RequestId:b02b48a2-b01e-002b-234c-c9c824000000 Time:2020-12-03T08:18:57.0548988Z</Message><QueryParameterName>comp</QueryParameterName><QueryParameterValue /><Reason /></Error>) 2020-12-03 13:48:57,062 - trains.metrics - ERROR - Not uploading 1/1 events because the data upload failed 2020-12-03 13:48:57,071 - trains.Task - INFO - Finished uploading

  
  
Posted 3 years ago

I'm not sure regarding the env vars - it should have worked with the config file too (that's how it works on my machine)

  
  
Posted 3 years ago

2020-12-03 13:31:27,296 - trains.storage - ERROR - Azure blob storage driver not found. Please install driver using "pip install 'azure.storage.blob>=2.0.1'"

  
  
Posted 3 years ago