I Am Looking For A "Hello World" Example To Do 3 Tasks: Data=Preprocessdata() //Local Processing Model=Trainmodel(Data) // On Cloud Processing On A Custom Container Say Using Pytorch Or Keras Results=Evaluatemodel(Model, Testdata) //Local Processing

Answered

I am looking for a "hello world" example to do 3 Tasks:
data=preprocessData() //local processing
model=trainModel(data) // on cloud processing on a custom container say using pytorch or keras
results=evaluateModel(model, testdata) //local processing

  				
Posted 
	4 years ago

					More
				  		
  Report
		
					LazyLeopard18
				
					0
					 × 1

Votes Newest

Answers 3

I have ~100GB of data that I do not wish to upload to the trains-server. Instead, I would like to have them copied only to host machine (azure container) at training time.
The data is in Azure blob storage and will be copied using a custom script just before training starts.

  				
Posted 
	4 years ago

					More
				  		
  Report
		
					LazyLeopard18
				
					0
					 × 1

Hi LazyLeopard18
I think that these toy examples will help:
uploading local datasethttps://github.com/allegroai/events/blob/master/odsc20-east/generic/dataset_artifact.py
2. pre-process data
https://github.com/allegroai/events/blob/master/odsc20-east/generic/process_dataset.py
3. Training example:
https://github.com/allegroai/events/blob/master/odsc20-east/scikit-learn/sklearn_jupyter.ipynb

  				
Posted 
	4 years ago

					More
				  		
  Report
		
					AgitatedDove14
				
					0
					 × 1

LazyLeopard18 you can point the artifact directly to your azure object storage and have StorageManager download and cache it for you:

  				
Posted 
	4 years ago

					More
				  		
  Report
		
					AgitatedDove14
				
					0
					 × 1

Write your answer

1K Views

3 Answers

4 years ago

one year ago