Hello Friends! I Am Trying To Play Around With The Configs For

Answered

Hello friends! I am trying to play around with the configs for gRPC for the triton server for clearml-serving .

I’m using the docker-compose setup, so not sure if the CLI will work for my usecase (perhaps passing them as env variables would work?)

For instance, I’d like to set some variables like this:

('grpc.max_send_message_length', 512 * 1024 * 1024), ('grpc.max_receive_message_length', 512 * 1024 * 1024)]

Is this possible currently? I’m getting an error from gRPC that my payload is more than the limit (8MB instead of 4MB…)

  				
Posted 
	2 years ago

					More  		
  Report
		
					TimelyRabbit96
				
					0
					 × 1

Votes Newest

Answers 4

Yes exactly, that’d be great! I’m not sure how flexible this can be (or should be), but perhaps following a pattern like CLEARML_GRPC_CONFIG_NAME would be possible?

  				
Posted 
	2 years ago

					More  		
  Report
		
					TimelyRabbit96
				
					0
					 × 1

We are working on 1.3.0 so this is right in time

  				
Posted 
	2 years ago

					More  		
  Report
		
					AgitatedDove14
				
					0
					 × 1

yes exactly my thinking 👍

  				
Posted 
	2 years ago

					More  		
  Report
		
					AgitatedDove14
				
					0
					 × 1

Hi TimelyRabbit96
You are absolutely correct, we need to allow to override configuration
The code you want to change is here:
None
You can try:

channel = self._ext_grpc.aio.insecure_channel(triton_server_address, options=dict([('grpc.max_send_message_length', 512 * 1024 * 1024),  ('grpc.max_receive_message_length', 512 * 1024 * 1024)]))

But I think we need to add it to the environment overrides options,
Maybe CLEARML_GRPC_max_send_message_length
wdyt?

  				
Posted 
	2 years ago

					More  		
  Report
		
					AgitatedDove14
				
					0
					 × 1

Write your answer

1K Views

4 Answers

2 years ago