Hello Friends! I Am Trying To Play Around With The Configs For

Answered

Hello friends! I am trying to play around with the configs for gRPC for the triton server for clearml-serving .

I’m using the docker-compose setup, so not sure if the CLI will work for my usecase (perhaps passing them as env variables would work?)

For instance, I’d like to set some variables like this:

('grpc.max_send_message_length', 512 * 1024 * 1024), ('grpc.max_receive_message_length', 512 * 1024 * 1024)]

Is this possible currently? I’m getting an error from gRPC that my payload is more than the limit (8MB instead of 4MB…)

  				
Posted 
	one year ago

					More
				  		
  Report
		
					TimelyRabbit96
				
					0
					 × 1

Votes Newest

Answers 4

Yes exactly, that’d be great! I’m not sure how flexible this can be (or should be), but perhaps following a pattern like CLEARML_GRPC_CONFIG_NAME would be possible?

  				
Posted 
	one year ago

					More
				  		
  Report
		
					TimelyRabbit96
				
					0
					 × 1

yes exactly my thinking 👍

  				
Posted 
	one year ago

					More
				  		
  Report
		
					AgitatedDove14
				
					0
					 × 1

Hi @<1547028116780617728:profile|TimelyRabbit96>
You are absolutely correct, we need to allow to override configuration
The code you want to change is here:
None
You can try:

channel = self._ext_grpc.aio.insecure_channel(triton_server_address, options=dict([('grpc.max_send_message_length', 512 * 1024 * 1024),  ('grpc.max_receive_message_length', 512 * 1024 * 1024)]))

But I think we need to add it to the environment overrides options,
Maybe CLEARML_GRPC_max_send_message_length
wdyt?

  				
Posted 
	one year ago

					More
				  		
  Report
		
					AgitatedDove14
				
					0
					 × 1

We are working on 1.3.0 so this is right in time

  				
Posted 
	one year ago

					More
				  		
  Report
		
					AgitatedDove14
				
					0
					 × 1

Write your answer

672 Views

4 Answers

one year ago