Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Is Clearml-Serving Using Either System Or Cuca Shared Memory? Or Planning To? In Our Experiments Using Perf_Analyzer The Shared Memory Experiments Showed A Huge Improvement And If We Wanted To Look Into This, Do You Have Any Pointers Of Where We Can Do T

Is ClearML-Serving using either System or CUCA shared memory? Or planning to? In our experiments using perf_analyzer the shared memory experiments showed a huge improvement

And if we wanted to look into this, do you have any pointers of where we can do the whole registering/transferring of data into shmem?

  
  
Posted 8 months ago
Votes Newest

Answers 6


Sorry @<1657918706052763648:profile|SillyRobin38> I missed this reply

Is ClearML-Serving using either System or CUCA shared memory? O

This needs to be set on the docker-compose:
and I think this line actually includes ipc: host which means there is no need to set the shm_size, but you can play around with it and let me know if you see a difference
None

  
  
Posted 7 months ago

@<1523701205467926528:profile|AgitatedDove14> Thanks

  
  
Posted 7 months ago

@<1657918706052763648:profile|SillyRobin38> ^

  
  
Posted 7 months ago

@<1523701205467926528:profile|AgitatedDove14> Actually our meant is something like the following example from the triton client examples:

None

Does clearml has any example for using shared memory? or it's out of context for clearml?

  
  
Posted 7 months ago

Hi @<1547028116780617728:profile|TimelyRabbit96>
Notice that if running with docker compose you can pass an argument to the clearml triton container an use shared mem. You can do the same with the helm chart

  
  
Posted 8 months ago

I see, okay so using

shm_size: '2gb'

we still need to modify the infrence logic to register and input and output on shmem, no?

  
  
Posted 8 months ago
615 Views
6 Answers
8 months ago
7 months ago
Tags