Should GrpcClient use a segmented buffer for large messages?

[fabianoliver]

Is your feature request related to a problem? Please describe.

• Grpc clients read messages into a buffer rented from ArrayPool.Shared, see here
• ArrayPool.Shared only pools arrays of up to 1024*1024 elements. Meaning, any message larger than 1MB is stored in a newly allocated array that lives outside of the array pool
• To make matters worse, buffers in those size regions immediately end up on the LOH
• That can cause a lot of LOH pollution for clients that deal with large messages

Describe the solution you'd like

• Instead of trying to load everything into one big buffer, the client should possibly try to read these into chunks, each chunk no larger than 1mb and rented from the array pool
  (- Potentially, it'd also be similar to leverage Microsoft.IO.RecylableMemoryStream here, though I imagine you may not want to add a dependency)
• The buffered result is already now passed down as a ReadOnlySequence of bytes already, which should work well here

Describe alternatives you've considered

• Services can somewhat circumvent this problem by allowing to deliver large data spread across several messages
• This solution is not very ideal though as

1. Every endpoint/message needs explicit support for it
2. It doesn't work for unary calls
3. Payload size is oftentimes not easy to predict, so it's not trivial for client/server to choose a sensible model that guarantees messages are close to, but just below 1mb