gRPC Message Size Limits and Large Payload Strategies

You add a field, a list grows, one customer uploads a bigger file than anyone planned for, and a call that worked for a year starts failing:

rpc error: code = ResourceExhausted desc = grpc: received message larger than max (5242885 vs. 4194304)

The number on the right is 4 MiB, the default receive limit in grpc-go. The number on the left is the size of the message that arrived. This is a fast error to fix and an easy one to fix in the wrong place, because there are four related limits and each belongs to a different side of the call. This article reproduces the error, lists the limits, shows the right fix for each case and then argues that raising a limit is usually the wrong long-term answer. It relates to streaming, performance and the HTTP-side equivalent in JSON size limits. It ran on Go 1.25.6.

Reproducing the error

The lab registers a small service by hand so it needs no code generation: Echo takes bytes and returns the length, Get returns a 5 MiB blob and Download streams the blob in chunks. Then it makes the same 5 MiB payload fail and succeed in turn.

package main

import (
	"context"
	"fmt"
	"io"
	"log"
	"net"

	"google.golang.org/grpc"
	"google.golang.org/grpc/credentials/insecure"
	"google.golang.org/grpc/status"
	"google.golang.org/protobuf/types/known/wrapperspb"
)

const mib = 1 << 20

// A hand-registered service so the lab needs no code generation: Echo returns what it receives; Download streams a blob in chunks.
func serviceDesc(blob []byte) *grpc.ServiceDesc {
	return &grpc.ServiceDesc{
		ServiceName: "blob.v1.Blob",
		HandlerType: (*any)(nil),
		Methods: []grpc.MethodDesc{{
			MethodName: "Echo",
			Handler: func(_ any, _ context.Context, dec func(any) error, _ grpc.UnaryServerInterceptor) (any, error) {
				in := new(wrapperspb.BytesValue)
				if err := dec(in); err != nil {
					return nil, err
				}
				return wrapperspb.Int64(int64(len(in.Value))), nil
			},
		}, {
			MethodName: "Get",
			Handler: func(_ any, _ context.Context, dec func(any) error, _ grpc.UnaryServerInterceptor) (any, error) {
				return wrapperspb.Bytes(blob), nil
			},
		}},
		Streams: []grpc.StreamDesc{{
			StreamName:    "Download",
			ServerStreams: true,
			Handler: func(_ any, stream grpc.ServerStream) error {
				if err := stream.RecvMsg(new(wrapperspb.StringValue)); err != nil {
					return err
				}
				for off := 0; off < len(blob); off += 256 << 10 {
					end := min(off+256<<10, len(blob))
					if err := stream.SendMsg(wrapperspb.Bytes(blob[off:end])); err != nil {
						return err
					}
				}
				return nil
			},
		}},
	}
}

func start(blob []byte, opts ...grpc.ServerOption) string {
	lis, err := net.Listen("tcp", "127.0.0.1:0")
	if err != nil {
		log.Fatal(err)
	}
	srv := grpc.NewServer(opts...)
	srv.RegisterService(serviceDesc(blob), struct{}{})
	go srv.Serve(lis)
	return lis.Addr().String()
}

func dial(addr string, opts ...grpc.DialOption) *grpc.ClientConn {
	conn, err := grpc.NewClient(addr, append([]grpc.DialOption{grpc.WithTransportCredentials(insecure.NewCredentials())}, opts...)...)
	if err != nil {
		log.Fatal(err)
	}
	return conn
}

func main() {
	ctx := context.Background()
	blob := make([]byte, 5*mib)
	report := func(label string, err error) {
		if err == nil {
			fmt.Printf("%-52s OK\n", label)
			return
		}
		fmt.Printf("%-52s %s: %s\n", label, status.Code(err), status.Convert(err).Message())
	}

	// 1. The server's receive limit (4 MiB by default) rejects a 5 MiB request.
	addr := start(blob)
	conn := dial(addr)
	err := conn.Invoke(ctx, "/blob.v1.Blob/Echo", wrapperspb.Bytes(blob), new(wrapperspb.Int64Value))
	report("5 MiB request, default server limit", err)

	// 2. Raising the server's receive limit fixes the request side.
	addr = start(blob, grpc.MaxRecvMsgSize(8*mib))
	conn = dial(addr)
	err = conn.Invoke(ctx, "/blob.v1.Blob/Echo", wrapperspb.Bytes(blob), new(wrapperspb.Int64Value))
	report("5 MiB request, server MaxRecvMsgSize 8 MiB", err)

	// 3. The client has its own receive limit for responses (also 4 MiB by default).
	err = conn.Invoke(ctx, "/blob.v1.Blob/Get", wrapperspb.String(""), new(wrapperspb.BytesValue))
	report("5 MiB response, default client limit", err)
	err = conn.Invoke(ctx, "/blob.v1.Blob/Get", wrapperspb.String(""), new(wrapperspb.BytesValue), grpc.MaxCallRecvMsgSize(8*mib))
	report("5 MiB response, MaxCallRecvMsgSize 8 MiB", err)

	// 4. The send side has a limit too, which defaults to the largest possible size, so it only bites if you set it.
	conn = dial(addr, grpc.WithDefaultCallOptions(grpc.MaxCallSendMsgSize(1*mib)))
	err = conn.Invoke(ctx, "/blob.v1.Blob/Echo", wrapperspb.Bytes(blob), new(wrapperspb.Int64Value))
	report("5 MiB request, client MaxCallSendMsgSize 1 MiB", err)

	// 5. Stream it: the same 5 MiB in 256 KiB messages, with every limit left at its default.
	conn = dial(start(blob))
	stream, err := conn.NewStream(ctx, &grpc.StreamDesc{ServerStreams: true}, "/blob.v1.Blob/Download")
	if err != nil {
		log.Fatal(err)
	}
	stream.SendMsg(wrapperspb.String("go"))
	stream.CloseSend()
	total, chunks := 0, 0
	for {
		var chunk wrapperspb.BytesValue
		if err := stream.RecvMsg(&chunk); err == io.EOF {
			break
		} else if err != nil {
			report("streamed download", err)
			return
		}
		total += len(chunk.Value)
		chunks++
	}
	fmt.Printf("streamed download: %d chunks, %d bytes in total, default limits, no error\n", chunks, total)
}

5 MiB request, default server limit                  ResourceExhausted: grpc: received message larger than max (5242885 vs. 4194304)
5 MiB request, server MaxRecvMsgSize 8 MiB           OK
5 MiB response, default client limit                 ResourceExhausted: grpc: received message larger than max (5242885 vs. 4194304)
5 MiB response, MaxCallRecvMsgSize 8 MiB             OK
5 MiB request, client MaxCallSendMsgSize 1 MiB       ResourceExhausted: trying to send message larger than max (5242885 vs. 1048576)
streamed download: 20 chunks, 5242880 bytes in total, default limits, no error

The first line is the error. A client sent 5 MiB to a server with default settings, and the server rejected it before the handler ran: ResourceExhausted, with 5242885 vs. 4194304. The five bytes beyond 5,242,880 are the protobuf tag and length prefix around the payload.

The four limits

Two limits matter by default (the receive limits), and each is enforced by the side doing the reading. That is the reason a fix in one place seems not to work.

Fixing the request

The second line of output: with grpc.MaxRecvMsgSize(8*mib) on the server, the same request succeeds. This is the server's receive limit, so it is the only change needed when the request is too big.

Fixing the response

The third line shows the trap. The server now accepts big requests, but the call to Get, which returns 5 MiB, fails with the same message. The server sent it fine (the send limit is effectively unlimited), and the client rejected it, because the client's receive limit is 4 MiB. The fourth line passes grpc.MaxCallRecvMsgSize(8*mib) on that call and it works. Set it on one call, or for all calls on a connection with grpc.WithDefaultCallOptions(grpc.MaxCallRecvMsgSize(...)) at dial time.

When you see the error, read the log line carefully: which side printed it tells you which limit to change. A client-side error with received message larger than max means the response was too big.

Send limits

The fifth line shows the send side. A client configured with MaxCallSendMsgSize(1*mib) refused to send the 5 MiB request at all: trying to send message larger than max. Note the different wording, and note that it failed locally. Send limits are useful as a guard: a server can set MaxSendMsgSize so a bug that builds a gigantic response fails with a clear error in your logs instead of exhausting memory and bandwidth. By default there is none, so a runaway response is bounded only by memory.

Should you raise the limit?

Sometimes. A limit exists to protect memory: a gRPC message is read fully into memory and unmarshalled before your handler sees it, and the same on the way out. 100 concurrent calls with 50 MiB messages is 5 GB. Before raising a limit, ask three questions.

Is the size expected and bounded? A configuration blob that is occasionally 6 MiB is a reason to raise the limit to 8 or 16 MiB. A "list everything" response that grows without bound is a bug to fix with pagination (pagination), and raising the limit only moves the failure.

Who can send it? Raising the server's receive limit raises the memory an unauthenticated or hostile client can make you allocate per call. Keep it as low as your real needs, and apply it only on the services that need it. A per-method limit is not available, but you can put the large-message method in a separate server or reject oversize requests early in an interceptor using the message size.

What else is in the path? Proxies and load balancers have their own limits. Envoy's per-stream buffer limit, an ingress controller's request body size, and a cloud load balancer's maximum can all reject large messages with different errors (often Unavailable or Internal with a generic message), even when grpc-go's limits are fine. Check the whole path.

The better answer: do not send big messages

The sixth line is the recommended design. The same 5 MiB payload is sent as a server stream of 20 chunks of 256 KiB. Every limit stayed at its default, and nothing failed. Streaming has three advantages over big messages.

• Memory stays bounded. Each chunk is read, processed and discarded. A handler can process a 5 GB upload with a few hundred KiB of buffer.
• Backpressure works. A slow receiver slows the sender through flow control, as shown in server streaming.
• Progress and early abort. The receiver can start work on the first chunk and cancel when it has enough.

For uploads, use client streaming: the client sends chunks and the server assembles or forwards them; include a sequence number and a checksum in each chunk or in the final message so the receiver can detect gaps (client streaming). Pick a chunk size well under the limit; 64 KiB to 1 MiB is typical.

For truly large objects (hundreds of megabytes and up, media files, backups), gRPC is not the right carrier. Return a reference and let the client transfer the bytes over a transport made for bulk data: an HTTP download from object storage with a signed URL, or a presigned upload (presigned URLs). The RPC then carries metadata and the URL, a few hundred bytes, and storage and CDNs do the heavy lifting with resumable transfers and caching.

Checklist when the error appears

1. Read which side logged it and whether it says received or trying to send.
2. Identify the direction: request (server receive) or response (client receive).
3. Decide whether the size is legitimate. If it is unbounded, add pagination or streaming.
4. If raising a limit, raise only the one that is failing, to a value you can justify, and note it in the service's documentation.
5. Check proxies on the path for their own limits.
6. Add a test with a message just over the old limit, so a later change that lowers it fails loudly.
7. Monitor message sizes (the histogram from the OpenTelemetry stats handler records sizes if enabled) so you see growth before it becomes an error.

The limits are documented in the grpc package options, and the gRPC project's performance guide recommends streaming for large payloads.