The Problem:
When using the OpenTelemetry SDK with a custom configuration to export traces to Honeycomb through the OTLP HTTP exporter, the spans fail to export with the error "FRAME_SIZE_ERROR: 4740180". The issue persists despite using different versions of the OTLP collector and adjusting the collector configuration.
The Solutions:
Solution 1: Check the port number
The error message “FRAME_SIZE_ERROR: 4740180” indicates that the request to export spans failed due to a frame size error. This error can occur when the gRPC request size exceeds the maximum allowed frame size. To resolve this issue, you should check the port number you are using for the OpenTelemetry gRPC exporter.
In your case, you were initially using port 4317, which is the internal container port. However, the external port that you have configured in your docker-compose.yaml file is 4343. Therefore, you should update the endpoint for the OtlpGrpcSpanExporter to use the correct external port, as shown below:
OtlpGrpcSpanExporter otlpGrpcSpanExporter = OtlpGrpcSpanExporter.builder().setEndpoint("http://localhost:4343").build();
After making this change, you should be able to successfully export spans without encountering the frame size error.
Solution 2: Collector configuration check
The error message `Failed to export spans. The request could not be executed. Full error message: FRAME_SIZE_ERROR: 4740180` indicates that the size of the gRPC message being sent to the collector is too large. There are a few possible solutions to this issue:
- Increase the `max_recv_msg_size_mib` setting in the collector’s OTLP receiver configuration: This setting controls the maximum size of the gRPC messages that the collector will accept. You can increase this setting in the `otel-collector-config.yaml` file, under the `receivers.otlp.protocols.grpc` section.
- Enable compression on the exporter: This will reduce the size of the exported payload. You can enable compression on the OtlpGrpcSpanExporter using the `withCompression()` method.
- Adjust the settings of your batch span processor: The batch span processor controls how often spans are exported to the collector. You can adjust the settings of the batch span processor to send spans more frequently, which will reduce the size of each individual gRPC message.
Q&A
What’s the root cause of that error Failed to export spans. The request could not be executed. Full error message: FRAME_SIZE_ERROR: 4740180 ?
The gRPC request message size is too large.
What are some possible solutions to this error?
Solution could be enabling compression on the exporter, adjusting the batch span processor, or adjusting the OTLP receiver configuration.