Repository navigation
feat(storage): add OpenTelemetry tracing support for gRPC bidi APIs - #18397
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
Code Review
This pull request introduces OpenTelemetry tracing support for zonal buckets (Rapid Storage) in async gRPC classes. It refactors tracing span creation to support both synchronous and asynchronous context managers, injects traceparent metadata into gRPC requests, and updates the bucket metadata cache to support asynchronous gRPC fetches. The review feedback recommends defensively checking for __aenter__ and __aexit__ in _TraceSpanHelperContext to ensure compatibility with synchronous context managers (such as mocked spans in unit tests) and refactoring the project number extraction in update_from_bucket to use getattr more idiomatically.
shradhakatyal
force-pushed
the
feat/zb-traces
branch
from
September 17, 2026 09:03
50474ec to
e572fb0
Compare
shradhakatyal
marked this pull request as ready for review
September 28, 2026 05:26
cpriti-os
self-requested a review
September 28, 2026 06:02
shradhakatyal
force-pushed
the
feat/zb-traces
branch
from
September 29, 2026 09:37
bb092c8 to
6a01c38
Compare
Contributor
Author
|
/gcbrun 6a01c38 |
Contributor
|
@shradhakatyal , can you check the performance regression caused because of this PR - screen/228ingun6e470 |
Contributor
Author
|
/gcbrun (bd89f56) |
cpriti-os
requested changes
Oct 19, 2026
Contributor
Author
|
/gcbrun (bd89f56) |
Contributor
Author
|
/gcbrun (7b87f4b) |
Contributor
Author
Fixed, please check. |
Contributor
|
nit: modify the title to "feat(storage): add OpenTelemetry tracing support for gRPC bidi APIs" reason: grpc bidi includes Rapid Storage ( zonal buckets) & they also started supporting Standard buckets. |
cpriti-os
approved these changes
Nov 1, 2026
cpriti-os
left a comment
Contributor
There was a problem hiding this comment.
LGTM, thanks for addressing the feedback!
chandra-siri
requested changes
Nov 1, 2026
Contributor
Author
Done Co-authored by AI Agent |
Contributor
Author
Done! |
Contributor
Author
|
/gcbrun (1df0886) |
cpriti-os
reviewed
Nov 2, 2026
chandra-siri
approved these changes
Nov 2, 2026
shradhakatyal
requested review from
julianxu-goog
and removed request for
a team
November 3, 2026 08:23
…adata_cache.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
…te leakage across unit tests
[Generated-by: AI]
…ache - Add T2 trace spans to AsyncAppendableObjectWriter.simple_flush and finalize - Prevent nested T2 spans during internal stream reconnects and finalize_on_close - Simplify _is_otel_traces_enabled() and clean up fixture teardown - Avoid duplicate exception events on GoogleAPICallError in _TraceSpanContext - Track background asyncio tasks in BucketMetadataCache and refresh cache on GetBucket - Add unit tests for 403/404 cache handling and span error recording [Generated-by: AI]
… is disabled - Add _NoOpTraceSpanContext singleton and fast-path create_trace_span and create_trace_span_helper when tracing is disabled - Add __slots__ to _TraceSpanContext and _TraceSpanHelperContext and avoid async coroutine wrapping for sync span context managers - Replace @asynccontextmanager in AsyncAppendableObjectWriter._trace_span with slotted _WriterTraceSpanContext and no-op fast path - Fast-path inject_traceparent_to_metadata without copying metadata when disabled and cache TraceContextTextMapPropagator - Avoid duplicate cache lookup in BucketMetadataCache.get_or_queue_fetch [Generated-by: AI]
[Generated-by: AI]
…tracing [Generated-by: AI]
…ppend span Aligns the payload size attribute on Storage.AsyncAppendableObjectWriter.append with the cross-SDK trace attributes specification (gcp.storage.payload.size_bytes). [Generated-by: AI]
shradhakatyal
force-pushed
the
feat/zb-traces
branch
from
November 3, 2026 09:21
60fb1f6 to
2831c07
Compare
Contributor
Author
|
/gcbrun (2831c07) |
chandra-siri
added a commit
to chandra-siri/google-cloud-python
that referenced
this pull request
Nov 4, 2026
…with UDS PoC Merge origin/main (be426ad) into the gRPC-over-UDS PoC branch and resolve conflicts with googleapis#18641 (Preconditions), googleapis#18628 (off-loop CRC32C), googleapis#18627 (zero-copy BidiReadObjectResponse parsing) and googleapis#18397 (OpenTelemetry tracing for gRPC bidi APIs): - async_grpc_client: keep upstream OTel spans, traceparent injection, _is_async_grpc_client and _bucket_metadata_cache next to the private UDS constructor state, DirectPath sentinel, signature override and prewarm hook. - async_multi_range_downloader / async_appendable_object_writer: run the UDS hooks inside the upstream top-level trace spans so open/download_ranges/ close and open/append/flush/simple_flush/state_lookup/close/finalize emit the same spans in GAPIC and UDS modes. - UDS reads wrap destinations like the GAPIC path (buffer-protocol destinations filled in place with BufferError on short reads, custom write() destinations receive bytes). - UDS writes map Preconditions onto OpenAppendSpec (all four on create, metageneration ones on reopen) and stop re-sending them after the writer created the generation, mirroring _AsyncWriteObjectStream. - Refresh the public-API snapshot test to origin/main's signatures.
chandra-siri
added a commit
to chandra-siri/google-cloud-python
that referenced
this pull request
Nov 4, 2026
…eads - test_upstream_integration: UDS-mode OpenTelemetry spans for the MRD and appendable writer (googleapis#18397), Preconditions mapping onto OpenAppendSpec on create/overwrite/reopen and after a created generation (googleapis#18641), the buffer-destination contract of download_ranges (googleapis#18627), runtime fallback and the AsyncGrpcClient DirectPath sentinel / prewarm hooks. - Hermetic E2E (real gcs-core-server + fake GCS): Preconditions on create, overwrite and reopen, and download_ranges into bytearray/memoryview destinations over shared memory and inline transports. - New test_runtime / test_config and extended read/write session, server manager and shm tests bring google/cloud/storage/asyncio/_uds and tests/unit/asyncio/uds to 100% line and branch coverage; dead test-helper branches were removed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds OpenTelemetry tracing support for gRPC bidi APIs (Rapid Storage and Standard buckets) across the asynchronous gRPC client and streaming classes (
AsyncGrpcClient,AsyncAppendableObjectWriter, andAsyncMultiRangeDownloader).The implementation aligns with existing regional bucket tracing conventions:
Storage.<Class>.<methodName>.ENABLE_GCS_PYTHON_CLIENT_OTEL_TRACESandDISABLE_GCS_PYTHON_CLIENT_OTEL_BUCKET_METADATA).rpc.system="grpc"and attaches App-Centric Observability (ACO) attributes (gcp.resource.destination.idandgcp.resource.destination.location).traceparentheaders directly into gRPC request metadata tuples.Fixes: b/489239033