Skip to content

feat(boto3): Add SQS/SNS trace propagation - #8145

Open
pabloDeputter wants to merge 23 commits into
major/3.0from
pablo/boto3-add-sqs/sns-propagation
Open

pabloDeputter wants to merge 23 commits into
major/3.0from
pablo/boto3-add-sqs/sns-propagation

Conversation

@pabloDeputter

@pabloDeputter pabloDeputter commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Description

Propagates the trace through SQS and SNS messages, so the consumer can be linked back to the request that sent the message.

Send side (SendMessage, SendMessageBatch, Publish, PublishBatch):

  • Adds sentry-trace and baggage as String message attributes, on the message or on each batch entry.
  • Only when trace_propagation_targets match the endpoint.
  • Skips a message if the caller already set these attributes, if it would go over the 10-attribute limit, or if it would go over 256 KiB. AWS would reject the message otherwise.
  • Works on a copy of the params. The caller's dict isn't changed.

Receive side (ReceiveMessage):

  • Adds sentry-trace and baggage to MessageAttributeNames, so SQS returns them. Only when the span is recorded, and not when the caller already asks for All or .*.
  • Adds a span link to each distinct producer context in the received messages, with sentry.link.type: message_origin.
  • Reads the context from SQS message attributes, and from the JSON envelope of an SNS notification without raw message delivery.

Tests: injection and extraction for single and batch calls, the limits above, and a round-trip test: send a message, receive what was injected, and check that the link points at the send span.

Stacked on #8142 (SNS), which is stacked on #8137 (SQS).

Issues

  • resolves: PY-3008

via @dingsdax.

--

View Session [Sentry]

@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Coverage Results 📊

✅ 58328 passed | ❌ 2 failed | ⏭️ 3070 skipped | Total: 61400 | Pass Rate: 95% | Execution Time: 161m 10s

📊 Comparison with Base Branch

Metric Change
Total Tests 📉 -85475
Passed Tests 📉 -80782
Failed Tests 📈 +2
Skipped Tests 📉 -4695

➕ New Tests (2)

View new tests
  • test_execution_outcome[start_sync_execution-TIMED_OUT-stepfunctions_execution_timed_out]
    • File: tests.integrations.boto3.test_stepfunctions
    • Status: ❌ Failing
  • test_db_transaction_spans_disabled_atomic
    • File: tests.integrations.django.test_db_transactions
    • Status: ❌ Failing

❌ Failed Tests

test_execution_outcome[start_sync_execution-TIMED_OUT-stepfunctions_execution_timed_out]

File: tests.integrations.boto3.test_stepfunctions
Suite: py3.10-boto3-v1.24.96
Error: assert '4242' not in "{'trace_id': '32e2a6d4c01f419f9ec9401bd4425381', 'span_id': 'bb458634300c457f', 'name': 'SFN.StartSyncExecution', 'status': 'error', 'is_segment': False, 'start_timestamp': 1794402452.342424, 'end_timestamp': 1794402452.343043, 'parent_span_id': 'b4058434e6f4b28d', 'attributes': {'sentry.origin': 'auto.aws.boto3', 'sentry.trace_lifecycle': 'stream', 'cloud.provider': 'aws', 'rpc.method': 'StartSyncExecution', 'rpc.system.name': 'aws-api', 'sentry.kind': 'client', 'rpc.servi... [+5 more lines; full report in element text]

Stack Trace
tests/integrations/boto3/test_stepfunctions.py:152: in test_execution_outcome
    assert "4242" not in str(span)
E   assert '4242' not in "{'trace_id': '32e2a6d4c01f419f9ec9401bd4425381', 'span_id': 'bb458634300c457f', 'name': 'SFN.StartSyncExecution', 'status': 'error', 'is_segment': False, 'start_timestamp': 1794402452.342424, 'end_timestamp': 1794402452.343043, 'parent_span_id': 'b4058434e6f4b28d', 'attributes': {'sentry.origin': 'auto.aws.boto3', 'sentry.trace_lifecycle': 'stream', 'cloud.provider': 'aws', 'rpc.method': 'StartSyncExecution', 'rpc.system.name': 'aws-api', 'sentry.kind': 'client', 'rpc.service': 'SFN', 'cloud.region': 'eu-north-1', 'server.address': 'states.eu-north-1.amazonaws.com', 'server.port': 443, 'aws.step_functions.state_machine.arn': 'arn:aws:states:eu-north-1:123456789012:stateMachine:orders', 'cloud.resource_id': 'arn:aws:states:eu-north-1:123456789012:stateMachine:orders', 'cloud.account.id': '123456789012', 'aws.step_functions.state_machine.name': 'orders', 'sentry.op': 'rpc', 'thread.id': '140058870849536', 'thread.name': 'MainThread', 'aws.step_functions.execution.arn': 'arn:aws:states:eu-north-1:123456789012:execution:orders:order-1234', 'error.type': 'stepfunctions_execution_timed_out', 'sentry.segment.id': 'b4058434e6f4b28d', 'sentry.segment.name': 'parent', 'sentry.sdk.name': 'sentry.python', 'sentry.sdk.version': '3.0.0b4', 'process.runtime.name': 'CPython', 'process.runtime.version': '3.10.12', 'sentry.environment': 'production', 'sentry.release': 'e2a7700209bf0838524b513e6eb59a99904da088'}}"
E     
E     '4242' is contained here:
E     ?                                                                                                                                                                                           ^^^^^^^^^^^^^^^^^^
E       {'trace_id': '32e2a6d4c01f419f9ec9401bd4425381', 'span_id': 'bb458634300c457f', 'name': 'SFN.StartSyncExecution', 'status': 'error', 'is_segment': False, 'start_timestamp': 1794402452.342424, 'end_timestamp': 1794402452.343043, 'parent_span_id': 'b4058434e6f4b28d', 'attributes': {'sentry.origin': 'auto.aws.boto3', 'sentry.trace_lifecycle': 'stream', 'cloud.provider': 'aws', 'rpc.method': 'StartSyncExecution', 'rpc.system.name': 'aws-api', 'sentry.kind': 'client', 'rpc.service': 'SFN', 'cloud.region': 'eu-north-1', 'server.address': 'states.eu-north-1.amazonaws.com', 'server.port': 443, 'aws.step_functions.state_machine.arn': 'arn:aws:states:eu-north-1:123456789012:stateMachine:orders', 'cloud.resource_id': 'arn:aws:states:eu-north-1:123456789012:stateMachine:orders', 'cloud.account.id': '123456789012', 'aws.step_functions.state_machine.name': 'orders', 'sentry.op': 'rpc', 'thread.id': '140058870849536', 'thread.name': 'MainThread', 'aws.step_functions.execution.arn': 'arn:aws:states:eu-north-1:123456789012:execution:orders:order-1234', 'error.type': 'stepfunctions_execution_timed_out', 'sentry.segment.id': 'b4058434e6f4b28d', 'sentry.segment.name': 'parent', 'sentry.sdk.name': 'sentry.python', 'sentry.sdk.version': '3.0.0b4', 'process.runtime.name': 'CPython', 'process.runtime.version': '3.10.12', 'sentry.environment': 'production', 'sentry.release': 'e2a7700209bf0838524b513e6eb59a99904da088'}}
E     ?                                                                                                                                                                                           ^^^^^^^^^^^^^^^^^^^^^^

test_db_transaction_spans_disabled_atomic

File: tests.integrations.django.test_db_transactions
Suite: py3.14-django-v6.1.3
Error: tests/integrations/django/test_db_transactions.py:263: in test_db_transaction_spans_disabled_atomic assert postgres_commit["is_segment"] is True E assert False is True

Stack Trace
tests/integrations/django/test_db_transactions.py:263: in test_db_transaction_spans_disabled_atomic
    assert postgres_commit["is_segment"] is True
E   assert False is True

✅ Patch coverage is 94.44% (68 of 72 changed executable lines covered; target 80%).
Project statement coverage is 90.41% (up 0.17 percentage points from base (36870b8) to head (0966c93)).

Changed files with executable lines (4)
File Patch coverage Changed executable lines
sentry_sdk/integrations/boto3/_utils.py 92.86% 26/28 covered; missed: 114, 115; partial branches: 111, 113
sentry_sdk/integrations/boto3/_services/sqs.py 94.74% 36/38 covered; missed: 53, 54
sentry_sdk/consts.py 100.00% 2/2 covered
sentry_sdk/integrations/boto3/_services/sns.py 100.00% 4/4 covered
Coverage diff
@@            Coverage Diff             @@
##        master     #8145       +/-##
==========================================
+ Coverage    90.24%    90.41%    +0.17%
==========================================
  Files          203       199        -4
  Tracked lines     27125     21784     -5341
  Branches     10142      7582     -2560
==========================================
- Hits         24478     19695     -4783
+ Misses        2647      2089      -558
+ Partials      1541      1255      -286

Generated by Coverage Action

pabloDeputter and others added 5 commits October 30, 2026 13:55
- stop reading operation type from attributes in `get_span_name()` so that failure in `get_request_attributes()` can't make this fail as well
- wrap URL parsing in `get_request_attributes()` so failure just loses the attributes that come from url
…sqs-extension

Co-Authored-By: Johannes Daxböck <114897+dingsdax@users.noreply.github.com>
…-extension

Co-Authored-By: Johannes Daxböck <114897+dingsdax@users.noreply.github.com>
Co-Authored-By: Johannes Daxböck <114897+dingsdax@users.noreply.github.com>

# Conflicts:
#	sentry_sdk/integrations/boto3/_services/sqs.py
Send a message, receive what was injected, and check that the receive
span links to the send span. Covers SQS and an SNS notification
delivered to SQS without raw message delivery.

Co-Authored-By: johdax <114897+dingsdax@users.noreply.github.com>
@sentry-junior
sentry-junior Bot marked this pull request as ready for review November 9, 2026 12:05
@sentry-junior
sentry-junior Bot requested a review from a team as a code owner November 9, 2026 12:05
@linear-code

linear-code Bot commented Nov 9, 2026

Copy link
Copy Markdown

PY-3008

sentry-junior Bot and others added 4 commits November 9, 2026 12:31
Botocore validates params after our hook, so a bad Entries or ARN value
now only loses that attribute, not every SQS request attribute. Move the
batch error.type code to _utils so SNS can share it, and link the SQS
docs for BatchResultErrorEntry.

Co-Authored-By: Johannes Daxböck <114897+dingsdax@users.noreply.github.com>
…-extension

Co-Authored-By: Johannes Daxböck <114897+dingsdax@users.noreply.github.com>
Co-Authored-By: Johannes Daxböck <114897+dingsdax@users.noreply.github.com>
…/sns-propagation

Co-Authored-By: Johannes Daxböck <114897+dingsdax@users.noreply.github.com>

# Conflicts:
#	sentry_sdk/integrations/boto3/_services/sns.py
#	sentry_sdk/integrations/boto3/_services/sqs.py
#	sentry_sdk/integrations/boto3/_utils.py
Base automatically changed from pablo/boto3-add-sns-extension to major/3.0 November 11, 2026 12:50
@sentrivana sentrivana self-assigned this Nov 11, 2026

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 7d93a71. Configure here.

Co-Authored-By: Ivana Kellyer <131587164+sentrivana@users.noreply.github.com>
@sentry-junior
sentry-junior Bot added this pull request to stack #8245 November 11, 2026 13:30
@sentry-junior
sentry-junior Bot removed this pull request from stack #8245 November 11, 2026 13:32
@sentry-junior
sentry-junior Bot added this pull request to stack #8246 November 11, 2026 13:32
@sentry-junior
sentry-junior Bot removed this pull request from stack #8246 November 11, 2026 13:35
sentrivana added a commit that referenced this pull request Nov 11, 2026
### Description

Adds `Boto3Integration(propagate_traces=True)`. Setting it to `False`
stops the integration from changing the request params of AWS calls:

- Lambda `Invoke` no longer gets trace data in `ClientContext`.
- Once #8145 lands: SQS `SendMessage` / `SendMessageBatch` and SNS
`Publish` / `PublishBatch` no longer get `sentry-trace` / `baggage`
message attributes, and SQS `ReceiveMessage` no longer appends our names
to `MessageAttributeNames`.

The option gates the shared `get_request_params` hook, so it covers
every extension that uses it without per-service changes. Spans are
still created as before.

This is an escape hatch for setups where the added data breaks calls,
e.g. a queue or topic whose `MaximumMessageSize` is set below the 256
KiB the #8145 size check assumes, or consumers that reject unknown
attributes. The name matches `CeleryIntegration(propagate_traces=...)`.

HTTP `sentry-trace` / `baggage` headers on AWS requests are not
affected. They're also added by the stdlib integration and are already
controlled by `trace_propagation_targets`.

Independent of #8145. The SQS/SNS test cases pass trivially until #8145
is merged, and guard the option after that. Whichever PR merges second
needs a small conflict fix at the end of `test_trace_propagation.py`,
since both append tests there.

<!-- junior-request-attribution:start -->
via @sentrivana.
<!-- junior-request-attribution:end -->

<!-- junior-session-footer:start -->
<!-- junior-conversation-id:slack%3AC0BTREZTS2Z%3A1794401901.814229 -->

--

[View
Session](https://junior-prod.sentry.dev/conversations/slack%3AC0BTREZTS2Z%3A1794401901.814229)
[[Sentry]](https://sentry.sentry.io/explore/conversations/slack%3AC0BTREZTS2Z%3A1794401901.814229/?project=4510944073809921)

<!-- junior-session-footer:end -->

---------

Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com>
Co-authored-by: Ivana Kellyer <131587164+sentrivana@users.noreply.github.com>
Co-authored-by: Ivana Kellyer <ivana.kellyer@sentry.io>
…sqs/sns-propagation

Co-Authored-By: Ivana Kellyer <131587164+sentrivana@users.noreply.github.com>

# Conflicts:
#	tests/integrations/boto3/test_trace_propagation.py

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants