Repository navigation
docs: DeferMessageAction on SQS FIFO preserves group order - #209
Conversation
|
|
||
| `DeferMessageAction` will de-order the message. This means you cannot rely on the order of messages in the channel. Either, your message must be able to processed independently of the others, or you must use a strategy to process out-of-order. For example you may add a version stamp to the message and only applying it if you have not already processed a later message. | ||
|
|
||
| **Exception: AWS SQS.** On SQS, a deferred message is not re-published. Brighter calls `ChangeMessageVisibility` on the same in-flight message, so the message ID, message group ID and deduplication ID do not change. On an SQS FIFO queue, SQS does not deliver later messages from the same message group while the deferred message is invisible, so order within the group is preserved. The group is blocked until the delay ends; other message groups keep flowing. On a standard SQS queue there is no ordering to preserve. See [Ack and Nack](/contents/AWSSQSConfiguration.md#ack-and-nack) for how requeue is implemented. |
There was a problem hiding this comment.
So, if you look further up, it indicates that DeferMessageAction will use the scheduler. This is not true for SQS, which uses visibility. It's also not entirely true for RMQ and AWS, which use non-scheduler-based retry mechanisms. Those will (I think), however, de-order
What we may need is a table that indicates:
Broker
Defer Message Uses
Ordering
You can have an entry for "All Others" that just says "Scheduler" and "Deorders" and then add the entries for SQS, RMQ and ADB
There was a problem hiding this comment.
Thanks, @iancooper - agreed.
@guilhermeappel got there before me and I see the table is coming in #217
I've included the BufferSize limit note from your testing. Are we happy to merge this PR in and then the end state with the table follows in Guilherme's PR?
|
@calebkoy I think this works better as part of the table. Now I have merged that, could you take a look at that approach |
@iancooper done in 8b6d00d |
|
Thanks @calebkoy |
HandlerFailure.mdsaysDeferMessageActionwill de-order the message. On SQS it does not. Defer calls ChangeMessageVisibility on the same in-flight message (Proactor.RequeueMessage -> SqsMessageConsumer.RequeueAsync), so message ID, group ID and dedup ID are unchanged.On a FIFO queue, later messages in the same group stay unavailable until the delay ends, so order within the group is preserved; other groups keep flowing.
Adds a short 'Exception: AWS SQS' paragraph and links to Ack and Nack in AWSSQSConfiguration.md. Existing sentence left unchanged (other transports not checked).