Skip to content

Flow.fromPublisher: allow releasing elements which are buffered but not emitted #528

Open

Description

@adamw

Flow.fromPublisher buffers elements in a channel. Some of them never reach the consumer:

  • when the publisher fails, channel.error drops the buffered elements
  • when the consumer stops early (an exception, an interruption), the buffered elements are never received
  • elements can still arrive after the subscription is cancelled

If the elements hold resources, e.g. Netty's reference-counted ByteBufs, nobody releases them, so they leak.

Proposal: an optional callback, e.g. fromPublisher(p, onDiscard: T => Unit), called for each element received from the publisher but not emitted downstream.

Possible implementation: on publisher error, store the error and call channel.done() (so buffered elements are kept). When the flow ends early, cancel the subscription, close the channel, and drain it, calling the callback. In onNext, call the callback if the channel is already closed.

Context: tapir now tracks the unconsumed elements itself, using a lock: softwaremill/tapir#5573

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions