Skip to content

parser provides access to chunked trailer fields #42

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ashtum merged 2 commits into from
Aug 10, 2026
Merged

parser provides access to chunked trailer fields #42

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c79b553
parser provides access to chunked trailer fields
ashtum Aug 8, 2026
3310219
parser body limit management is simplified
ashtum Aug 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 5 additions & 17 deletions include/boost/burl/message_reader.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,10 +70,10 @@ namespace burl
buffer. Use @ref read_some instead when the
octets have to land in memory of your choosing.

Every operation drives @ref parser::parse_header
first, so reading a body without having read the
header explicitly works as expected. This serves
the caller who has no interest in the header:
Every operation parses the header first, so
reading a body without having read the header
explicitly works as expected. This serves the
caller who has no interest in the header:
anything decided from it, installing a decoder
above all, needs @ref read_header called
explicitly, because @ref parser::set_decoder
Expand Down Expand Up @@ -290,14 +290,10 @@ capy::io_task<std::string_view>
message_reader<S>::
read_body_(S& stream, parser& pr)
{
if(!pr.got_header())
if(auto [ec] = co_await read_header_(stream, pr); ec)
co_return { ec, {} };

for(;;)
{
system::error_code ec;
auto const sv = pr.body(ec);
auto const sv = pr.flatten_body(ec);
if(ec != http::error::need_data)
co_return { std::error_code(ec), sv };
if(auto [rec] = co_await refill_(stream, pr); rec)
Expand All @@ -314,10 +310,6 @@ read_some_(
parser& pr,
MB buffers)
{
if(!pr.got_header())
if(auto [ec] = co_await read_header_(stream, pr); ec)
co_return { ec, 0 };

capy::buffer_param bp(buffers);

for(;;)
Expand Down Expand Up @@ -380,10 +372,6 @@ pull_(
parser& pr,
std::span<capy::const_buffer> dest)
{
if(!pr.got_header())
if(auto [ec] = co_await read_header_(stream, pr); ec)
co_return { ec, {} };

for(;;)
{
system::error_code ec;
Expand Down
141 changes: 93 additions & 48 deletions include/boost/burl/parser.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,14 +59,21 @@ namespace burl
The body can be retrieved three ways, which
differ in where the octets end up:

@li @ref body returns the whole body in place,
without copying,
@li @ref flatten_body returns the whole body in
place, without copying,
@li @ref read_some copies into caller-supplied
memory, or lets an installed decoder write
into it directly, and
@li @ref pull borrows the parser's own buffers,
which @ref consume then releases.

Each parses the header first when it has not
been parsed already, so a caller with no
interest in the header never has to call @ref
parse_header. Installing a decoder does require
it, because @ref set_decoder must run after the
header and before any body octet.

@par Errors

Every parsing operation reports through an
Expand Down Expand Up @@ -317,19 +324,36 @@ class parser

/** Return the octets which may be received directly.

Returns the number of octets which may be
read from the stream straight into
caller-supplied memory, bypassing the
parser's buffer entirely, or zero when that
is not permitted. Report octets received
this way with @ref commit_direct.
Body octets may be read from the stream
straight into caller-supplied memory,
bypassing the parser's buffer, when every
one of these holds:

@li the header has been parsed,

@li the payload has a known size or is
delimited by the end of the stream,

@li no @ref decoder is installed,

@li the buffer holds no payload octets,

@li @ref commit_eof has not been called, and

@li the body limit permits more octets: a
known size must fit within what remains of
the limit, and a payload delimited by the
end of the stream must not have reached it.

Otherwise body octets must be received
through @ref prepare and @ref commit.

This is zero unless the header has been
parsed, no decoder is installed, the payload
has a known size or is delimited by the end
of the stream, the buffer holds no payload
octets, the stream has not ended, and the
body limit has not been reached.
@return The number of octets which may be
received directly, or zero when that is not
permitted. For a payload of known size this
is what remains of the payload; for one
delimited by the end of the stream it is
what remains of the body limit.

@see @ref commit_direct.
*/
Expand Down Expand Up @@ -373,36 +397,42 @@ class parser
void
parse_header(system::error_code& ec);

/** Return the complete body in place.
/** Flatten the body in place and return it.

Reads the remainder of the body into the
parser's own buffer and returns a view of
the whole body, without copying. A chunked
payload is coalesced in place.
Coalesces the buffered body octets into a
contiguous range in the parser's own buffer
and returns a view of them, without copying.
A chunked payload is de-chunked in place.
Parses the header first if @ref got_header
returns false.

@par Preconditions
@li `this->got_header() == true`
@li No octet of the body has been retrieved
by @ref read_some or @ref pull.
@ref start has been called.

@param ec Set to the error, if any occurred.
Set to @ref http::error::in_place_overflow if
the body does not fit in the buffer.

@return A view of the body, valid until the
parser is modified.
Set to @ref http::error::need_data until the
complete body is buffered, or to
@ref http::error::in_place_overflow if the
body does not fit in the buffer.

@return A view of the body octets flattened
so far, valid until the parser is modified.
The body is complete when no error is
reported.
*/
BOOST_BURL_DECL
std::string_view
body(system::error_code& ec);
flatten_body(system::error_code& ec);

/** Copy body octets into caller-supplied memory.

When a decoder is installed, it writes its
output into `buffers` directly.
output into `buffers` directly. Parses the
header first if @ref got_header returns
false.

@par Preconditions
`this->got_header() == true`
@ref start has been called.

@param buffers The destination.

Expand All @@ -422,10 +452,11 @@ class parser

Fills `dest` with descriptors referring to
the parser's own buffers. Release them with
@ref consume.
@ref consume. Parses the header first if
@ref got_header returns false.

@par Preconditions
`this->got_header() == true`
@ref start has been called.

@param dest The descriptors to fill.

Expand Down Expand Up @@ -458,13 +489,39 @@ class parser
void
consume(std::size_t n) noexcept;

/** Copy the trailer fields into a container.

Appends each field in the trailer section
of a chunked payload to `f`, in the order
received.

@par Preconditions
`this->got_header() == true`

@par Exception Safety
Basic guarantee. An exception from the
container leaves the parser unchanged;
fields already appended remain, and the
call may be retried.

@param f The container to append to.

@param ec Set to the error, if any
occurred.
*/
BOOST_BURL_DECL
void
parse_trailer(
fields_base& f,
system::error_code& ec);

protected:
parser() = default;

BOOST_BURL_DECL
parser(
config const& cfg,
bool is_request);
bool is_req);

parser(parser&& other) noexcept = default;

Expand Down Expand Up @@ -497,16 +554,7 @@ class parser
need_more() const noexcept;

std::size_t
raw_limit_rem() const noexcept;

std::size_t
dec_limit_rem() const noexcept;

bool
payload_sized() const noexcept;

std::size_t
payload_rem() const noexcept;
trailer_extent() const noexcept;

std::error_code
walk_chunks(chunk_fn f, bool dry = false);
Expand All @@ -524,14 +572,11 @@ class parser
decoder * dec_ = nullptr;
detail::circular_buffer in_;
detail::circular_buffer out_;
std::uint64_t chunk_rem_ = 0;
std::uint64_t transferred_ = 0;
std::uint64_t decoded_ = 0;
std::uint64_t rem_ = 0;
std::uint64_t body_limit_ = 0;
std::uint64_t payload_size_ = 0;
std::uint64_t limit_rem_ = 0;
std::error_code dec_err_;
http::payload payload_ = http::payload::none;
bool is_req_ : 1 = true;
bool head_ : 1 = false;
bool started_ : 1 = false;
bool got_header_ : 1 = false;
Expand Down
41 changes: 27 additions & 14 deletions src/detail/circular_buffer.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -150,21 +150,34 @@ linearize(char* floor) noexcept
else if(len != 0)
{
auto const bufs = data();
auto const* a = static_cast<char const*>(bufs[0].data());
auto an = bufs[0].size();
auto const* b = static_cast<char const*>(bufs[1].data());
auto const bn = bufs[1].size();
char* base = floor;
do
auto const* a = static_cast<char const*>(bufs[0].data());
auto an = bufs[0].size();
auto const* b = static_cast<char const*>(bufs[1].data());
auto const bn = bufs[1].size();
auto const gap = static_cast<std::size_t>(ptr - floor) + (cap - len);

// leapfrogging to the floor runs ceil(an / gap) rounds
// and re-moves the wrapped range each round; past a
// bound the in-place rotate is cheaper, and it is the
// only option when gap == 0
if(an <= gap * 16)
{
auto* bp = (std::min)(base + an, const_cast<char*>(a) - bn);
b = static_cast<char const*>(std::memmove(bp, b, bn));
auto chunk_a = static_cast<std::size_t>(b - base);
std::memcpy(base, a, chunk_a);
an -= chunk_a;
base += chunk_a;
a += chunk_a;
} while(an);
char* base = floor;
do
{
auto const k = (std::min)(an, gap);
b = static_cast<char const*>(std::memmove(base + k, b, bn));
std::memcpy(base, a, k);
an -= k;
base += k;
a += k;
} while(an);
}
else
{
std::rotate(ptr, ptr + pos, ptr + cap);
p = ptr;
}
}
cap = static_cast<std::size_t>((ptr + cap) - p);
ptr = p;
Expand Down
Loading
Loading