Skip to content

Handle Zlib inputs left hanging at the middle of a block#2514

Open
etcimon wants to merge 1 commit intovibe-d:masterfrom
etcimon:patch-1
Open

Handle Zlib inputs left hanging at the middle of a block#2514
etcimon wants to merge 1 commit intovibe-d:masterfrom
etcimon:patch-1

Conversation

@etcimon
Copy link
Copy Markdown
Contributor

@etcimon etcimon commented Jan 5, 2021

Handle inputs left hanging at the middle of a block

Handle inputs left hanging at the middle of a block
@s-ludwig
Copy link
Copy Markdown
Member

Wow, so this is really a case that needs to be handled? So many small fixes were necessary already here, because the zlib API is so brittle.

Based on the documentation, it seems like the input should always be fully drained, as long as enough output space is provided:

The detailed semantics are as follows. inflate performs one or both of the following actions:

  • Decompress more input starting at next_in and update next_in and avail_in accordingly. If not all input can be processed (because there is not enough room in the output buffer), then next_in and avail_in are updated accordingly, and processing will resume at this point for the next call of inflate().
  • Generate more output starting at next_out and update next_out and avail_out accordingly. inflate() provides as much output as possible, until there is no more input data or no more space in the output buffer (see below about the flush parameter).

...or at least not having enough output space is the only reason given. So if this indeed needs to be done anyway, I would suggest to only do it if there is still something in the input buffer and inflate did not write anything to the output buffer - just to avoid adding unnecessary memmove overhead in the regular cases.

@s-ludwig
Copy link
Copy Markdown
Member

So I looked at this some more and tested with different kinds of inputs (different levels of compressibility), as well as input streams that only yielded one byte at a time. I couldn't make the existing code fail that way, unfortunately.

Do you remember where it failed for you? Did it fail an assertion (e.g. assert(m_zstream.avail_out != m_outbuffer.peekDst.length || m_zstream.avail_in != avins)), or did it hang in an infinite loop?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants