Engine: Strip whitespace between consecutive end tags after expression blocks

[joelhawksley]

Fix: Strip whitespace between consecutive <% end %> tags after expression blocks

Co-written with Claude Opus 4.6 - I've reviewed the fix and generally think it's right, but there might be a simpler solution?

Problem

Herb emits @output_buffer.safe_append= for whitespace between consecutive <% end %> tags that Erubi (with trim: true) strips. This breaks CaptureHelper#capture in Rails 8.2 for helpers like form_tag that return HTML via the block's return value rather than calling concat.

When Herb adds @output_buffer.safe_append=' ' between end statements:

1. The block's Ruby return value changes from the form HTML to @raw_buffer (the return of safe_append=)
2. The capture buffer gets whitespace (e.g. " ")
3. buffer.presence returns nil (whitespace is .blank?)
4. The value fallback is just @raw_buffer (which only contains whitespace)
5. The form HTML is completely lost

Triggering template:

<% outer do %>
  <% form_tag("/t") do %>
    <%= tag.p do %>
      text
    <% end %>
  <% end %>
<% end %>

Herb compiled (before fix):

    end )
 _buf << '  '.freeze; end; _buf << '\n'.freeze; end

Erubi compiled (and Herb after fix):

    end )
   end 
 end

Root Cause

In preceding_token_ends_with_newline?, :expr_block_end was grouped with expression tokens in a blanket return false list. This prevented at_line_start? from recognizing that a <% end %> tag following an expression block's closing end is at line start — so the subsequent code tag was treated as inline, and the whitespace between them was emitted as buffer appends instead of being trimmed.

Fix

Remove :expr_block_end from the blanket return false list and instead check whether the expr_block_end token's value actually ends with "\n" (which it does when the end tag was on its own line). This lets at_line_start? correctly trigger trim behavior for subsequent code-only tags.

[joelhawksley]

@marcoroth green!

[marcoroth]

Thank you @joelhawksley! 🙏🏼