End of thinking response sometimes missed.

[jvantuyl]

Sometimes my session just stops with an unusual message. There doesn't seem to be any rhyme or reason that I can tell about what exactly triggers the response. But the response is pretty consistent.

I'm using GLM-5 with "Show Thinking" enabled. When these stops happen, I always see a Thinking: output followed by what's obviously it's inner monologue. However, eventually there will be a </thinking> which I presume should've been detected and caused it to continue its loop.

Typically, there are tool calls and the like after this. I think this is the action it wants to take next, but it just gets displayed as "thinking" because it never saw the end tag.

Here's one example (lightly redacted):

Thinking:                                                            hide in /settings
  Now let me force push to update the GitHub branch.


   Execute cd /tmp/pdf_oxide_fix
           && git push fork fix/encrypted-pdf-decryption --force
    ↳ To github.com:user/pdf_oxide.git
       + 18c5b6f...a93ec27 fix/encrypted-pdf-decryption -> fix/encrypted-pdf-decrypti...

Thinking:                                                            hide in /settings
  The push succeeded. Now I need to update the Cargo.lock in the main project to fetch
   the new commit.
  </thinking>Now let me update the Cargo.lock in the main
  project:<tool_call>Execute<arg_key>command</arg_key><arg_value>cd
  /home/user/project/app/native/file_tools && cargo
  update -p pdf_oxide 2>&1 | tail -10</arg_value><arg_key>riskLevel</arg_key><arg_valu
  e>medium</arg_value><arg_key>riskLevelReason</arg_key><arg_value>Updating Cargo.lock
  </arg_value><arg_key>timeout</arg_key><arg_value>120</arg_value></tool_call>

I can get it going again by just saying "Continue please." or something like that.

This appears to be sort of the opposite of #885. Where that bug seems to fail to notice the beginning of the thinking block, this one misses the end. I suspect these may be related since it's presumably the same parsing code.

I've seen problems with this where a parser doesn't recognize a token split over packet boundaries. It stops at the end of one read and begins again on the next read. While the beginning of the token was matching when the input stopped, that state isn't preserved until it begins reading data again.

So the token is never fully visible to it so it just passes it through. No idea if that's what's happening here, but it's consistent with that, at least.

I used /bug to upload my logs and such. Bug Report ID is d0fa2108-9dac-4796-8394-357d800ed6a2.

[zwen64657]

5.1 I have encountered two instances where there was no output after the compilation completed, and the output only resumed after I typed "continue". I am unsure whether I should report this issue to the model provider or to the official team of the tool (Droid).

[jvantuyl]

@zwen64657 If you turn on the showThinkingInMainView setting, you can see what it's thinking. If it's anything like mine, you'll see where the thinking block ended but it's still shown as "Thinking" by droid.

To me, that looks like it's a smoking gun indicating a failure to parse the response correctly. That would probably be a Factory problem in that case.

If you can try that out for awhile, it should give you some indication if you're facing this bug or some other one.

[jvantuyl]

Interesting development. I caught one this time that clearly is missing part of the tag. Here's an excerpt:

Thinking:                                                            hide in /settings
  The issue is that the ^new_path pin operator doesn't work in the set: keyword list -
   it's not inside a match context. I need to use a different approach. I can use
  fragment to pass the value, or just directly interpolate it into a fragment.

  Actually, looking at how Ecto update_all works, I should use set:
  [delivered_pathname: fragment("?", new_path)] instead of the pin operator.

  Let me fix this.
  </thinkingThe pin operator doesn't work in set: context. Let me use fragment
  instead:<tool_call>Edit<arg_key>file_path</arg_key><arg_value>redacted_pathname</arg_value><a
  rg_key>new_str</arg_key><arg_value>Enum.each(paths_to_update, fn {id, old_path} ->
          # Remove source dir prefix, add new prefix
          rest = String.replace_prefix(old_path, "#{source_dir}/", "")
          new_path = new_prefix <> rest

          from(dp in DeliveredPath, where: dp.id == ^id)
          |> Repo.update_all(set: [delivered_pathname: fragment("?", ^new_path)])

  end)</arg_value><arg_key>old_str</arg_key><arg_value>Enum.each(paths_to_update, fn
  {id, old_path} ->
          # Remove source dir prefix, add new prefix
          rest = String.replace_prefix(old_path, "#{source_dir}/", "")
          new_path = new_prefix <> rest

          from(dp in DeliveredPath, where: dp.id == ^id)
          |> Repo.update_all(set: [delivered_pathname: ^new_path])
        end)</arg_value></tool_call>

[jvantuyl]

These seem to have stopped happening. Was this fixed?

[jvantuyl]

Was this actually fixed? Or even examined?

Automatic closing of issues with no actual review or response is a red flag regarding product security--especially in this case. As an engineer, I understand the sentiment of "there are too many open tickets". I cannot, however, endorse nor accept an automated policy of ignoring security-critical and stability-critical feedback.

Anybody up to closing this properly?