gh-146385: Switch back to re to detect shlex.quote slow path

[bonzini]

Commit 06a26fd ("gh-118761: Optimise import time for shlex (#132036)") when the input has to be quoted. This is because the regular expression search was able to short-circuit at the first unsafe character.

Go back to the same algorithm as 3.13, but make the "import re" and compilation of the regular expression lazy.

Testing s.isascii() makes shlex.quote() twice as fast in the non-ASCII case, but costs up to 25% of the full run time (because it necessitates an earlier isinstance check) if the string is ASCII. The latter is probably the common case, so drop the check.

• Issue: Performance regression in shlex.quote from 3.13 to 3.14 #146385

[bedevere-app]

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

[python-cla-bot]

All commit authors signed the Contributor License Agreement.

[picnixz]

@AA-Turner Can you check if the results are consistent on Windows as well please?

[picnixz]

Here is what I suggest:

• For 3.14, we can fix the regression (I will consider it a bug) if this reproduces on multiple systems. We might have other things at play here depending on where Python is built.
• For 3.15, we can use lazy imports to make it cleaner if necessary.

I will also try to check what happens on older systems (openSUSE + older GCC/clang).

[bedevere-app]

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

[bonzini]

Yeah I didn't use lazy import to cater for backports. Would you prefer the lazy import version to be in this fix PR, or in a follow-up?

[picnixz]

For now, let's check whether the regression is consistent across platforms.

[bonzini]

For now, let's check whether the regression is consistent across platforms.

It is, and also it's really a complexity change. shlex.quote('a ' + 'b' * N) will always be slower in 3.14+ if you make N high enough. (And in practice, 10 is enough to have a substantial effect).

[picnixz]

What I want to know is why Adam reported numbers that went in the other direction. Ideally I also want to know the stdev of the numbers using pyperf (which I will do this w-e)

[picnixz]

I suggested in #132036 (comment) to replace the re call if we had something more efficient, and Adam found out that it was faster using bytes.translate so I'd like to know which data it was tested on:

With this, we go from ~26 microseconds for quote to ~2 (with only safe characters) and ~6 (with characters needing quotation).