gh-69919: Raise SyntaxError for all invalid sources in compile() - #157586
serhiy-storchaka wants to merge 5 commits into
Conversation
compile(), exec(), eval() and ast.parse() now raise SyntaxError instead of ValueError if the source string contains surrogate characters, and instead of MemoryError or RecursionError if the source is too complex to parse or compile. Consumers that handle SyntaxError, like the code module and IDLE, no longer need to handle those exceptions. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
@pablogsal @lysnikolaou This and #157585 are alternatives for the same problem; I would like your opinion on which to take. This one changes the exceptions raised by |
Documentation build overview
27 files changed ·
|
A line longer than 2**31 bytes, more than 2**31 lines, or a string literal longer than 2**31 bytes are syntax errors too. Drop the stale mentions of OverflowError and ValueError for invalid literals in the code and codeop documentation.
|
The other alternative has been merged (hence merge conflicts) and backported. Close this one? |
# Conflicts: # Doc/builtins/functions.rst # Doc/library/code.rst # Lib/code.py
Too complex source now raises SyntaxError, so the tests inject MemoryError to test that any compile() error is reported. Compiling a too deeply nested AST object still raises RecursionError. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
Not necessarily, they are not exclusive. #157585 makes |
2 participants
compile(),exec(),eval()andast.parse()now raiseSyntaxErrorinstead ofValueError(UnicodeEncodeError) if the source string contains surrogate characters — with the line, column and text of the offending character, like other syntax errors — instead ofMemoryError("Parser stack overflowed") orRecursionError("Stack overflow during compilation") if the source is too complex to parse or compile, and instead ofOverflowErrorif the source is too large (a line or string literal longer than 231 bytes, or more than 231 lines; verified manually, no tests at that size). This matches how null bytes and undecodable sources are already reported, and means consumers that handleSyntaxError—code.InteractiveInterpreter, the REPL, IDLE — need nothing else: IDLE's Shell no longer gets stuck after such input.Compiling an AST object is unchanged: a cyclic or too deep tree still raises
RecursionError, and an out-of-rangelinenostill raisesOverflowError.The
codeandcodeopdocumentation no longer mentionOverflowErrorandValueError"for an invalid literal": that described Python 1.5/2.0 behavior (integer literals too large for a C long, bad escapes in string literals), which has been aSyntaxErrorsince Python 2.4/3.0.This is an alternative to #157585, which instead makes the consumers catch all exceptions and can be backported.
🤖 Generated with Claude Code