fix: fall back to plain text when RSS item content triggers RecursionError - #2333
Open
Jeremy Schoemaker (shoemoney) wants to merge 1 commit into
Open
fix: fall back to plain text when RSS item content triggers RecursionError#2333Jeremy Schoemaker (shoemoney) wants to merge 1 commit into
Jeremy Schoemaker (shoemoney) wants to merge 1 commit into
Conversation
…Error Deeply nested HTML inside an RSS item's description or content:encoded previously hit the broad except in _parse_content and returned the raw unconverted HTML, silently embedding HTML tags in the markdown output. Catch RecursionError specifically and fall back to BeautifulSoup's iterative get_text(), mirroring the HTML converter fix from microsoft#1644. Includes a deterministic regression test.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
RssConverter._parse_contentwraps markdownify conversion in a broadexcept BaseExceptionthat returns the original content. When an RSS item'sdescriptionorcontent:encodedcontains deeply nested HTML (roughly 400+ levels), markdownify's recursive DOM traversal raisesRecursionError, and the converter silently embeds the raw unconverted HTML in the markdown output.This PR mirrors the fix merged in #1644 for the HTML converter: catch
RecursionErrorspecifically, emit a warning, and fall back to BeautifulSoup's iterativeget_text()so callers still get usable plain-text content. Other exceptions keep the existing behavior.Includes a deterministic regression test using the same
sys.setrecursionlimitpattern as the #1644 test: a feed item with 500 nested divs must produce text output with no raw HTML tags. Before the fix the output contained 500 raw<divtags; after, none.black --checkclean.