8.7 HIGH
- CVSS version (CVSS): 4.0
- Attack Vector (AV): Network (N)
- Attack Complexity (AC): Low (L)
- Attack Requirement (AT): None (N)
- Privileges Required (PR): None (N)
- User Interaction (UI): None (N)
- Vulnerable System Impact Confidentiality (VC): None (N)
- Vulnerable System Impact Integrity (VI): None (N)
- Vulnerable System Impact Availability (VA): High (H)
- Subsequent System Impact Confidentiality (SC): None (N)
- Subsequent System Impact Integrity (SI): None (N)
- Subsequent System Impact Availability (SA): None (N)
- Modified Attack Vector (MAV): Network (N)
- Modified Attack Complexity (MAC): Low (L)
- Modified Attack Requirement (MAT): None (N)
- Modified Privileges Required (MPR): None (N)
- Modified User Interaction (MUI): None (N)
- Modified Vulnerable System Impact Confidentiality (MVC): None (N)
- Modified Vulnerable System Impact Integrity (MVI): None (N)
- Modified Vulnerable System Impact Availability (MVA): High (H)
- Modified Subsequent System Impact Confidentiality (MSC): Negligible (N)
- Modified Subsequent System Impact Integrity (MSI): Negligible (N)
- Modified Subsequent System Impact Availability (MSA): Negligible (N)
- Safety (S): Not Defined (X)
- Automatable (AU): Not Defined (X)
- Recovery (R): Not Defined (X)
- Value Density (V): Not Defined (X)
- Vulnerability Response Effort (RE): Not Defined (X)
- Provider Urgency (U): Not Defined (X)
- Confidentiality Req. (CR): Not Defined (X)
- Integrity Req. (IR): Not Defined (X)
- Availability Req. (AR): Not Defined (X)
- Exploit Maturity (E): Not Defined (X)
by @LeSuisse Activity log
- Created suggestion
- @LeSuisse accepted
- @LeSuisse published on GitHub
NLTK TweetTokenizer URL Pattern Backtracks Catastrophically on Naked-Domain-Like Input
The URLS regular expression in nltk/tokenize/casual.py, compiled into TweetTokenizer.WORD_RE and applied by TweetTokenizer.tokenize, contains a naked-domain branch whose domain-label prefix [a-z0-9]+(?:[.\-][a-z0-9]+)* is unbounded. Input consisting of many alternating label separators can be partitioned in exponentially many ways, and because the branch also requires a trailing top-level domain that such input never supplies, the engine explores those partitions before failing at each offset. A few kilobytes of input therefore consumes seconds to minutes of single-threaded CPU, and the HANG_RE substitution performed before matching does not collapse the pattern. TweetTokenizer is intended for tokenizing untrusted social-media text, so any service that applies it, or the module-level casual_tokenize, to submitted text can be stalled per request without authentication. Version 3.10.1 bounds the label repetition.
References
-
GitHub Issue #3704 issue-tracking
-
Unbounded naked-domain branch in the URLS pattern at 3.9.4 technical-description
-
NLTK v3.10.1 release-notes
-
https://github.com/nltk/nltk product
Affected products
- <3.10.1
- ==3.10.1
Matching in nixpkgs
pkgs.python313Packages.nltk
Natural Language Processing ToolKit
Package maintainers
-
@bengsparks Ben Sparks <benjamin.sparks@protonmail.com>