apache/stormcrawler

#2063

· jnioche opened

on Aug 22, 2026

Credentials are sent over connections the okhttp protocol never authenticated, and the trust-all path uses an "SSL" context with an always-true hostname verifier

#2079

· rzo1 opened

on Aug 29, 2026

Nothing in the shipped configuration keeps the fetcher off private address space

#2080

· rzo1 opened

on Aug 29, 2026

The file scheme is enabled by default and FileProtocol reads any path the worker can open

#2081

· rzo1 opened

on Aug 29, 2026

Politeness queues and the robots.txt cache key on the raw host string, so percent-escaped host aliases get their own queue and their own robots.txt fetch

#2082

· rzo1 opened

on Aug 29, 2026

SiteMapParserBolt sniffs every document, parses sitemaps non-strict, and turns a failed sitemap parse into a terminal ERROR

#2083

· rzo1 opened

on Aug 29, 2026

URLFilters.filter() returns the URL as accepted when a filter in the chain throws

#2084

· rzo1 opened

on Aug 29, 2026

Persistence spouts emit stored URLs without checking the scheme

#2085

· rzo1 opened

on Aug 29, 2026

IPFilterRules has no keyword for link-local addresses and the documented example leaves them allowed

#2086

· rzo1 opened

on Aug 29, 2026

The robots.txt fetch replaces the configured http.content.limit with no limit at all

#2087

· rzo1 opened

on Aug 29, 2026

Redirect hops taken inside okhttp are not run through the URL filters

#2088

· rzo1 opened

on Aug 29, 2026

http.basicauth credentials are attached to every request, whatever the host

#2089

· rzo1 opened

on Aug 29, 2026

A response header named set-header becomes a request header directive

#2090

· rzo1 opened

on Aug 29, 2026

Response headers can populate the reserved verbatim request and response metadata keys

#2091

· rzo1 opened

on Aug 29, 2026

CharsetIdentification retries the meta charset lookup ten bytes at a time by recursing, and the call sits outside JSoupParserBolt's catch block

#2092

· rzo1 opened

on Aug 29, 2026

AbstractLLMTextExtractor pastes page HTML into the prompt unescaped and returns the reply unchecked

#2093

· rzo1 opened

on Aug 29, 2026

Playwright browser context stops validating certificates as soon as a proxy is configured

#2094

· rzo1 opened

on Aug 29, 2026

Playwright protocol applies neither the IP filter nor the URL filters to the requests it makes

#2095

· rzo1 opened

on Aug 29, 2026

SQL IndexerBolt interpolates metadata labels into the statement as unquoted identifiers

#2096

· rzo1 opened

on Aug 29, 2026

Tika ParserBolt puts no limit on the extracted text and no time limit on the parse

#2097

· rzo1 opened

on Aug 29, 2026

URLFrontier channels are always plaintext and the module has no TLS option

#2098

· rzo1 opened

on Aug 29, 2026

Response headers named metrics.* are fed straight into the fetcher metric registry, and a non-numeric one fails the fetch

#2099

· rzo1 opened

on Aug 29, 2026

Gzipped sitemap files are expanded with no size bound, and SiteMapParserBolt reads Content-Type under a key that is never present

#2100

· rzo1 opened

on Aug 29, 2026

HostURLFilter memoizes the source host in plain fields shared by all fetcher threads

#2101

· rzo1 opened

on Aug 29, 2026

SCProxy.toString() rebuilds the proxy URL with user and password, and the protocol logs it

#2102

· rzo1 opened

on Aug 29, 2026

Navigation Menu

Uh oh!

4.0.0

Upgrade to Tika 4.0

Credentials are sent over connections the okhttp protocol never authenticated, and the trust-all path uses an "SSL" context with an always-true hostname verifier

Nothing in the shipped configuration keeps the fetcher off private address space

The file scheme is enabled by default and FileProtocol reads any path the worker can open

Politeness queues and the robots.txt cache key on the raw host string, so percent-escaped host aliases get their own queue and their own robots.txt fetch

SiteMapParserBolt sniffs every document, parses sitemaps non-strict, and turns a failed sitemap parse into a terminal ERROR

URLFilters.filter() returns the URL as accepted when a filter in the chain throws

Persistence spouts emit stored URLs without checking the scheme

IPFilterRules has no keyword for link-local addresses and the documented example leaves them allowed

The robots.txt fetch replaces the configured http.content.limit with no limit at all

Redirect hops taken inside okhttp are not run through the URL filters

http.basicauth credentials are attached to every request, whatever the host

A response header named set-header becomes a request header directive

Response headers can populate the reserved verbatim request and response metadata keys

CharsetIdentification retries the meta charset lookup ten bytes at a time by recursing, and the call sits outside JSoupParserBolt's catch block

AbstractLLMTextExtractor pastes page HTML into the prompt unescaped and returns the reply unchecked

Playwright browser context stops validating certificates as soon as a proxy is configured

Playwright protocol applies neither the IP filter nor the URL filters to the requests it makes

SQL IndexerBolt interpolates metadata labels into the statement as unquoted identifiers

Tika ParserBolt puts no limit on the extracted text and no time limit on the parse

URLFrontier channels are always plaintext and the module has no TLS option

Response headers named metrics.* are fed straight into the fetcher metric registry, and a non-numeric one fails the fetch

Gzipped sitemap files are expanded with no size bound, and SiteMapParserBolt reads Content-Type under a key that is never present

HostURLFilter memoizes the source host in plain fields shared by all fetcher threads

SCProxy.toString() rebuilds the proxy URL with user and password, and the protocol logs it

Uh oh!

List view