Update of docus

Author: stamparm

doc/ARCHITECTURE.md

@@ -19,6 +19,7 @@ non-ASCII byte is unavoidable. Compatibility shims live in `lib/core/compat.py`
 |-------|------|---------|
 | CLI | `sqlmap.py` -> `main()` | the scanner. Applies runtime patches, parses options, runs a scan. |
 | REST API | `sqlmapapi.py` | `-s` server / `-c` client wrappers around `lib/utils/api.py`. |
+| Library | `import sqlmap` -> `lib/utils/library.py` | `scan()` / `scanFromRequest()` for programmatic callers; like the API, it drives the engine as a subprocess. |
 
 `main()` (sqlmap.py) does, in order: `dirtyPatches()` (monkey-patches stdlib for
 quirks/security - see below), `setPaths()`, `init()` (option parsing + environment
@@ -58,7 +59,7 @@ Identifiers in the codebase are camelCase.
 | `lib/core/` | conf/kb model, common helpers, settings, enums, dump, session, agent, option parsing |
 | `lib/controller/` | the scan orchestrator (`controller.py`), detection checks (`checks.py`), enumeration dispatch (`action.py`), DBMS handler selection (`handler.py`) |
 | `lib/request/` | HTTP layer: `connect.py` (sending), `comparison.py` (the true/false oracle), `inject.py` (value extraction), protocol handlers, response processing |
-| `lib/techniques/` | the exploitation engines: `blind/inference.py`, `error/use.py`, `union/{test,use}.py`, `dns/` |
+| `lib/techniques/` | the exploitation engines: `blind/{inference,multibit}.py`, `error/use.py`, `union/{test,use}.py`, `dns/`, plus one directory per non-SQL family (`ssti/`, `nosql/`, `xpath/`, ...) |
 | `lib/parse/` | parsing of inputs: CLI, config, HTTP request/log files, HTML, sitemap, and the XML payload/boundary loader (`payloads.py`) |
 | `lib/utils/` | feature modules: `api.py` (REST), `hashdb.py` (session), `crawler.py`, `hash.py` (cracking), `har.py`, `brute.py`, `search.py`, ... |
 | `lib/takeover/` | OS-level takeover: shells, file access, UDF, registry, Metasploit, `xp_cmdshell` |
@@ -130,6 +131,7 @@ Once a parameter is injectable, value extraction is dispatched by
 | Technique | Engine | Mechanism |
 |-----------|--------|-----------|
 | boolean-based blind | `blind/inference.py: bisection()` | binary-search each character via true/false oracle |
+| multi-bit blind (`--multi-bit`) | `blind/multibit.py` | opt-in; reads several bits per request out of the rows a listing page renders |
 | time-based blind / stacked | `blind/inference.py` (time compare) | same bisection, oracle is a measured delay |
 | error-based | `error/use.py: errorUse()` | parse the value straight out of a provoked DB error |
 | UNION query | `union/{test,use}.py` | column-count detection then `UNION SELECT` extraction |
@@ -141,6 +143,14 @@ Once a parameter is injectable, value extraction is dispatched by
 (intentional). Multi-threaded extraction is coordinated via `kb.locks` and
 `getCurrentThreadData()` (`lib/core/threads.py`).
 
+**Non-SQL engines.** `--nosql`, `--xpath`, `--ldap`, `--ssti`, `--graphql`, `--hql`, `--sparql`,
+`--odata`, `--xslt`, `--xxe` and `--jwt` do not go through the pipeline above at all. Each one is a
+self-contained scanner in `lib/techniques/<family>/inject.py`, dispatched from `controller.py` and
+registered once in `NONSQL_TECHNIQUES` (`lib/core/settings.py`) - that tuple is what the rest of the
+code tests against, so a new family is declared there and nowhere else. The response comparison,
+reflection stripping and blind-bit classification they all need live in `lib/utils/nonsql.py`
+instead of being copied per family.
+
 ---
 
 ## 7. DBMS abstraction
@@ -155,7 +165,10 @@ Enumeration is DBMS-agnostic at the top and specialized underneath:
   pieces and supplying dialect specifics.
 - **`data/xml/queries.xml`** - per-DBMS SQL query templates (banner, current user, table
   enumeration, casting, etc.) keyed by DBMS. The generic code asks for a query by name;
-  the dialect comes from XML.
+  the dialect comes from XML. It also carries the per-DBMS `<gadgets>` - side-effecting scalar
+  expressions (e.g. PostgreSQL `dblink_exec`) that `getGadget()` (`lib/request/inject.py`) probes
+  once and caches in `kb.gadget`, so `--sql-query` / `--file-write` / `--os-cmd` still work from an
+  injection point that has no stacked queries.
 
 `conf.dbmsHandler` (set in `handler.py`) is the live object that `action()` calls into.
 
@@ -180,12 +193,16 @@ Enumeration is DBMS-agnostic at the top and specialized underneath:
 `lib/request/connect.py` (`Connect.getPage`) is the single HTTP chokepoint. Around it:
 protocol handlers (`httpshandler`, `redirecthandler`, `chunkedhandler`, `rangehandler`,
 persistent connections via `lib/request/keepalive.py`), response processing (`basic.py`), and the
-comparison oracle (`comparison.py`).
+comparison oracle (`comparison.py`). Alternative transports hang off the same chokepoint: the
+standard-library HTTP/2 client (`http2.py`, `--http2`, which also carries the `--timeless` oracle in
+`timeless.py`) and WebSocket targets (`websocket.py`).
 
 **Tamper scripts** (`tamper/`) mutate the payload just before sending to evade WAF/IPS.
 Each file exposes a `tamper(payload, **kwargs)` and a `__priority__`; `--tamper=a,b,c`
 chains them in priority order. They are payload-string transforms only (no engine
-coupling), which is why they compose freely.
+coupling), which is why they compose freely. When a WAF/IPS is identified, `lib/utils/wafbypass.py`
+ranks the plausible candidates and `_autoWafBypass()` (`controller.py`) trials them one at a time -
+a candidate is adopted only if re-running the detection through it brings the injection back.
 
 ---
 
@@ -227,6 +244,7 @@ Two complementary layers:
 | a constant/threshold | `lib/core/settings.py` |
 | how injection is *detected* | `data/xml/boundaries.xml` + `data/xml/payloads/*.xml`, then `lib/controller/checks.py` |
 | how a value is *extracted* | `lib/request/inject.py` + the relevant `lib/techniques/` engine |
+| a non-SQL technique (SSTI, NoSQL, XPath, ...) | `lib/techniques/<family>/inject.py` (+ `NONSQL_TECHNIQUES` in `settings.py`) |
 | the true/false decision | `lib/request/comparison.py` |
 | a per-DBMS query/dialect | `data/xml/queries.xml` + `plugins/dbms/<dbms>/` |
 | enumeration behavior | `plugins/generic/*.py` |

doc/CHANGELOG.md

@@ -15,10 +15,12 @@
 * Added the switch `--xslt`. It tests for XSLT injection. The engine names itself in the response. sqlmap then dumps the XML document that the stylesheet transforms. It also reads the files that the engine can reach. When the engine exposes an extension bridge (PHP `php:function` or the Xalan `java:` namespace), sqlmap reads any file through it, and with `--os-cmd` or `--os-shell` it runs operating system commands.
 * Added the switch `--xxe`. It tests for XML External Entity injection. It uses in-band, error-based, and out-of-band channels.
 * Added the switch `--jwt`. It examines JSON Web Tokens for weak keys and for injection in the claims.
+* sqlmap now offers to get around a WAF/IPS on its own. Once it identifies one and the ordinary payloads come back blocked, it drops the scanner fingerprint and tries the tamper scripts that suit that WAF, and it keeps the first one that brings the injection back.
 
 ## Speed
 
 * Added the switch `--timeless`. It reads each blind bit from the HTTP/2 response order. It does not use a delay. sqlmap calibrates the target first, and it uses the usual time-based technique if the target is not applicable.
+* Added the switch `--multi-bit`. On a page that renders rows, one request reads a bit from each row that comes back instead of a single bit altogether.
 * Added set-membership (Huffman) retrieval for blind dumps. It needs fewer requests for each character. Use `--no-huffman` to stop it.
 * Added keyset (seek) pagination for blind table dumps. Use `--no-keyset` to stop it.
 * Added parallel retrieval of values in blind mode. Each thread retrieves a different value.
@@ -53,10 +55,13 @@
 * Added time-based payloads for CUBRID.
 * Added out-of-band DNS channels for H2 and ClickHouse.
 * Added PostgreSQL command execution through a PL extension.
-* Added the running of non-query statements without stacked queries through a gadget. On PostgreSQL, when the `dblink` extension is present, `--sql-query`, `--file-write`, `--os-cmd`, and `--os-shell` now work from a plain (e.g. boolean-based) injection point.
+* Added the running of non-query statements without stacked queries through a gadget. On PostgreSQL, when the `dblink` extension is there and the current user can run it, `--sql-query`, `--file-write`, `--os-cmd`, and `--os-shell` now work from a plain (e.g. boolean-based) injection point.
 * Added file read and file write support for SQLite through the `fileio` extension functions `readfile` and `writefile`.
 * Added the data access and the security type of a routine to the output of `--procs` on MySQL and PostgreSQL, so a routine that runs as its definer or that modifies data stands out.
-* Added the tamper scripts `blindbinary`, `dollarquote`, `infoschema2innodb`, `oraclequote`, and `sign`.
+* Added stacked query payloads for Microsoft SQL Server and Sybase that carry no semicolon, so a filtered semicolon alone no longer hides the technique.
+* Added the error signatures that tell the MySQL and the PostgreSQL forks apart: Doris, StarRocks, CockroachDB, YugabyteDB, OpenGauss, DuckDB, and Trino. These forks keep the wording of the engine that they come from, so only a leaked driver package or an engine-internal source reference gives them away.
+* Made the fingerprinting payloads friendlier to a WAF, so the version detection survives where it used to get blocked.
+* Added the tamper scripts `blindbinary`, `castprefix`, `dollarquote`, `infoschema2innodb`, `mid2leftright`, `mssqlnosemicolon`, `odbcbrace`, `oraclequote`, `quote2ltat`, `sign`, `sleep2hex`, and `uniontable`.
 
 ## Fewer dependencies
 
@@ -93,6 +98,11 @@
 * Made the heuristic hints of the non-SQL switches exclusive. A signature no longer matches the errors of a different engine, an ordinary SQL error, or a page that only contains the name of a template engine.
 * Corrected the GraphQL validation signatures. They now match the quotes in the way that the JSON body escapes them.
 * Added the error signatures of Mako and of DynamoDB. sqlmap did not recognise the errors of these two back-ends.
+* sqlmap no longer stops with an exception on a JWT whose signature is not valid base64url. Such a signature can not be an HMAC that sqlmap could verify, so it is reported as not crackable instead.
+* Corrected the uppercasing of the keywords in the shown queries. A word that merely contains a keyword (e.g. a table named `selected`) and a keyword inside a quoted string are now left alone.
+* Corrected the cracking of the old Oracle password hashes when the user name or the password does not come in as text.
+* sqlmap now skips a UNION test whose request holds no character instead of stopping with an exception. Only a hand-edited `payloads/union_query.xml` can hold such a test.
+* The gadget check now runs the gadget itself instead of only looking for the extension. A user that can not use it no longer gets a false success.
 
 ## Quality
 

lib/core/settings.py

@@ -20,7 +20,7 @@
 from thirdparty import six
 
 # sqlmap version (<major>.<minor>.<month>.<monthly commit>)
-VERSION = "1.10.9.1"
+VERSION = "1.10.9.2"
 TYPE = "dev" if VERSION.count('.') > 2 and VERSION.split('.')[-1] != '0' else "stable"
 TYPE_COLORS = {"dev": 33, "stable": 90, "pip": 34}
 VERSION_STRING = "sqlmap/%s#%s" % ('.'.join(VERSION.split('.')[:-1]) if VERSION.count('.') > 2 and VERSION.split('.')[-1] == '0' else VERSION, TYPE)