feat(config): resolve settings through a layered chain

README.md

@@ -42,15 +42,41 @@ pip install git+https://github.com/DOI-USGS/dataretrieval-python.git
 Access USGS water-monitoring data.
 
 **Important:** Users are strongly encouraged to obtain an API key for higher
-rate limits. [Register for an API key](https://api.waterdata.usgs.gov/signup/)
-and set it as an environment variable:
+rate limits. [Register for an API key](https://api.waterdata.usgs.gov/signup/),
+then supply it in whichever of these ways suits you. They are listed from
+highest to lowest precedence, so an explicit block or deployment environment
+can override a file without editing it:
 
 ```python
-import os
+# 1. a configure() block - for one call, an interactive prompt, or when
+#    different threads/tasks need different credentials.
+from getpass import getpass
 
-os.environ["API_USGS_PAT"] = "your_api_key_here"
+import dataretrieval
+from dataretrieval import waterdata
+
+with dataretrieval.configure(api_key=getpass("USGS API key: ")):
+    df, metadata = waterdata.get_daily(monitoring_location_id="USGS-01646500")
+```
+
+```bash
+# 2. an environment variable (the R dataRetrieval package uses the same
+#    variable, so one export serves both)
+export API_USGS_PAT="your_api_key_here"
+```
+
+```toml
+# 3. ~/.dataretrieval/config.toml - keeps the key out of your shell
+#    environment, where every process you start inherits it.
+#    Restrict it afterwards:  chmod 600 ~/.dataretrieval/config.toml
+api_key = "your_api_key_here"
 ```
 
+`dataretrieval.show_config()` reports what is in effect and where each setting
+came from, without printing the key. Concurrency, retries, and the progress
+line are configured the same way — see the
+[configuration guide](https://doi-usgs.github.io/dataretrieval-python/userguide/configuration.html).
+
 The following example retrieves daily streamflow data for a specific
 monitoring location. The `/` in the `time` argument separates the start and
 end of the desired range:
@@ -112,12 +138,14 @@ By default the getters split a multi-value request only as far as the server's
 ~8 KB URL limit forces — the fewest sub-requests. For a **large, paginated**
 pull that is needlessly conservative: every sub-request pages through its own
 results, so dividing the query into more, smaller sub-requests lets those pages
-be fetched **in parallel**. `parallel_chunks(n)` opts a single call into that
-finer split, fanning it out into `n` sub-requests. It pays off only when the
-result is large enough to span many pages *and* the query has a multi-value
-argument to divide (such as a list of monitoring locations); on a small query —
-or one with nothing to split — it just adds requests, so it is a deliberate,
-scoped `with` block, never the default.
+be fetched **in parallel**. `parallel_chunks(n)` opts a single call into finer
+optional splitting, up to `n` sub-requests when the input divides that way. It
+pays off only when the result spans many pages and the query has a multi-value
+argument to divide (such as a list of monitoring locations). A query with
+nothing to split remains one request. On a small but splittable query, extra
+chunks only spend quota, so the scoped block is the recommended usage. A
+deliberate config-file or profile baseline is also available for repeated large
+pulls.
 
 ```python
 from dataretrieval import waterdata
@@ -126,37 +154,39 @@ from dataretrieval import waterdata
 # enough to span many pages, so it profits from a finer split.
 sites, _ = waterdata.get_monitoring_locations(state="Ohio", site_type_code="ST")
 
-with waterdata.parallel_chunks(32):  # fan out into 32 sub-requests
+with waterdata.parallel_chunks(32):  # request up to 32 optional chunks
     df, md = waterdata.get_daily(
         monitoring_location_id=sites["monitoring_location_id"],
         parameter_code="00060",  # discharge
         time="2004-01-01/2023-12-31",
     )
 ```
 
-`n` is the number of sub-requests to fan the call out into. It is capped by how
-many values there are to split, and each sub-request costs a request against
-your hourly [rate limit](https://api.waterdata.usgs.gov/signup/); since how many
-run *at once* is capped separately by `API_USGS_CONCURRENT` (default 32), the
-useful range is roughly `2` up to that value.
+`n` is the ceiling for optional refinement, not a hard ceiling on URL-safety
+chunking: an oversized request may already require more than `n` sub-requests,
+while indivisible inputs may produce fewer. Each sub-request costs a request
+against your hourly [rate limit](https://api.waterdata.usgs.gov/signup/). How
+many run *at once* is controlled separately by the effective `concurrency`
+setting (`API_USGS_CONCURRENT`), so the useful optional range is
+roughly `2` up to that value.
 
 Benchmark — a fixed 271-site subset of Ohio stream gages
 (`get_daily`, `parameter_code="00060"`), with a small fixed page size
 (`limit=250`) so every run fetches roughly the same number of pages (isolating
 the effect of parallelism). Each `n` was run against its own cold 1-year time
 window so no run is served from the server's data-window cache:
 
-| `n`  | parallelism | pages | wall-clock              | speedup |
-| ---- | ----------- | ----- | ----------------------- | ------- |
-| off  | 1           | ~30   | 9.5 s / 9.1 s (2 runs)  | 1×      |
-| `8`  | 8           | ~32   | 2.2 s / 1.9 s           | ~4.5×   |
-| `32` | 32          | 54    | 1.2 s                   | ~8×     |
+| `n`  | optional fan-out | pages | wall-clock             | speedup |
+| ---- | ---------------- | ----- | ---------------------- | ------- |
+| off  | 1                | ~30   | 9.5 s / 9.1 s (2 runs) | 1×      |
+| `8`  | 8                | ~32   | 2.2 s / 1.9 s          | ~4.5×   |
+| `32` | 32               | 54    | 1.2 s                  | ~8×     |
 
 The gain comes from overlapping each sub-request's per-page latency and
 server-side work, so the exact multiplier scales with how many pages the pull
-spans — a larger pull (more pages) has more parallelism to exploit. The extra
-sub-requests each cost quota, so reserve a large `n` for pulls you know are
-large.
+spans — a larger pull gives the executor more independent chunks to schedule.
+The extra sub-requests each cost quota, so reserve a large `n` for pulls you
+know are large.
 
 Visit the
 [API Reference](https://doi-usgs.github.io/dataretrieval-python/reference/waterdata.html)

dataretrieval/__init__.py

@@ -17,6 +17,12 @@
 ``nldi`` requires geopandas (``pip install dataretrieval[nldi]``) and is
 imported on demand: ``from dataretrieval import nldi``.
 
+Settings -- the Water Data API key, fan-out concurrency, retries, the progress
+line -- resolve through :mod:`dataretrieval.config`: a
+``with dataretrieval.configure(...)`` block, then the ``API_USGS_*`` environment
+variables, then ``~/.dataretrieval/config.toml``. ``dataretrieval.show_config()``
+reports what is in effect and where each value came from.
+
 A failed request raises a subclass of :class:`dataretrieval.DataRetrievalError`
 (the taxonomy lives in ``dataretrieval.exceptions``); connection-level failures
 (timeouts, DNS) are wrapped as :class:`dataretrieval.NetworkError`. A large
@@ -31,6 +37,10 @@
 except PackageNotFoundError:
     __version__ = "version-unknown"
 
+# Layered configuration: a ``with configure(...)`` block, the environment, then
+# the config file. The canonical home is ``dataretrieval.config``;
+# the callable is named ``configure`` so it doesn't shadow that module.
+from dataretrieval.config import ConfigError, configure, show_config
 from dataretrieval.exceptions import (
     DataRetrievalError,
     HTTPError,
@@ -62,6 +72,7 @@
 )
 
 from . import (
+    config,
     exceptions,
     ngwmn,
     nwis,
@@ -73,6 +84,11 @@
 )
 
 __all__ = [
+    # layered configuration (canonical home: ``dataretrieval.config``)
+    "config",
+    "configure",
+    "show_config",
+    "ConfigError",
     # service modules
     "ngwmn",
     "nwis",