Skip to content

Split integration dependencies into extras #175

Description

@MrlixiangWE

pip install apache-otava installs 364.3 MB across 48 packages. A user who only reads CSV files needs 153.5 MB across 19 of them; the rest is there for integrations they may never configure. #55 raises the same thing from the dependency-count side.

extra covers needs cost over core
CSV, JSON, HistoStat, Graphite, analysis stdlib only core is 153.5 MB / 19 pkgs
influxdb importer influxdb3-python +131.9 MB / +5 pkgs
bigquery importer + --update-bigquery google-cloud-bigquery +74.1 MB / +22 pkgs
slack --notify-slack slack-sdk +4.6 MB / +1 pkg
grafana --update-grafana, remove-annotations requests +3.9 MB / +5 pkgs
postgres importer + --update-postgres pg8000 +1.7 MB / +3 pkgs

Sizes measured on Python 3.14, wheels only.

The InfluxDB importer in #170 grew the default install from 233.8 MB to 364.3 MB, +56% in one PR. Every data source added after it does the same.

Is it time to change how Otava is installed?

command installs
pip install apache-otava core, 153.5 MB
pip install "apache-otava[bigquery]" core + BigQuery
pip install "apache-otava[bigquery,postgres]" core + both
pip install "apache-otava[all]" everything, as today
docker pull apache/otava unchanged, the Dockerfile installs [all]

Core is always there and the extras only add to it, so there is one package name and one thing to learn. An existing install that upgrades without naming an extra drops to the core: CSV, JSON, HistoStat and Graphite users would see no difference, and everyone else gets a message naming the extra to add. Keeping pip install apache-otava complete would need a second distribution, since extras only add, and that costs a second wheel and sdist to sign and publish on every release.

main.py imports slack_sdk at module level, and config.py imports the configuration dataclasses from bigquery.py, postgres.py, influxdb.py and grafana.py, each of which imports its client at module level, so otava --help currently needs all five installed. The extras would come with:

  • moving the third-party imports into the client constructors, with TYPE_CHECKING imports for the annotations, so the CLI, the config parser and the analysis core stay importable
  • a small helper that reports pip install 'apache-otava[bigquery]' when a package is missing, instead of an ImportError raised from an unrelated module
  • constructing the Slack notifier lazily. Otava.__init__ calls __maybe_create_slack_notifier() unconditionally and config.py always builds a SlackConfig, so the if not self.__conf.slack guard never fires. __get_grafana, __get_postgres and __get_bigquery next to it are all lazy

uv sync --all-extras --dev and tox -e py still install everything, so the current CI matrix is unaffected; a core-only job would keep the smaller path from regressing.

@Gerrrr, is this something the project wants? Or do you have other suggestions? I'd be glad to open a PR if you want it.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions