Skip to content

feat(metrics): extend node_info and add node status gauges - #7049

Open
warku123 wants to merge 26 commits into
tronprotocol:release_v4.8.3from
warku123:feature/sr-status-metrics
Open

warku123 wants to merge 26 commits into
tronprotocol:release_v4.8.3from
warku123:feature/sr-status-metrics

Conversation

@warku123

@warku123 warku123 commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

What does this PR do?

Adds four low-cost node-status metrics and removes the earlier design that computed SR status during maintenance periods.

  • tron:node_info (Info, labels version, genesis_block_id): static node identity — the running version string plus the full genesis block hash as the canonical chain identifier.
  • tron:node_type (Gauge, no labels): startup identity, 0 = fullnode, 1 = witness. Published once in ConsensusService.start(); the identity is fixed at startup and cannot change at runtime.
  • tron:node_role (Gauge, no labels): backup-election role, 0 = init, 1 = master, 2 = slaver. Published by BackupManager at startup and refreshed on every status transition. Independent of node type: both fullnodes and witness nodes report their backup role. A node without backup configured reports master; a node with backup configured but not yet initialized reports init.
  • tron:witness_address{address="..."} (Gauge, value 1): one series per configured consensus witness account address, Base58Check-encoded. Published once in ConsensusService.start(); a plain fullnode publishes no series. When a single key signs for an explicitly configured witness account, the witness account address is used rather than the signing-key address.

Why are these changes required?

Operators need to distinguish chain/version, startup identity, and primary/backup role from metrics alone. Identity and role must be separate series: encoding fullnode as a role value would mask the backup state of a fullnode that has backup configured. All values are published at startup or on status transitions — no maintenance-period computation and no database access.

This PR has been tested by:

  • Unit Tests: 35 tests passed across SRMetricsTest (19), BackupManagerTest (10), and PrometheusApiServiceTest (6), covering metric values and text export, identity/role independence, partial and absent backup configuration, disabled metrics, and single-node recovery from init to master through the real backup initialization and keep-alive timer path.
  • Checkstyle: :framework:checkstyleMain, :framework:checkstyleTest, and :plugins:checkstyleMain passed.

Follow up

None.

Extra details

No election, consensus, or maintenance-period behavior changes: the backup state machine, keep-alive loop, and member resolution are untouched; metric publication failures are isolated from startup and status transitions. When backup is configured with no reachable peer, the node reports init until the keep-alive timeout (default ~18s) promotes it to master.

@github-actions
github-actions Bot requested a review from halibobo1205 October 20, 2026 03:41
@warku123 warku123 closed this Oct 20, 2026
@warku123 warku123 reopened this Oct 20, 2026
@SeriousCoding789
SeriousCoding789 self-requested a review October 20, 2026 05:20
Comment thread common/src/main/java/org/tron/common/prometheus/MetricsInfo.java Outdated
Comment thread framework/src/main/java/org/tron/common/backup/BackupManager.java Outdated
Comment thread framework/src/main/java/org/tron/program/FullNode.java Outdated
Stub BackupManager and MaintenanceManager lookups in both mocked startup contexts. Verify startup, node info, witness gauges and shutdown ordering without changing production code.
Address review feedback: keep node_info in its original form
(version, genesis_block_id) recorded in FullNode, and move the
witness address and role into a dedicated one-hot gauge
tron:node_role{address, role} where the full role set
(init/master/slaver) is always written and only the current role is
1. The gauge is exposed on witness-configured nodes only.

BackupManager now derives the role: with no backup election
configured the node is master; with backup configured but the
election not yet initialized it reports init, covering the pre-init
window where the status field default previously read as an elected
master. MetricsInfo is reverted to its original two-label
implementation, removing the mutable-label mechanism and the
label-swap race with concurrent scrapes.
tron:node_role now exposes one series per witness address whose value
encodes the role: 0=init, 1=master, 2=slaver. This replaces the
previous one-hot form that published three series per address with a
role label.

- SRMetrics owns the mapping as package-visible constants shared with
  tests; unknown role names fall back to init so a misderived state
  cannot masquerade as an elected one.
- The gauge is registered with the single label 'address' and its help
  text documents the numeric mapping.
- docs/metrics-changelog.md updated with the 0/1/2 mapping table.
- Tests updated to assert the numeric encoding; a plain fullnode with
  no local witness still exposes no series.
Comment thread framework/src/main/java/org/tron/common/backup/BackupManager.java Outdated
Comment thread framework/src/main/java/org/tron/common/backup/BackupManager.java
Comment thread framework/src/main/java/org/tron/common/backup/BackupManager.java Outdated
Review follow-ups on the backup role gauge:

- Post-init the election status is a real outcome, so map it 1:1:
  INIT -> init(0), MASTER -> master(1), SLAVER -> slaver(2).
  Previously a post-init INIT was misreported as slaver(2). The
  pre-init window is still gated by isInit, since the status field
  initializer (MASTER) is not an election result and cannot change:
  BlockHandleImpl.getState() relies on the MASTER default to decide
  whether this node may produce a block, and a node without backup
  configured never runs init().
- Multi-key setups (testnets) get one node_role series per
  configured witness identity: the witness account address when set,
  otherwise one address derived per private key, all carrying the
  same node-level role value.
Comment thread docs/metrics-changelog.md Outdated
Comment thread framework/src/main/java/org/tron/common/backup/BackupManager.java Outdated
Comment thread framework/src/main/java/org/tron/common/backup/BackupManager.java Outdated
- sr_is_active / sr_votes / sr_rank now carry an address label and emit
  one series per configured local witness address, mirroring
  tron:node_role; a node without a local witness keeps the existing
  empty-address zero-value series
- SRMetrics caches the full address set resolved by BackupManager
  (explicit witness account address, or one address per configured
  private key); MaintenanceManager scans the witness list per address
- a malformed address no longer skips recording for the remaining
  addresses until the next maintenance round (per-address catch + warn)
- gauge help text: rank is 0 when the witness is unknown
localWitnessAddresses() preferred getWitnessAccountAddress(), but the production
assembly path (WitnessInitializer -> LocalWitnesses.initWitnessAccountAddress)
always derives it from the first private key, leaving it non-null even when
multiple keys are configured. Multi-key nodes therefore published role/sr
series only for the first key, while every key is its own SR identity.

Branch on the private key count instead: with several keys (an explicit
witness account address is rejected at startup in that case) derive one
address per key; with a single key keep using the witness account address,
covering multisig setups where it differs from the derived one.

Add a regression test that builds LocalWitnesses through the production
assembly path (WitnessInitializer.initFromCFGPrivateKey) with two keys and
asserts both derived addresses get role series. Also fix a stale NODE_ROLE
comment that still said one-hot after the numeric encoding change.
Comment thread common/src/main/java/org/tron/common/prometheus/MetricsGauge.java
Comment thread consensus/src/main/java/org/tron/consensus/dpos/MaintenanceManager.java Outdated
Comment thread common/src/main/java/org/tron/common/prometheus/SRMetrics.java Outdated
Comment thread framework/src/main/java/org/tron/program/FullNode.java Outdated

@halibobo1205 halibobo1205 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Before merge: the title still says "extend node_info", but node_info is unchanged now. Since it becomes the squash subject, could it be updated, e.g. feat(metrics): add node role and witness address gauges?

}
for (Miner miner : miners) {
ByteString witnessAddress = miner.getWitnessAddress();
if (witnessAddress == null) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can delete this if (witnessAddress == null) {, since the below already has a try/catch.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. The existing per-miner try/catch already handles address conversion failures, so I removed the redundant null check locally.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

3 participants