Repository navigation
feat: alert when an agreement holder's DIPs info stays down - #767
Merged
Merged
Conversation
A new table keeps, for each indexer holding an accepted agreement, how long its /dips/info has given no readable answer. Each cycle adds time for the missing ones and forgets the rest.
…able IISA can't score an indexer without DIPs info, so nothing replaces it. A new watcher asks each agreement holder for its DIPs info every 5 minutes and alerts after 24 hours without an answer.
Counting missing DIPs info can stay paused for as long as most agreement holders lack it, as after a release that changes /dips/info, so its alert now repeats instead of firing once.
A missing share hovering around the trip fraction paused and resumed counting every cycle, repeating its alert. Counting now resumes after 3 calm cycles in a row.
Slack posts 1 message per event in each throttle window and only counts the rest, so indexers reaching the limit together lost their IDs. 1 alert per cycle now names all of them.
A cycle that failed to read or write the database threw away the time since the last one, so missing DIPs info added up slower. That time now carries into the next recorded cycle.
A stop waited for every /dips/info request in the cycle, up to 10 seconds each, well past the 5 seconds shutdown allows each service. It now drops the cycle before anything is recorded.
A check that crashed left its indexer off the missing list, so its built-up time was wiped as if it had answered. It now counts as missing, and the error log names the indexer.
A timeout of 0 makes every /dips/info request fail at once, so every agreement holder would look as if its DIPs info were missing. Dipper now refuses to start with it.
A huge interval let a cycle's time overflow the stored missing-time count, which then failed to save on every cycle. Dipper now refuses an interval over 1 day.
The setting's description said the share applied when none of the indexers answered, which misread it. It now says that share answering unreadably at once pauses counting.
MoonBoi9001
marked this pull request as ready for review
October 30, 2026 14:50
…ose-dips-info-stays-down
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
IISA never replaces a current indexer it has no readable DIPs info for, so an indexer can remove its
/dips/infoendpoint to avoid being replaced for poor performance. This adds a watcher that checks every agreement holder's DIPs info every 5 minutes, reading it as IISA's score job does, and raisesdips_info_missingafter 24 hours without a readable answer, not counting cycles in which half or more fail at once.Important
Off until the deployed config sets
dips_info_watcher.enabledtotrue.