Skip to content

Split the SSH extension telemetry category by failure outcome - #6497

Draft
anton-107 wants to merge 1 commit into
mainfrom
split-ssh-extension-error-category
Draft

Split the SSH extension telemetry category by failure outcome#6497
anton-107 wants to merge 1 commit into
mainfrom
split-ssh-extension-error-category

Conversation

@anton-107

Copy link
Copy Markdown
Contributor

Changes

CheckIDESSHExtension returns four distinct outcomes as one telemetry category,
IDE_SSH_EXTENSION_MISSING. Split it into four, each attributed through an
errors.Is sentinel rather than message matching:

Outcome Category
--list-extensions failed, so what is installed is unknown IDE_SSH_EXTENSION_LIST_FAILED
Missing or too old, install attempted, IDE rejected it IDE_SSH_EXTENSION_INSTALL_FAILED
User was asked and declined IDE_SSH_EXTENSION_INSTALL_DECLINED
Consent could not be obtained (no --auto-approve, no usable prompt) IDE_SSH_EXTENSION_INSTALL_UNAVAILABLE

IDE_SSH_EXTENSION_MISSING is retired rather than reused, and the enum comment
says so: rows written before this change still carry it, so a query spanning the
release has to accept both spellings.

Error messages change wording slightly, since each now leads with its sentinel.
No acceptance golden depends on the old text.

Why

This category is now the largest single source of failed IDE-mode tunnels, and
the stickiest: of the users whose first attempt hit it, only about an eighth ever
established a tunnel, despite averaging 2.7 attempts. It is also the category we
can say least about, because the four outcomes need different fixes — a list
failure means the check never ran, while a rejected install points at the
marketplace or a policy forbidding it. One value cannot tell us which to fix, so
this is a prerequisite for fixing the dominant failure rather than a fix itself.

Only LIST_FAILED and INSTALL_FAILED are reachable under --auto-approve,
which the VS Code extension's tunnel button always passes. The split therefore
also separates button traffic from direct CLI use, which the single category
could not.

The companion universe proto PR adds the same four values to
SshTunnelErrorCategory and deprecates the old one. Until it lands the new names
are dropped at ingestion (unknown enum values are ignored), so this PR is safe to
merge in either order — it just is not queryable until both are in.

No changelog fragment: this is internal telemetry under experimental/, matching
#4881, #6058 and #6321.

Tests

Six new or extended unit tests:

  • TestSshExtensionErrorCategory pins the sentinel-to-category mapping, including
    through wrapping, and that an unsentinelled error falls back to UNKNOWN.
  • TestCheckIDESSHExtension_ListFails covers a command that resolves on PATH but
    whose --list-extensions exits non-zero — the case that must not read as an
    install problem.
  • TestCheckIDESSHExtension_AutoApprove_InstallFails covers a rejected
    --install-extension, the one outcome the IDE button can produce here.
  • The three existing missing/outdated/no-prompt tests now also assert the
    sentinel they carry.

go build ./..., go vet, the full ./experimental/ssh/... and
./libs/telemetry/... suites, and ./task lint-q are all clean. I also rendered
each new error message once by hand to check they read correctly with the
sentinel prefix.

This pull request and its description were written by Isaac.


This PR was created with GitHub MCP.

CheckIDESSHExtension reports four distinct outcomes as the single
IDE_SSH_EXTENSION_MISSING category: the --list-extensions call failing, a
declined prompt, no way to ask for consent, and the install itself being
rejected. That category is now the largest source of failed IDE-mode tunnels,
and the four call for different fixes -- a list failure means the check never
ran, while a rejected install points at the marketplace or a policy blocking
it -- so one value cannot tell us which to fix.

Give each outcome its own category, attributed through errors.Is sentinels
rather than message matching. Only LIST_FAILED and INSTALL_FAILED are reachable
under --auto-approve, which the VS Code extension's tunnel button always
passes, so the split also separates button traffic from direct CLI use.

IDE_SSH_EXTENSION_MISSING is retired rather than reused: rows written before
this change still carry it, so a query spanning the release has to accept both.

Co-authored-by: Isaac <no-reply@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant