Split the SSH extension telemetry category by failure outcome - #6497
Draft
anton-107 wants to merge 1 commit into
Draft
Split the SSH extension telemetry category by failure outcome#6497anton-107 wants to merge 1 commit into
anton-107 wants to merge 1 commit into
Conversation
CheckIDESSHExtension reports four distinct outcomes as the single IDE_SSH_EXTENSION_MISSING category: the --list-extensions call failing, a declined prompt, no way to ask for consent, and the install itself being rejected. That category is now the largest source of failed IDE-mode tunnels, and the four call for different fixes -- a list failure means the check never ran, while a rejected install points at the marketplace or a policy blocking it -- so one value cannot tell us which to fix. Give each outcome its own category, attributed through errors.Is sentinels rather than message matching. Only LIST_FAILED and INSTALL_FAILED are reachable under --auto-approve, which the VS Code extension's tunnel button always passes, so the split also separates button traffic from direct CLI use. IDE_SSH_EXTENSION_MISSING is retired rather than reused: rows written before this change still carry it, so a query spanning the release has to accept both. Co-authored-by: Isaac <no-reply@databricks.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
CheckIDESSHExtensionreturns four distinct outcomes as one telemetry category,IDE_SSH_EXTENSION_MISSING. Split it into four, each attributed through anerrors.Issentinel rather than message matching:--list-extensionsfailed, so what is installed is unknownIDE_SSH_EXTENSION_LIST_FAILEDIDE_SSH_EXTENSION_INSTALL_FAILEDIDE_SSH_EXTENSION_INSTALL_DECLINED--auto-approve, no usable prompt)IDE_SSH_EXTENSION_INSTALL_UNAVAILABLEIDE_SSH_EXTENSION_MISSINGis retired rather than reused, and the enum commentsays so: rows written before this change still carry it, so a query spanning the
release has to accept both spellings.
Error messages change wording slightly, since each now leads with its sentinel.
No acceptance golden depends on the old text.
Why
This category is now the largest single source of failed IDE-mode tunnels, and
the stickiest: of the users whose first attempt hit it, only about an eighth ever
established a tunnel, despite averaging 2.7 attempts. It is also the category we
can say least about, because the four outcomes need different fixes — a list
failure means the check never ran, while a rejected install points at the
marketplace or a policy forbidding it. One value cannot tell us which to fix, so
this is a prerequisite for fixing the dominant failure rather than a fix itself.
Only
LIST_FAILEDandINSTALL_FAILEDare reachable under--auto-approve,which the VS Code extension's tunnel button always passes. The split therefore
also separates button traffic from direct CLI use, which the single category
could not.
The companion universe proto PR adds the same four values to
SshTunnelErrorCategoryand deprecates the old one. Until it lands the new namesare dropped at ingestion (unknown enum values are ignored), so this PR is safe to
merge in either order — it just is not queryable until both are in.
No changelog fragment: this is internal telemetry under
experimental/, matching#4881, #6058 and #6321.
Tests
Six new or extended unit tests:
TestSshExtensionErrorCategorypins the sentinel-to-category mapping, includingthrough wrapping, and that an unsentinelled error falls back to
UNKNOWN.TestCheckIDESSHExtension_ListFailscovers a command that resolves on PATH butwhose
--list-extensionsexits non-zero — the case that must not read as aninstall problem.
TestCheckIDESSHExtension_AutoApprove_InstallFailscovers a rejected--install-extension, the one outcome the IDE button can produce here.sentinel they carry.
go build ./...,go vet, the full./experimental/ssh/...and./libs/telemetry/...suites, and./task lint-qare all clean. I also renderedeach new error message once by hand to check they read correctly with the
sentinel prefix.
This pull request and its description were written by Isaac.
This PR was created with GitHub MCP.