Fix metadata breaking during overlapping or unparseable adaptor installs - #5077
Open
hirad121 wants to merge 1 commit into
Open
Fix metadata breaking during overlapping or unparseable adaptor installs#5077hirad121 wants to merge 1 commit into
hirad121 wants to merge 1 commit into
Conversation
AdaptorService stored a placeholder for an adaptor before its npm install finished, marked :installing with no path. A second lookup during that window found the placeholder instead of waiting for the real result: for a pinned version this returned a pathless entry that MetadataService couldn't use, and for an unpinned job the placeholder's version was the literal word "latest", which isn't valid semver and raised when compared. AdaptorService is now a GenServer instead of an Agent. An overlapping install for the same package/version is queued behind the one already running rather than started again or served a half-built entry; no placeholder is ever added to the adaptor list, so a lookup only ever sees a real, present adaptor or nothing. Version comparisons no longer raise on a value that isn't valid semver, whether it's the requested version or one already stored, so an unexpected value like "local" or "next" fails the lookup cleanly instead of crashing it. The editor's metadata channel handler also resolves "latest" to a concrete version before calling MetadataService, the same way the worker run payload and AI assistant already do, so the common case of an unpinned job no longer relies on the safety net at all. Fixes OpenFn#5059
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes a race and a raise in
AdaptorServicethat together broke jobmetadata (autocomplete/docs in the editor) whenever an adaptor was still
installing, or when its version wasn't valid semver:
install/2stored a placeholder for anadaptor (
status: :installing, path: nil) before itsnpm installfinished, with no lock around the gap. A second lookup for the same
package during that window found the placeholder instead of waiting
for the real result — for a pinned version this returned a pathless
entry
MetadataServicecouldn't use (silently failing metadata, analert but no crash); for an unpinned job the placeholder's stored
version was the literal word
"latest", which isn't valid semver andraised when compared against.
latestnever got resolved before reaching the adaptor service.The editor's metadata channel handler passed
job.adaptorstraightthrough, unlike the worker run payload (
RunWithOptions) and the AIassistant, which already resolve
latest/localto a concreteversion via
AdaptorRegistry.resolve_adaptor/1first.Fix, three parts:
AdaptorServiceis now aGenServerinstead of anAgent. Anoverlapping install for the same package/version is queued behind the
one already in flight (
Task.Supervisor.async_nolink, repliesdeferred via
GenServer.reply/2) rather than started again or serveda half-built entry. No placeholder is ever added to the adaptor list
at all, so a lookup only ever sees a real, present adaptor or nothing.
semver — neither the requested version nor a stored one. An
unexpected value (
local,next, empty) now just fails the lookupcleanly instead of crashing it.
request_metadatachannel handler resolveslatestviaAdaptorRegistry.resolve_adaptor/1before callingMetadataService,matching the two other call sites that already do this.
Closes #5059
Validation steps
web pod (or restart the app so nothing is installed) — metadata
loads normally once the install finishes, no error/alert, even if
you trigger it from two tabs/jobs at once for the same adaptor.
@openfn/language-common@latestnowresolves to a concrete version before the adaptor lookup — no raise,
no alert, real metadata.
AdaptorService.find_adaptor/2/install/2called with a versionthat isn't valid semver (
local,next, empty string) returns aclean "not found" instead of raising.
Additional notes for the reviewer
lib/lightning/adaptor_service.ex(the actual fix),lib/lightning_web/channels/workflow_channel.ex(10 lines — resolveslatestbefore the lookup), two test files,test/test_helper.exs(one
Mimic.copy/1line, needed to stubMetadataServicein the newchannel-level test),
CHANGELOG.md.AdaptorService.install/2has exactly one external caller in thewhole codebase (
MetadataService.get_adaptor_path/1) andfind_adaptor/2/get_adaptors/1/installed?/2have none outside themodule's own tests — checked before changing the module's internals,
so this is a contained, self-consistent change, not a public-API break.
gated by project role, only by
AdaptorRegistry's allowlist check(
exists?/2), which is unchanged and still runs first, unconditionally,before anything else in
install/2.(via
git stashon just the changed file) to confirm they actuallydetect the bug, not just pass vacuously.
mix format,mix credo --strict --all,mix dialyzer, andmix sobelowall pass with no new findings(
sobelow's output is byte-identical before/after this diff).AI Usage
Please disclose whether you've used AI anywhere in this PR (it's cool, we just
want to know!):
You can read more details in our
Responsible AI Policy
Pre-submission checklist
/reviewwith Claude Code)
(e.g.,
:owner,:admin,:editor,:viewer)