Skip to content

[runtime] Resolve relative file: URLs when loading classpath skills (#966) - #1033

Open
Olawoyin007 wants to merge 2 commits into
apache:mainfrom
Olawoyin007:fix-966-relative-file-url
Open

[runtime] Resolve relative file: URLs when loading classpath skills (#966)#1033
Olawoyin007 wants to merge 2 commits into
apache:mainfrom
Olawoyin007:fix-966-relative-file-url

Conversation

@Olawoyin007

Copy link
Copy Markdown
Contributor

What is the purpose of the change

Fixes #966.

A Flink deployment can add user-code JARs relative to the TaskManager working directory, so a classloader may expose a skill resource through a relative file: URL such as file:../../flink/usrlib/job.jar (or jar:file:../../flink/usrlib/job.jar!/diagnosis-skills). Such a URL parses to an opaque URI, which new File(URI) / Paths.get(URI) reject with IllegalArgumentException: URI is not hierarchical, so ClasspathSkillRepository initialization fails and skills never load.

Brief change log

  • Add LocalUrls.toLocalFile(URL), a single place that converts a local file: URL to a File:
    • absolute hierarchical file: URLs keep their existing new File(uri) behavior;
    • opaque relative file: URLs are resolved against the process working directory via the decoded scheme-specific part;
    • non-file URLs are rejected explicitly.
  • Reuse it in the three affected sites so their behavior can't drift:
    • ClasspathSkillRepository.materializeFileUrl (direct file/dir/zip resource),
    • SkillMaterializer.copyJarEntries (JAR entry extraction),
    • ClasspathSkillRepository.findAllMatches (URLClassLoader fallback scan).
  • Remove now-unused imports.

Verifying this change

  • New LocalUrlsTest covers absolute resolution, relative/opaque resolution against the working directory, and non-file rejection.
  • New ClasspathSkillRepositoryTest#loadFromRelativeJarUrl loads skills end-to-end through a relative jar URL (exercises both the fallback scan and jar extraction). Verified that this test fails against the pre-fix code with the reported Classpath resource not found / URI is not hierarchical error.
  • Full runtime module test suite passes (668 tests, 0 failures).

Does this pull request potentially affect one of the following parts

  • Dependencies (does it add or upgrade a dependency): no
  • The public API: no
  • The serializers: no
  • The runtime per-record code paths: no
  • Anything that affects deployment or recovery: no
  • The threading model: no

Documentation

  • Does this pull request introduce a new feature? no

@github-actions github-actions Bot added doc-label-missing The Bot applies this label either because none or multiple labels were provided. fixVersion/0.4.0 priority/major Default priority of the PR or issue. labels Aug 21, 2026
…pache#966)

A Flink deployment can add user-code JARs relative to the TaskManager
working directory, so a classloader may expose a skill resource through a
relative file: URL such as file:../../flink/usrlib/job.jar. That URL parses
to an opaque URI, which new File(URI) / Paths.get(URI) reject with "URI is
not hierarchical", failing skill initialization.

Centralize local file: URL resolution in LocalUrls.toLocalFile, which
resolves an opaque relative URI against the process working directory and
keeps existing behavior for absolute hierarchical file URLs. Reuse it in
the three affected sites: direct file resource materialization, JAR entry
extraction, and the URLClassLoader fallback scan. Non-file URLs are
rejected explicitly.

Add a LocalUrls unit test and an end-to-end ClasspathSkillRepository test
that loads skills through a relative jar URL.
@Olawoyin007
Olawoyin007 force-pushed the fix-966-relative-file-url branch from 6143698 to 991ff6b Compare August 21, 2026 20:10
@wenjin272 wenjin272 added doc-not-needed Your PR changes do not impact docs and removed doc-label-missing The Bot applies this label either because none or multiple labels were provided. labels Aug 23, 2026

@wenjin272 wenjin272 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this fix! The overall approach looks good. I left two inline comments about an exception-handling edge case and regression-test coverage.

// directory, which is exactly how a relative File is interpreted.
return new File(uri.getSchemeSpecificPart());
}
return new File(uri);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we also wrap the IllegalArgumentException thrown by new File(uri)? For example, file://host/share/job.jar is hierarchical, but new File(uri) throws IllegalArgumentException: URI has an authority component. Since the callers only catch IOException, this escapes the fallback scan instead of being skipped and also changes the previous copyJarEntries() exception contract. Shall we keep the conversion inside the try and wrap both URISyntaxException and IllegalArgumentException as IOException?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, fixed in 05b4a46. new File(uri) is now inside the try, and the catch handles both URISyntaxException and IllegalArgumentException, wrapping them as IOException. So a hierarchical-but-non-representable URL like file://host/share/job.jar now surfaces as IOException and is skipped by the fallback scan, restoring the previous copyJarEntries contract instead of escaping unchecked. Added a LocalUrlsTest case asserting the authority-component URL comes back as IOException.

}

@Test
void loadFromRelativeJarUrl(@TempDir Path tempDir) throws IOException {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test covers the URLClassLoader fallback scan and JAR extraction paths, but the PR also changes direct resource materialization. Could we add regression cases where getResources() directly returns a relative jar:file: URL and a relative file: directory URL? That would exercise all three affected call paths described in the PR.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, added in 05b4a46. Two new regression tests feed the URLs through getResources() directly (via a non-URLClassLoader loader, so the fallback scan is bypassed and each path is exercised in isolation): loadFromDirectRelativeJarUrl returns a relative jar:file: URL to drive SkillMaterializer.copyJarEntries, and loadFromDirectRelativeDirectoryUrl returns a relative file: directory URL to drive ClasspathSkillRepository.materializeFileUrl. Together with the existing fallback-scan test, all three affected call paths are now covered.

…apache#966)

Address review feedback on the relative file: URL fix.

LocalUrls.toLocalFile only wrapped URISyntaxException, so a hierarchical
but non-representable file: URL (for example file://host/share/job.jar,
which new File(URI) rejects with "URI has an authority component") escaped
as an unchecked IllegalArgumentException. Callers only catch IOException,
so this broke the fallback scan's graceful skip and changed the previous
copyJarEntries exception contract. Move the conversion inside the try and
wrap both URISyntaxException and IllegalArgumentException as IOException.

Add regression tests for the two directly-affected call paths not yet
covered: getResources() returning a relative jar:file: URL (drives
SkillMaterializer.copyJarEntries) and a relative file: directory URL
(drives ClasspathSkillRepository.materializeFileUrl), plus a LocalUrls
test that the authority-component URL surfaces as IOException.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

doc-not-needed Your PR changes do not impact docs fixVersion/0.4.0 priority/major Default priority of the PR or issue.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] ClasspathSkillRepository fails to load skills from relative file URLs

2 participants