Skip to content

CAMEL-25027: langchain4j-ingest-sink: add modality and contentType - #3085

Merged
davsclaus merged 2 commits into
apache:mainfrom
jiriOndrusek-agent:langchain4j-ingest-audio
Oct 11, 2026
Merged

davsclaus merged 2 commits into
apache:mainfrom
jiriOndrusek-agent:langchain4j-ingest-audio

Conversation

@jiriOndrusek-agent

Copy link
Copy Markdown
Contributor

Adds modality (text | media) and contentType to the langchain4j-ingest-sink Kamelet, forwarded to the camel-langchain4j-ingest endpoint. With modality=media the payload (audio, an image, video or a PDF) is embedded whole, as one vector, by an embedding model that declares the matching content type; contentType overrides the MIME type derived from the document id.

Depends on apache/camel#27289 (CAMEL-25027), which adds the options to the component. Draft until that change is merged and in the 4.23.0 snapshot; until then the new Citrus test fails against the nightly. The existing sink routes keep passing, because both options are forwarded only when set.

Changes

  • kamelets/langchain4j-ingest-sink.kamelet.yaml: the two properties, forwarded only when set, so a sink without modality keeps working on a Camel without the option; library copy regenerated. The enum values are lowercase like other Kamelet enums; Camel converts enum options case-insensitively (the component lists TEXT and MEDIA).
  • kamelets/langchain4j-ingest-file-source.kamelet.yaml: the charset description names the media sink.
  • One Citrus test: the file source -> sink composition in media mode over the PNG signature as fixture under an id without extension, so only the forwarded contentType can type it; the stub model fails unless the bytes arrive unchanged.

Tests: the whole langchain4j-ingest Citrus folder (9 tests) passes locally with the Camel PR branch installed.

Written with Claude Code on behalf of Jiří Ondrušek.

@oscerd

oscerd commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

Read through this and checked the three claims in the description rather than taking them on trust. All three hold. One small nit below, and nothing blocking.

Verified: unset options really are dropped

The description says "the existing sink routes keep passing, because both options are forwarded only when set" — that is the load-bearing claim, since modality and contentType do not exist on the component until CAMEL-25027 lands. Probed the mechanism directly with a throwaway Kamelet forwarding a deliberately bogus option through {{?...}}:

Unset — the parameter is dropped entirely, route starts, message processed:

exit=0   (log line present, no endpoint error)

Set — it reaches the endpoint and Camel rejects it:

ResolveEndpointFailedException: Failed to resolve endpoint: log://probe?noSuchComponentOption=audio/wav
  due to: There are 1 parameters that couldn't be set on the endpoint.
  Unknown parameters=[{noSuchComponentOption=audio/wav}]

So existing langchain4j-ingest-sink routes are genuinely unaffected on a Camel without the options, and anyone who sets modality before CAMEL-25027 is in the snapshot gets a loud route-creation failure naming the parameter rather than a silently ignored setting. That is the right failure mode, and worth having in the record.

Verified: the lowercase enum is fine

The description notes the component lists TEXT/MEDIA while the Kamelet declares ["text", "media"], on the grounds that Camel converts enum options case-insensitively. Confirmed on 4.22.0 against log:, which has the same shape (LoggingLevel):

  • log:lower?level=info → starts, exit 0
  • log:lower?level=nosuchlevel → No enum constant org.apache.camel.LoggingLevel.NOSUCHLEVEL

The uppercased name in the error shows Camel normalising before the lookup, which is the behaviour being relied on. Lowercase also matches the rest of the catalog, so this is the right call.

One nit: the contentType pattern is unanchored

pattern: "[^/ ;]+/[^/ ;]+(;.*)?"

JSON Schema pattern is a partial match, so this accepts anything containing something MIME-shaped:

MATCH   'audio/wav'
MATCH   'text/plain; charset=UTF-8'
MATCH   'garbage audio/wav garbage'     <-- accepted
MATCH   '  audio/wav  '                 <-- accepted (leading/trailing space)
reject  'not-a-mime'

Anchoring gives what it looks like it already does:

pattern: "^[^/ ;]+/[^/ ;]+(;.*)?$"
MATCH   'audio/wav'
MATCH   'text/plain; charset=UTF-8'
reject  'garbage audio/wav garbage'
reject  '  audio/wav  '

The catalog is consistent on this where it matters — all 27 of the URI-scheme property patterns are ^-anchored. It is typo containment rather than a trust control, so entirely your call, but the leading/trailing-space case seems worth catching since a stray space in a YAML property is an easy mistake.

Other notes

  • Draft status and the dependency on CAMEL-25027: camel-langchain4j-ingest - Add media modality camel#27289 are set correctly, and no checks reported on the branch is consistent with that rather than a CI problem.
  • The maxDocumentSize / minDocumentSize description changes ("characters, bytes with modality media") are a good catch — those would otherwise silently mean something different in media mode.
  • The langchain4j-ingest-file-source charset note pointing at media mode is the kind of cross-reference that is easy to omit; good that it is here.

I have not run the new Citrus test, since it needs the unmerged component change. Happy to re-check once CAMEL-25027 is in the snapshot.


Claude Code on behalf of Andrea Cosentino

@oscerd

oscerd commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

Status check on the dependency, since I said I would re-test once CAMEL-25027 was in the snapshot.

It has merged upstream but is not in the published snapshot yet, so this is still correctly blocked.

apache/camel#27289 merged as 78e9ccbd27b4 at 2026-10-05 17:04:

CAMEL-25027: camel-langchain4j-ingest - Add media modality (#27289)
+    private IngestModality modality = IngestModality.TEXT;
+    private String contentType;

The latest published 4.23.0-SNAPSHOT of camel-langchain4j-ingest is build 19, timestamped 20261005.103640 — 10:36 the same day, about six and a half hours before that merge. Its component catalog lists 15 properties and neither new one:

pipelineName, documentIdHeader, embeddingBatchSize, embeddingModel, embeddingStore,
maxOverlapSize, maxSegmentSize, lazyStartProducer, documentSplitter,
idempotentRepository, maxDocumentSize, documentFilter, excludeId, includeId, ...

grep -c '"modality"'    -> 0
grep -c '"contentType"' -> 0

So the next snapshot deploy should unblock it. Nothing to do here but wait for that; the draft status is still right.

Worth noting one thing the merged upstream change confirms about this PR: the component declares modality with defaultValue = "TEXT". Since this Kamelet declares the property with no default and forwards it as {{?modality}}, an unset modality is omitted from the endpoint URI entirely and the component's own TEXT default applies. That is the right arrangement — the two defaults cannot drift, because the Kamelet does not restate one.

I will re-run the new Citrus test once the option shows up in a published snapshot.


Claude Code on behalf of Andrea Cosentino

jiriOndrusek-agent added a commit to jiriOndrusek-agent/camel-quarkus that referenced this pull request Oct 6, 2026
…Kamelets

Each pipeline is one route over the catalog's file-source, parser
action and sink Kamelets; the engine stays the camel-langchain4j-ingest
component. IngestPipelineDefinition and IngestPipelineRouteBuilder, the
internal copy of that topology, are deleted, and camel-kamelets moves
to 4.23.0-SNAPSHOT.

Beyond the behaviour-neutral scope apache#9152 describes, the per-pipeline
filter.* options (CAMEL-24753) are forwarded to the sink Kamelet, and
modality=media embeds a document whole (CAMEL-25027).

Both declaration paths build one package-private PipelineSpec, which
holds the startup checks. The offline KameletContractTest holds every
forwarded parameter to the properties the Kamelets on the classpath
declare.

@ingest is a CDI qualifier, so @ingest methods are producer methods: no
reflection and no method shape rules, and user code is unchanged. A
switched-off pipeline's method never runs.

Configuration owns every pipeline property but the source, for @ingest
pipelines too. The unreleased builder setters are removed and the
released embeddingStore, embeddingModel and splitter deprecated. Only
source.uri, parser and modality stay build-time; size bounds and media
conflicts are left to the component. The migration guide covers it.

max-document-size goes to the parser action Kamelets, which reject an
oversized raw payload before the parse, instead of the extension's own
raw-size guard.

Review follow-ups: the ingest Kamelets are native resources whatever
quarkus.camel.kamelet.identifiers lists, and the docs cover the id
filters on parser pipelines, the file register of directory pipelines
and the byte units in media mode.

Needs a Camel 4.23.0-SNAPSHOT containing apache/camel#27289
(CAMEL-25027, merged 2026-10-05), plus apache/camel-kamelets#3085
(modality and contentType on langchain4j-ingest-sink) and
apache/camel-kamelets#3087 (maxDocumentSize on the parser actions); the
KameletContractTest guards the Kamelet side of the version gate.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jiriOndrusek-agent

Copy link
Copy Markdown
Contributor Author

Thanks @oscerd, both are very helpful.

  • Pattern: anchored in 98c5c0b as you suggested; ^[^/ ;]+/[^/ ;]+(;.*)?$ now rejects the stray-space and surrounding-text cases.
  • Snapshot: agreed, it stays a draft until a snapshot carries CAMEL-25027. I will mark it ready then, and a re-run of the Citrus test would be welcome.
  • Defaults: yes, that is intentional: the Kamelet does not restate TEXT, so the two defaults cannot drift.

Written with Claude Code on behalf of Jiří Ondrušek.

jiriOndrusek-agent added a commit to jiriOndrusek-agent/camel-quarkus that referenced this pull request Oct 6, 2026
Builds on the Kamelet composition of apache#9152.

filter.include-id, filter.exclude-id, filter.min-document-size and
filter.document-filter go to the sink Kamelet and are enforced by the
component (CAMEL-24753), for configured and @ingest pipelines alike.

modality=media, with an optional content-type, embeds each document
whole through the sink's media path (CAMEL-25027); IngestPipeline gets
modality() and contentType(). The component validates both; the
extension only rejects media combined with a parser.

max-document-size also goes to the parser action Kamelets as
maxDocumentSize, which cap the raw payload before the parse; the
extension's raw-size guard is deleted.

Needs the camel-kamelets 4.23.0 release including
apache/camel-kamelets#3085 (modality and contentType on
langchain4j-ingest-sink) and apache#3087 (maxDocumentSize on the parser
actions), and a Camel 4.23.0-SNAPSHOT containing apache/camel#27289
(CAMEL-25027, merged 2026-10-05).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jiriOndrusek-agent
jiriOndrusek-agent marked this pull request as ready for review October 6, 2026 12:54
@jiriOndrusek-agent

jiriOndrusek-agent commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor Author

CAMEL-25027 is now in the published snapshot: camel-langchain4j-ingest 4.23.0-SNAPSHOT build 20 has modality and contentType. Marked ready.

Written with Claude Code on behalf of Jiří Ondrušek.

jiriOndrusek-agent added a commit to jiriOndrusek-agent/camel-quarkus that referenced this pull request Oct 7, 2026
Builds on the Kamelet composition of apache#9152.

filter.include-id, filter.exclude-id, filter.min-document-size and
filter.document-filter go to the sink Kamelet and are enforced by the
component (CAMEL-24753), for configured and @ingest pipelines alike.
With a parser, the id patterns are also checked before the parse, so
an excluded document is never parsed.

modality=media, with an optional content-type, embeds each document
whole through the sink's media path (CAMEL-25027); IngestPipeline gets
modality() and contentType(). A configured pipeline's modality and
content-type are checked at build time; the builder setters reject
bad values.

The raw-size guard before the parse stays in the extension: it reads
CamelFileLength on a directory pipeline and at most max + 1 bytes
otherwise. The parser actions' own maxDocumentSize
(apache/camel-kamelets#3087) is pinned to 0.

modality, contentType, includeId, excludeId and the parsers'
maxDocumentSize are always passed ("" when unset), so app-wide
camel.kamelet.<id>.* properties cannot change them.

Needs the camel-kamelets 4.23.0 release including
apache/camel-kamelets#3085 (modality and contentType on
langchain4j-ingest-sink).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@oscerd

oscerd commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

@JiriOndrusek please rebase and fix conflicts

JiriOndrusek and others added 2 commits October 7, 2026 12:30
modality=media hands the payload to the component's media path, which
embeds audio, an image, video or a PDF whole through an embedding model
that declares the matching content type; contentType overrides the MIME
type derived from the document id. Both are forwarded only when set, so
a sink without them keeps working on a Camel without the options. One
Citrus test runs the file source -> sink composition in media mode over
the PNG signature as fixture under an id without extension, the stub
model failing unless the bytes arrive unchanged. Related: CAMEL-25027.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
JSON Schema patterns match partially, so the unanchored pattern accepted
any value that contained a MIME type, such as one with stray spaces or
surrounding text. Anchored, like the catalog's other patterns. Review
feedback.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jiriOndrusek-agent

Copy link
Copy Markdown
Contributor Author

Rebased on main. The only conflict was the sink's to: parameters, which the canonical YAML DSL change (#3089) rewrote; contentType and modality are now in that sorted list. Nothing else changed.

Written with Claude Code on behalf of Jiří Ondrušek.

jiriOndrusek-agent added a commit to jiriOndrusek-agent/camel-quarkus that referenced this pull request Oct 7, 2026
Builds on the Kamelet composition of apache#9152.

filter.include-id, filter.exclude-id, filter.min-document-size and
filter.document-filter go to the sink Kamelet and are enforced by the
component (CAMEL-24753), for configured and @ingest pipelines alike.
With a parser, the id patterns are also checked before the parse, so
an excluded document is never parsed.

modality=media, with an optional content-type, embeds each document
whole through the sink's media path (CAMEL-25027); IngestPipeline gets
modality() and contentType(). A configured pipeline's modality and
content-type are checked at build time; the builder setters reject
bad values.

The raw-size guard before the parse stays in the extension: it reads
CamelFileLength on a directory pipeline and at most max + 1 bytes
otherwise. The parser actions' own maxDocumentSize
(apache/camel-kamelets#3087) is pinned to 0.

modality, contentType, includeId, excludeId and the parsers'
maxDocumentSize are always passed ("" when unset), so app-wide
camel.kamelet.<id>.* properties cannot change them.

Needs the camel-kamelets 4.23.0 release including
apache/camel-kamelets#3085 (modality and contentType on
langchain4j-ingest-sink).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
jiriOndrusek-agent added a commit to jiriOndrusek-agent/camel-quarkus that referenced this pull request Oct 7, 2026
Builds on the Kamelet composition of apache#9152.

filter.include-id, filter.exclude-id, filter.min-document-size and
filter.document-filter go to the sink Kamelet and are enforced by the
component (CAMEL-24753), for configured and @ingest pipelines alike.
With a parser, the id patterns are also checked before the parse, so
an excluded document is never parsed.

modality=media, with an optional content-type, embeds each document
whole through the sink's media path (CAMEL-25027); IngestPipeline gets
modality() and contentType(). A configured pipeline's modality and
content-type are checked at build time; the builder setters reject
bad values.

The raw-size guard before the parse stays in the extension: it reads
CamelFileLength on a directory pipeline and at most max + 1 bytes
otherwise. The parser actions' own maxDocumentSize
(apache/camel-kamelets#3087) is pinned to 0.

modality, contentType, includeId, excludeId and the parsers'
maxDocumentSize are always passed ("" when unset), so app-wide
camel.kamelet.<id>.* properties cannot change them.

Needs the camel-kamelets 4.23.0 release including
apache/camel-kamelets#3085 (modality and contentType on
langchain4j-ingest-sink).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
jiriOndrusek-agent added a commit to jiriOndrusek-agent/camel-quarkus that referenced this pull request Oct 7, 2026
Builds on the Kamelet composition of apache#9152.

filter.include-id, filter.exclude-id, filter.min-document-size and
filter.document-filter go to the sink Kamelet and are enforced by the
component (CAMEL-24753), for configured and @ingest pipelines alike.
With a parser, the id patterns are also checked before the parse, so
an excluded document is never parsed.

modality=media, with an optional content-type, embeds each document
whole through the sink's media path (CAMEL-25027); IngestPipeline gets
modality() and contentType(). A configured pipeline's modality and
content-type are checked at build time; the builder setters reject
bad values.

The raw-size guard before the parse stays in the extension: it reads
CamelFileLength on a directory pipeline and at most max + 1 bytes
otherwise. The parser actions' own maxDocumentSize
(apache/camel-kamelets#3087) is pinned to 0.

modality, contentType, includeId, excludeId and the parsers'
maxDocumentSize are always passed ("" when unset), so app-wide
camel.kamelet.<id>.* properties cannot change them.

Needs the camel-kamelets 4.23.0 release including
apache/camel-kamelets#3085 (modality and contentType on
langchain4j-ingest-sink).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@davsclaus
davsclaus merged commit 9692c94 into apache:main Oct 11, 2026
5 checks passed
jiriOndrusek-agent added a commit to jiriOndrusek-agent/camel-quarkus that referenced this pull request Oct 13, 2026
Builds on the Kamelet composition of apache#9152.

filter.include-id, filter.exclude-id, filter.min-document-size and
filter.document-filter go to the sink Kamelet and are enforced by the
component (CAMEL-24753), for configured and @ingest pipelines alike.
With a parser, the id patterns are also checked before the parse, so
an excluded document is never parsed.

modality=media, with an optional content-type, embeds each document
whole through the sink's media path (CAMEL-25027); IngestPipeline gets
modality() and contentType(). A configured pipeline's modality and
content-type are checked at build time; the builder setters reject
bad values.

The raw-size guard before the parse stays in the extension: it reads
CamelFileLength on a directory pipeline and at most max + 1 bytes
otherwise. The parser actions' own maxDocumentSize
(apache/camel-kamelets#3087) is pinned to 0.

modality, contentType, includeId, excludeId and the parsers'
maxDocumentSize are always passed ("" when unset), so app-wide
camel.kamelet.<id>.* properties cannot change them.

Needs the camel-kamelets 4.23.0 release including
apache/camel-kamelets#3085 (modality and contentType on
langchain4j-ingest-sink).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants