Skip to content

fix(ai): write analysis, letters and chat replies in the language of … - #142

Open
Dwarf1er wants to merge 1 commit into
Gsync:devfrom
Dwarf1er:fix/multilingual-output
Open

Dwarf1er wants to merge 1 commit into
Gsync:devfrom
Dwarf1er:fix/multilingual-output

Conversation

@Dwarf1er

Copy link
Copy Markdown

Summary

Every system prompt is written in English, so models answer in English even when the resume and job
posting are in another language (seen with French documents). This adds a language-agnostic rule to the
generating prompts so the output follows the language of the documents, and the chat reply follows the
language of the user's message. Nothing is specific to one language.

Changes

  • New src/lib/ai/prompts/language.ts with the rules and short reminders.
  • Resume review, job match, automation match and cover letter system prompts end with a language rule.
  • Each user prompt also gets a one-line reminder beside the documents. A rule in the system prompt
    alone was not enough for a small local model (qwen3.5:9b still answered in English); the reminder next
    to the document fixed it.
  • Chat agent prompt gets a language section: reply in the user's most recent message's language, and
    keep tool arguments in the language the user supplied.
  • The SCORES: line and the ## section headings stay in English, because jobMatch/parse.ts,
    resumeReview/parse.ts and coverLetter/matchGuidance.ts parse them.
  • Cover letter salutation and sign-off are translated into the posting's language.
  • New eval suite evals/multilingual (French resume and posting) with language-neutral assertions,
    plus an eval:multilingual script.

Related Issues

Testing

  • npm run lint and tsc --noEmit are clean.
  • Unit tests: no new failures. 193 tests fail identically on a clean dev in my environment
    (jsdom/localStorage setup), unrelated to this change.
  • npm run eval:multilingual against local qwen3.5:9b: review, match and both chat cases pass. A
    French cover letter generated with the app's exact prompt came back in French with a localized
    salutation and sign-off. I could not grade the cover-letter llm-rubric case because I had no DeepSeek
    key.
  • Evals so far cover French only. Before the reminder was added, review and match answered in English
    on the same model.
  • Also checked manually in the Docker build against Ollama.

Notes

This assumes the resume and posting are in the same language. Mixed-language input uses the language
most of the text is written in.

…the documents

Every prompt is written in English, so models answered in English even for
a French resume or posting. Add a language-agnostic rule to the chat,
resume review, job match, automation match and cover letter prompts, plus a
one-line reminder beside the documents in each user prompt, which small
local models (qwen3.5:9b) need. The SCORES line and "##" headings stay
English because the parsers key on them.

Add an evals/multilingual suite (French resume and posting) with
language-neutral assertions, and an eval:multilingual script.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant