Senior SDET  ·  LLM Evaluation and AI Quality  ·  Test Architecture  ·  Regulated Healthcare and Payments

Senior SDET, 10 years building test frameworks, runners and CI release gates for systems where reliability matters: regulated healthcare, airline payments, a payment service provider, fintech, financial market data. I work at the framework and infrastructure level rather than only writing tests, and I own what gets tested, at which level, and when a release ships.

Since 2023 that has included LLM evaluation, two years of it on a regulated healthcare product: the evaluation system for a RAG assistant answering researcher questions against FDA and EMA guidance and tenant submissions. Retrieval and generation scored separately, golden datasets versioned in Langfuse, evaluation runs gated in CI.

Because I have also shipped production code, I review developer written tests credibly and design harnesses that teams adopt.

→ Full background, experience and competencies on mariusargatu.com/about

• LLM evaluation: does the model answer faithfully, and can you prove it? retrieval and generation scored separately, IR metrics on one side (hit rate, MRR, precision/recall@k), groundedness and relevancy on the other · agentic multi turn, tool call and trajectory testing · golden datasets versioned, evaluation runs gated in CI
• Test architecture: the frameworks and runners under the tests. test strategy across teams and products, integration and system tests carrying most of the coverage (testing trophy, honeycomb) · model based testing from state machines (xState) · property based and metamorphic testing (Schemathesis, Hypothesis) · contract testing
• Regulated domains and payments: the release gate is a decision, not a green tick. multitenant healthcare SaaS, payment endpoints, checkout and settlement flows, and the regression planning around them

• QARoom: a multitenant social platform built to demonstrate testability as an architectural property, using chaos engineering, contract testing, mutation testing, and model based and property based testing, all mapped to the boundary they defend.
• PayFlow: a payment system built by an AI agent from a frozen spec, zero hand written tests, guarded by a four layer verification pyramid (structural, behavioral, agent judgment, mutation). 73.1% mutation kill rate.
• Atlas: a testable broadband support agent (LangGraph + MCP), the runnable reference system for the "Evals Are Checks, Not Tests" series.
• TemporalMutate: semantic mutation testing over Temporal workflows, where an agent hunts for blind spots in a payment workflow's test suite and every verdict comes from replaying recorded histories in milliseconds, with no live server.

• LLM and eval: DeepEval, RAGAS, Langfuse, golden datasets, RAG and agent evaluation, MCP
• Languages and APIs: Python, TypeScript, REST, GraphQL, OpenAPI
• Test tooling: Pytest, Vitest, Playwright, Hypothesis, Schemathesis, xState, Pydantic
• Test management: Jira, Xray, TestRail, Qase.io, Allure
• Platform: Docker, Kubernetes, Jenkins, GitHub Actions, GitLab CI, Datadog, OpenTelemetry, Grafana

“A test suite is a liability as much as an asset. Every test earns its place.”