Skip to content

feat(dedup): java-sdk 2.2.0: faster, steadier per-test coverage - #193

Draft
slayerjain wants to merge 14 commits into
feat/java-dedup-app-classesfrom
feat/dedup-sdk-220
Draft

slayerjain wants to merge 14 commits into
feat/java-dedup-app-classesfrom
feat/dedup-sdk-220

Conversation

@slayerjain

@slayerjain slayerjain commented Nov 19, 2026 •

Copy link
Copy Markdown
Member

What this branch does today

Stacked on #192 (it includes that PR's INDEX record commit).

Per-test speed

  • The app's class set is built once, not on every END. With 50k generated app classes, building it again cost 1.4 to 2.2 ms of each test.
  • With JaCoCo's agent in the app's JVM, a START and an END now make one pass each over the probe arrays of the app's classes only, read in place from JaCoCo's runtime. Before, each test took three passes over every class JaCoCo had seen, library classes too: the START's reset, then the END's dump and its reset.
    • The pass is split across min(4, cores / 2) threads, the caller's included, once there are 2048 classes or more for each. The other threads are daemons that stop after a minute idle.
    • A probe read as hit is cleared in the same step, so a hit made while the pass runs is in that pass or the next one. JaCoCo's dump then reset loses a hit made between the two.
    • Library classes are neither read nor cleared.
    • Measured on generated app-only classes (about 61 probes a class), JaCoCo in the JVM, on a quiet 24-CPU machine, 200 tests after 30: 50k app classes cost 1.6 ms per test at p50 (START plus END answered) and 3.3 ms at p95; 7k app classes cost 0.7 ms. With the three passes before, the same 50k app cost 20 to 44 ms per test at p50.
  • KEPLOY_JAVA_DEDUP_DIAGNOSTICS=1 logs one line per START and END: how long the command took, and each step (scan, or dump/parse/filter, then json and send).

Warmup (the static initializers of the app's own classes, run once on the first START)

  • It now finds the app's class loaders: those that defined the app's classes already loaded (from the -javaagent's Instrumentation), the one that defined the most first; then the live threads' context loaders, the oldest thread first (a launcher sets its main thread's to the app's loader), read from the root thread group, which pauses no thread; then the system loader. Before, it used only the control thread's context loader and the system loader, so on a Spring Boot fat jar, whose classes only its launcher's loader can load, it initialized none of them and the first tests were charged with their lines.
  • A class already loaded is initialized in the loader that loaded it; of a class loaded in two loaders (Spring Boot DevTools' restart loader beside the system loader, say), in the one that defined more of the app's classes. A class not loaded yet goes on to the next loader only when a loader cannot find it: one a loader found but could not link or initialize is left to lazy init, never loaded again in another loader, where its static initializer would run twice. Each class is initialized with the thread's context loader set to the loader it goes through, as on the app's own threads.
  • It runs on a thread of its own. The first START waits for it for at most 10 s, then goes on, and the warmup stops after the class it is initializing. The WARNING names that class (or the step before the classes: reading the class files, finding the loaders). Its static initializer goes on while the tests run, and each line it runs from then on counts in the test running at the time (between two tests, in none). The classes it did not reach initialize when the app first uses them, as with the warmup off, and the log says how many. Before, the START was answered only once every class was initialized, however long that took, and Keploy's k8s collector gives up on an answer after 30 s by default. If the thread cannot start (a pids limit), the START goes on with no warmup and says so.
  • Behaviour change for fat jars: a Spring Boot fat jar (and the exploded launcher layout of buildpack images) is now warmed up for the first time. At the first START every app class not loaded yet is loaded, instrumented by JaCoCo and initialized, as other apps' classes already were, including classes the app never uses. On an app with many such classes that is more metaspace than the app alone would take (a buildpack image sizes its metaspace limit from an estimate of the classes the app loads). KEPLOY_JAVA_DEDUP_WARMUP_DISABLED=true turns the warmup off.

Background hits

  • What a START clears ran between two tests (a health check, a scheduled job, the warmup's static initializers that ended before the first START's scan). It is never counted in a test's hits. A static initializer the warmup was still running when the first START stopped waiting for it goes on: its later lines count in the test running when they run.
  • Keploy can ask for it, so that it leaves those classes' probes out of what it compares: it sends BACKGROUND on the control socket. The SDK answers ACK when it reads the probes in place and Keploy's receiver is on the data socket, else IGNORED (as an older SDK answers a line it does not know). From then on, each START that clears hits made since the last test's END (ACKed) sends one record on the data socket, right after that START is answered (after the INDEX record when that receiver does not have it yet): {"kind":"background","v":1,"build":"<the INDEX's build id>","after":"<test that ended>","before":"<test that starts>","executedLinesByFile":{class:[probe...]}}, the probes under a test's record's key and in its shape (with its classMismatch, left out when empty), plus "lost":N when the records of the N windows before it could not be made or sent.
    • Not background, never sent: what ran before the first test (the app's start, the warmup), and what ran after an END that was not taken or a START whose END never came (a test's own hits).
    • The ask holds for one run of the app and one receiver: an app started again (native keploy test does so for each test set) and a receiver bound anew (the next docker-compose test set) get none until Keploy asks again, so Keploy asks after each INDEX record. A Keploy that never asks gets none, short of a new receiver bound within one file-system clock tick of the one that asked (receivers are told apart by the socket file's inode and modification time, and a socket file's modification time is its bind time).
    • A test's own work that goes on after its END (an async task it started) lands in the next window and is reported as background. The record names after and before so that Keploy can weigh that.
  • KEPLOY_JAVA_SCAN=off turns the in-place read off: each test takes JaCoCo's reset and dump, and BACKGROUND is answered IGNORED.

A test's requests are done before its END reads its hits

  • A server sends the response before it is done with the request: a filter's code after the chain (Fineract's request log and authentication filters) ran after Keploy had the response and had sent END. Under load the END's pass ran first, that code landed between two tests, and the next START cleared it as background, so one test's coverage changed from run to run.
  • The SDK now counts the requests of Tomcat (CoyoteAdapter.service and asyncDispatch), Jetty (HttpChannel.handle; on Jetty 12 and 12.1 the servlet channel's handle, matched by name) and Undertow (Connectors.executeRootHandler), Spring Boot's three servers. Each of those methods becomes one that counts the request in and out around the server's own code, which moves to a private keploy$request$<name> beside it. Only classes loaded after the agent starts change (the JVM refuses a method added to a loaded class), and only when their class loader sees the agent's own copy of the counter. The SDK's transformer is retransform-capable, so the JVM runs it after JaCoCo's, which is not, whatever the -javaagent order: a JaCoCo agent attached before or after keploy-sdk.jar, or in JAVA_TOOL_OPTIONS, instruments and ids the server's own class, so classMismatch never names it for that. When another agent retransforms or redefines a class the SDK changed, the SDK changes it again (the JVM would refuse the change otherwise), unless the new bytes already carry the change (a copy taken from the class as the SDK changed it), which the JVM then takes as they are; a server class loaded before the SDK started is left as it is. A jar that carries the SDK without Can-Retransform-Classes: true still counts the requests, in the order its transformer was added, and, when the JVM has a JaCoCo agent of its own, says at WARNING that a JaCoCo attached after it is then named in classMismatch. A server in a named module (its jars on the module path) is counted too: the JVM lets a module whose class an agent changed read the class path. The JaCoCo keploy-sdk.jar starts adds its transformer before this one, so it instruments, and ids, the server's class as the index read it.
  • END waits until every request that came in since the test's START is done, for at most KEPLOY_JAVA_END_WAIT_MS (200 ms by default; a value above 2000 waits 2000 ms, said once at WARNING: a longer wait would cost each test more than the overhead target). A request already done costs END one read. A request that came in before the START (a stream, the request of a test cut before), or once END has come (a readiness probe, a metrics scrape, or a call the test's own code makes after its response), is never waited for.
  • Past the bound: a WARNING names the test, its record ends with "windowCut":true (after classMismatch, left out when false, as the receiver side agreed), and what the next START clears is not sent as a BACKGROUND window, since the cut test's own late hits are in it. The record still holds what was read at the cut. A Keploy that reads the flag writes that test as not measured; an older Keploy judges the hits the record holds.
  • KEPLOY_JAVA_END_WAIT_MS=0 turns the wait off and leaves the servers' classes unchanged.

Failure behaviour (fail closed)

  • If JaCoCo's runtime cannot be read in place (JaCoCo over TCP, or an agent not laid out as JaCoCo's), the SDK uses JaCoCo's reset and dump, as before.
  • If a pass fails, the SDK uses JaCoCo's reset and dump for the rest of the run. A pass that failed before it cleared a probe leaves the test's hits to JaCoCo's dump. One that failed after it may have cleared probes (an Error in a part, such as out of memory) waits for every part to end and answers the END with IGNORED, so Keploy leaves that test out of the comparison instead of using a partial hit set.

Class ids, and classes JaCoCo saw changed

  • A test's record names, in "classMismatch":[class, ...] after its probes, each class whose hits come from bytes other than those the SDK indexed: a class loader that weaves the bytes it defines (load-time weaving) or an agent listed before JaCoCo's that does not ask to retransform changed it before JaCoCo saw it, or the JVM loaded a multi-release jar's versioned copy. JaCoCo's id for it is not the CRC64 of the indexed bytes, so its probe indices do not belong to the indexed class. The key is left out when there is none; Keploy's receivers decode the record with encoding/json and ignore a key they do not know. The CRC64 of a class's indexed bytes is worked out the first time a test reports the class.
  • Of two classes of one name (loaded twice, from other bytes) that a test ran, the hits sent are those of the copy the SDK indexed, else of the copy the coverage manifest names, else of the lowest id compared as an unsigned number. A BACKGROUND record chooses alike and names such classes in a classMismatch of its own (Keploy takes background out probe by probe), and so does JaCoCo's dump path (KEPLOY_JAVA_SCAN=off, JaCoCo over TCP), which before kept whichever copy the dump listed last.
  • Every class id the SDK writes (the coverage manifest's id) is 16 lowercase hex digits, zero-padded. Before 2.2.0 it was unpadded, so readers parse it as a hex number.

Fixes

  • Coverage could count lines that did not run, for an app that loads one class from two sources (two class loaders, a woven copy, a multi-release jar's versioned copy): a test's record carried the probes of whichever copy JaCoCo's dump listed last, and Keploy read them against the class the SDK indexed. 2.2.0 sends the indexed copy's probes when the test ran it, and names in classMismatch a class whose probes come from other bytes.

  • The same test could get different coverage from run to run on a loaded machine: code a request ran after its response was sent (a servlet filter's code after the chain) was missed by the test's END and dropped as background by the next START. 2.2.0 waits for the test's requests to finish on Tomcat, Jetty and Undertow.

  • An app that ships its own org.jacoco.agent lost its own JaCoCo, or the SDK lost its own, with feat(dedup): find the Java app's own classes and start JaCoCo on them #192's keploy-sdk.jar, which carried JaCoCo's AgentJar under its own name and jacocoagent.jar at the jar's root: with the app's jar ahead of keploy-sdk.jar on the class path (a Jib launch), the SDK started the app's JaCoCo version (JaCoCo 0.7.9 failed to start on JDK 21, and every test's coverage was empty); in a Spring Boot fat jar, whose loader asks the class path first, the app's AgentJar was the SDK's and read the SDK's jacocoagent.jar. The jar now carries jacocoagent.jar alone, unchanged, at io/keploy/dedup/jacocoagent.jar, and none of that artifact's classes, so each gets its own. No release carried that jar: the SDK starts JaCoCo itself since 2.2.0.

Java 8 API in builds on newer JDKs

  • keploy-sdk and keploy-junit5 run on Java 8, and are released from JDK 17 with source and target 1.8. Those do not stop a call to a method Java 8 lacks: it compiles on JDK 17 and fails only on Java 8, with a NoSuchMethodError. A call to a Java 8 method that JDK 9 overrode with a narrower return type (ByteBuffer.position(int), flip()) is the case CI's JDK 8 leg cannot catch: it compiles on every JDK, and on JDK 17 refers to the method JDK 9 added. Each project's pom now sets maven.compiler.release to 8 in a profile active on JDK 9 and later (the release profile keeps it), so javac checks main and test code against Java 8's API and such a call fails the build or resolves as on Java 8. JDK 8 builds are unchanged. Today's code has no such call: on JDK 17 both projects' main classes come out the same, instruction for instruction, with the profile and without it.
  • The release setting is read only by maven-compiler-plugin 3.6 and later. keploy-sdk named no version of that plugin, so it got the running Maven's default, which on Maven 3.8 and older is 3.1: that version ignores the setting without a warning, and a jar built there with tests skipped was not checked. The root pom now pins 3.15.0, the version CI's runners bind today, so CI and the release compile keploy-sdk as before; keploy-junit5's pom keeps its own pin, 3.13.0. A local Maven 3.9.9 build moves from 3.13.0 to 3.15.0, and keploy-sdk's classes from the two are byte for byte the same on JDK 8, 17 and 21.

Licenses in keploy-sdk.jar

  • The jar now holds META-INF/LICENSE (Apache 2.0, the repo's), JaCoCo's EPL-2.0 text (META-INF/LICENSE-EPL-2.0.txt), and META-INF/THIRD-PARTY-NOTICES.txt, which names every part of the jar that is not keploy-sdk's own code in the version the jar carries: JaCoCo (EPL-2.0, and where its source is: jacocoagent.jar, unchanged at io/keploy/dedup/jacocoagent.jar, and the renamed org.jacoco.core classes), the ASM it depends on (BSD 3-Clause, with its text), Gson and junixsocket (Apache 2.0, with Gson's copyright line and junixsocket's NOTICE). The about.html at the jar's root is org.jacoco.core's, and no longer depends on the order of the dependencies.
  • The published pom still names only the Apache License 2.0, keploy-sdk's own license: Maven's POM reference reads several <licenses> entries as a choice between them, not as all of them applying, and keploy-sdk's own code is not offered under the EPL-2.0 or the BSD 3-Clause License. META-INF/THIRD-PARTY-NOTICES.txt is the record of what the jar carries under which license.

Limits of this branch today

  • END's wait does not count what the app runs on threads of its own: an async task, an executor's work between two dispatches of an async request (Spring MVC writes the response in the container's dispatch, which is counted), or an Undertow completion listener that runs on the IO thread after an async write. Such code still lands after END, as before.
  • Netty, and so WebFlux, is not counted, nor is a server loaded before the agent started (agent attached to a running JVM) or one whose class loader does not see the agent's classes (a module system such as JBoss Modules): END reads at once there, as before, and the log says once why.
  • Control socket: today a second JVM can still take over a live control socket, and the socket name is not unique per pid namespace.
  • Reading the app's classes: when a JaCoCo agent is attached separately, keploy-sdk.jar does not read the app's classes at startup, so the first START waits for that read (its scan needs the class set). With keploy-sdk.jar alone, they are read before the app starts.
  • If the in-place read fails after Keploy asked for BACKGROUND, the records stop (the WARNING says the read failed): Keploy then holds part of that run's background, so it leaves fewer probes out of what it compares, never more.
  • Background on the TCP transport used in Kubernetes: not sent (BACKGROUND is a unix control socket command).
  • classMismatch in Kubernetes: the SDK's CoverageReporter and k8s-proxy's coverage union do not read it. When the coverage manifest names the indexed copy of a class and a test ran only another copy, they credit that copy's probes to the indexed class.

Tests

  • CI: each JDK job of the Java Agent workflow now stops after 10 minutes instead of GitHub's 6 hours. Its steps take about 2 minutes with an empty Maven cache, so a job that reaches 10 is stuck, not slow.
  • CI: each JDK job now builds as release.yml does, with the release profile (signing skipped), records the sha256 of keploy-sdk's jar and sources jar, and in its last step runs that build again without clean and checks both. release.yml deploys to Maven Central from such a pass, so a build change that changed the jars' bytes there would publish jars other than those of its clean build, whose jar it smoke-tested. On JDK 17, with jacocoagent.jar's copy set to overwrite the file on every build, or with a plugin under the release profile that touches a file in the classes on every build, the unit tests pass and this step fails.
  • mvn -B -pl keploy-sdk -am test: 211 rows (three skipped when run as root, and on JDK 8 the module-path row) pass on JDK 8, 17 and 21, with KEPLOY_PROVEN unset and =1. New rows cover: a hit reported once and cleared; background never reaching the next test; a class loaded after a pass; order and completeness across four parts; a writer racing the passes, each of its 16384 hits reported exactly once; a failed part ending the pass only once every part has; a part the pool cannot take scanned by the caller; an Error reading JaCoCo's classes clearing nothing; agents not laid out as JaCoCo's refused; and, through the control socket, the background left out, a class loaded twice, a pass that lost hits, and the fallback to JaCoCo's dump and reset. Warmup rows: a fat jar's classes warmed up through its launcher's loader; the -javaagent's Instrumentation finding the loader of the app's loaded classes with no thread holding it; a class the app loaded in a second loader initialized there; of two copies of a class, the copy of the loader that defined more initialized, whichever the JVM lists first; a class whose static initializer throws run once, not again in a second loader; the loaders' order; every live thread seen from the root group; a slow static initializer holding the first START only for the wait, the WARNING naming it, the warmup stopped after it, and a second START not waiting; a warmup thread that cannot start leaving the START and the control server working; the WARNING's steps; an empty class list asking the JVM nothing. BACKGROUND rows: a window's hits sent once to a receiver that asked, never into a test; a class loaded twice reported as a test's record would; no record for a receiver that did not ask, after a START with no END, after an END for another test (with or without a START before it), or after an END whose record could not be sent; a receiver bound anew getting none until it asks, and the INDEX record before its first window; no receiver answered IGNORED and a window with none going nowhere; records that could not be sent counted in the next (said once, kept across a re-ask, not carried to a new receiver); KEPLOY_JAVA_SCAN=off taking JaCoCo's dump and answering IGNORED, and any other value keeping the scan. Class id rows: ids as 16 hex digits; the manifest's id over the TCP transport, with classMismatch in its COV; a class JaCoCo saw other bytes of named, and none for a test that ran only indexed classes; of two copies, the indexed one over the manifest's and over a lower id, the manifest's over a lower id (also when its id has fewer than 16 hex digits), the lower id compared unsigned; JaCoCo's dump path choosing alike and leaving library classes out; a BACKGROUND record naming a copy JaCoCo saw other bytes of, and none for the indexed copy; the record's shape with and without the key. License rows: the notices name the versions the build resolves (JaCoCo's core and agent, and no AgentJar; ASM with asm-commons and asm-tree; Gson; junixsocket), so a dependency bumped without its notice fails CI's unit tests; they quote the attributions the licenses ask for (JaCoCo's, Gson's and ASM's copyright lines, ASM's three conditions, the start and the damages sentence of its disclaimer, and the first and last lines of junixsocket's NOTICE); and both license texts sit next to them. keploy-sdk's io/keploy/dedup/jacocoagent.jar is org.jacoco.agent's, byte for byte, and none sits at the root of keploy-sdk's classes; with an app's own jacocoagent.jar ahead of keploy-sdk, the SDK still reads its own; a loader without it is a named error.
  • END-wait rows: the counter (a request from before the START never waited for, one that comes in during the wait not waited for, the wait ending when the last request is done and not at its bound, an interrupt or the bound cutting it, eight threads counting exactly); the changed classes on stand-ins of every entry's class, verified and run (JaCoCo's instrumented copy at class versions 52, 55, 61 and 65, as far as the JVM can define them, counted and keeping the id of the class's own bytes; arguments of every type, static, synchronized, a long return, a throw, a pre-Java 6 class, the handler's frame from Java 6 on; abstract, native, bridge and synthetic methods and interfaces left alone, an interface without a WARNING; a loader that does not see the counter left unchanged; a class changed as it loaded changed the same way again on a retransform, and none else: not one loaded before the SDK, one of another loader, or one another instance changed; bytes that already carry the change taken as they are on a redefine, and counted as they are when a class loads with them); through the control socket and on TCP, an END that waits for its request's tail (released only once END waits) and, past the bound, the record cut with no BACKGROUND window after it; KEPLOY_JAVA_END_WAIT_MS read and capped. Real servers, each in a JVM of its own with the agent: Tomcat 9.0.98, Jetty 9.4.57 and Undertow 2.2.37 with the wait on (the filter's tail in the record, and the diagnostics line showing the wait), Tomcat with the wait off (the tail missing: the loss this fixes) and past the bound (windowCut); Tomcat's jars on the module path (JDK 17 and 21); and the index holding Tomcat's jar, where the embedded JaCoCo, and JaCoCo's own agent attached before keploy-sdk.jar, after it, or after it with keploy-sdk.jar in JAVA_TOOL_OPTIONS, measure CoyoteAdapter and the record does not name it in classMismatch (the two later orders named it before this change); a jar without Can-Retransform-Classes still counting, with its WARNING when JaCoCo's own agent is attached after it, and with none when the SDK starts its own JaCoCo; another agent retransforming CoyoteAdapter once Tomcat runs, which succeeds and keeps the requests counted, redefining it with a dump of it as the SDK changed it (as a hot-swap tool does), which succeeds and keeps them counted, and retransforming it after loading it ahead of the SDK, which succeeds and leaves it uncounted. Not run on a real server: Tomcat 8.5, 10.1 and 11 and Jetty 10 to 12.1, matched by class and method name (Jetty 12.0's and 12.1's servlet channels checked against their jars). These rows run the SDK from the test class path; the smoke below runs the shaded keploy-sdk.jar on Tomcat.
  • Java8ApiTest, in keploy-sdk's rows and keploy-junit5's: a call to ByteBuffer.position(int) must refer to Buffer's method, as on Java 8. On JDK 17 and 21 it fails with the profile off, and on JDK 17 with release 9. A second row fails when the pom names no compiler plugin version, or one before 3.6 (the pin dropped, or set to 3.1 or 3.5.1), and when a pom it reads is not the one it expects (a run outside Maven from another directory).
  • On JDK 8, 17 and 21, for both projects: the tree compiles to class file version 52; a planted java.util.List.of() fails to compile, also with -P release, and compiles on JDK 17 and 21 with the profile off. On JDK 17 the release builds of both (javadoc jars included, signing skipped) pass, and Java 8 runs scripts/smoke-javaagent.sh on the keploy-sdk jar built there. keploy-junit5's rows pass on JDK 8 with JUnit 5.10.5, JDK 17 with 5.14.4 and JDK 21 with 6.1.3.
  • With Maven 3.8.8 on JDK 17 and 21: before the pin, keploy-sdk compiles with compiler 3.1, a planted java.util.List.of() builds with tests skipped, and a planted ByteBuffer.position(0) refers to the method Java 8 lacks; with it, the first fails to compile and the second refers to Buffer's.
  • The racing-writer row passed 200 of 200 runs in four concurrent loops, and the warmup rows 100 of 100.
  • scripts/smoke-javaagent.sh checks that the real JaCoCo runtime is read in place, both the agent the command attaches and the one keploy-sdk.jar starts itself. On real JaCoCo it also fails a record that names any class in classMismatch when the app ran only its own classes, and runs a test in which only another copy of a class runs (built from other source, in a class loader of its own), whose record must name that class alone: on the in-place read with both JaCoCo agents, and on JaCoCo's dump (KEPLOY_JAVA_SCAN=off). It runs the built keploy-sdk.jar on a real Tomcat 9.0.98 with JaCoCo's own agent attached after it and the index holding Tomcat's classes: the record must measure CoyoteAdapter, and the servlet's code that runs after the response while END waits, and name no class in classMismatch, which fails for a jar built without Can-Retransform-Classes: true.
  • scripts/smoke-javaagent.sh also checks that the built jar holds the repo's LICENSE, the EPL-2.0 text and the notices as the sources hold them, and org.jacoco.core's about.html; that its io/keploy/dedup/jacocoagent.jar is org.jacoco.agent's, byte for byte, and that org.jacoco.agent's about.html, which the jar leaves out with the rest of that artifact, is the one inside it; and that each part of the jar maps onto a part the notices name, so a new package, or a dependency from another project, shaded in without its notice fails. Another artifact of ASM or junixsocket maps onto its project's part, whose license the notices already carry.
  • scripts/smoke-javaagent.sh runs an app with its own ASM, Gson, JaCoCo core and JaCoCo agent (org.jacoco.agent 0.7.9), other versions than the jar's: with them ahead of keploy-sdk.jar on the class path, the INDEX record must name the JaCoCo 0.8.15 the SDK started; in a loader of the app's own that asks the class path first, as Spring Boot's does, each must load from the app's jars and the app's AgentJar must read the app's own jacocoagent.jar. Both fail on the jar before this change.

Part of keploy/enterprise#3435 (code coverage + dynamic dedup for large Java apps). Draft: this branch is updated at milestones while the work lands; it is marked ready for review when its checks pass. Do not merge before then.

Every END built a new set of the app's class names from the index, to
keep only the app's classes from JaCoCo's dump. The index never changes
once it is loaded, so the set is now built once. In a profile with 50k
generated app classes, building it again took 1.4 to 2.2 ms of each
test.

KEPLOY_JAVA_DEDUP_DIAGNOSTICS now logs one line per START and END: how
long the command took, and how long each step did. A START has warmup
and reset. An END has the JaCoCo dump, the parse, the filter to the
app's classes, the JSON and the send, plus the check for new classes
(and the class-zip frame) on TCP. A step only takes the time when it
ends; the line is formatted once the command is done, so no step is
charged with formatting. A run on the real app then says where its
per-test time goes. With diagnostics off nothing is logged and no step
is timed.

The warmup's log line now says how long the warmup took. The README
lists both the diagnostics and the warmup switches.

Signed-off-by: slayerjain <shubhamkjain@outlook.com>
With JaCoCo's agent in the app's JVM, each test took three passes over
the probe arrays of every class JaCoCo had seen, library classes too:
the START's reset, then the END's dump and its reset. On large apps
those passes are most of the SDK's time per test.

A START and an END now make one pass each, over the probe arrays of the
app's classes only, read in place from JaCoCo's runtime:
- The arrays are kept sorted by class name and added to only when
  JaCoCo's store has grown (a class was loaded); a pass with no class
  loaded since does not walk the store.
- The pass is split across min(4, cores / 2) threads, the caller's
  included, once there are 2048 classes or more for each; the others are
  daemon threads that stop after a minute idle. A part no thread can be
  made for is scanned on the caller's thread.
- A probe read as hit is noted and cleared in the same step, so a hit
  made while the pass runs is in that pass or the next one. JaCoCo's
  dump then reset loses a hit made between the two.
- What a START clears ran between two tests (a health check, a
  scheduled job, the warmup's static initializers): it is background,
  never counted in a test's hits. With diagnostics on, the START logs
  how many classes and probes it was. It is not sent to Keploy: no frame
  of the current protocol carries it.
- Library classes are neither read nor cleared.

JaCoCo's runtime is reached by reflection, through the types of its
fields, as the agent shades them under a package of its own. When JaCoCo
is reached over TCP, or its agent is not laid out that way, the SDK uses
JaCoCo's reset and dump, as before. When a pass fails, the SDK uses them
for the rest of the run. A pass that failed before it cleared a probe
leaves the test's hits to JaCoCo's dump. One that failed after (an
Error in a part, such as out of memory) has lost hits, so its END is not
taken; it waits for every part to end first. Of two classes of one name
(loaded twice, from other bytes), the hits sent are those of the one the
coverage manifest names when that one ran, else those of the lowest id,
as JaCoCo's dump sends a name's hits whichever copy made them.

The diagnostics line names the new step `scan`. The smoke test checks
that the real JaCoCo agent is read in place, both one the command
attaches and the one keploy-sdk.jar starts itself.

Rows: a hit is reported once and cleared; background never reaches the
next test; a class loaded after a pass is scanned by the next; order and
completeness across four parts; a writer racing the passes has each of
its 16384 hits reported exactly once, over at least two passes for each
512 writes; a failed part ends the pass only once every part has; a part
the pool cannot take is scanned by the caller; an Error reading JaCoCo's
classes clears nothing; agents not laid out as JaCoCo's are refused;
and, through the control socket, the background is left out, a class
loaded twice is reported once, as the copy the manifest names when it
ran and as the other copy when only that one ran, a pass that lost hits
leaves its END not taken, an Error reading JaCoCo's classes or an
agent's fields keeps the server and falls back, and a failed pass falls
back to JaCoCo's dump and reset for good.

Signed-off-by: slayerjain <shubhamkjain@outlook.com>
The first START runs the static initializers of the app's own classes,
so that no test is charged with their lines. It looked the classes up
through the control thread's context loader, the system loader and the
SDK's own loader. A Spring Boot fat jar's classes are reachable only
through the loader its launcher makes, so the warmup initialized none
of them ("initialized=0") and the first tests were charged with them.
It also ran on the START's own thread, so the START was answered only
once every class was initialized, however long that took; Keploy's k8s
collector gives up on an answer after 30 s by default.

The warmup now:
- finds the app's loaders: those that defined the app's classes already
  loaded (from premain's Instrumentation), the one that defined the most
  first; then the live threads' context loaders, the oldest thread first
  (a launcher sets its main thread's to the app's loader), read from the
  root thread group, which pauses no thread; then the system loader and
  the SDK's own. A class already loaded is initialized in the loader
  that loaded it; of a class loaded in two loaders, in the one that
  defined more of the app's classes. A class not loaded yet goes on to
  the next loader only when a loader cannot find it: one that a loader
  found but could not link or initialize is left to lazy init, never
  loaded again in another loader, where its static initializer would
  run twice. Each class is initialized with the thread's context loader
  set to the loader it goes through, as on the app's own threads.
- runs on a thread of its own, which also reads the app's classes when
  premain has not. The first START waits for it for at most 10 s, then
  goes on, and the warmup stops after the class it is initializing. The
  WARNING then names that class (or the step before the classes). Its
  static initializer goes on while the tests run, and each line it runs
  from then on counts in the test running at the time (between two
  tests, in none). The classes it did not reach initialize when
  the app first uses them, as with the warmup off, and the log says how
  many ("not reached=N"). If the thread cannot start (a pids limit), the
  START goes on with no warmup and says so.

Fat jars are now warmed up for the first time: every app class not
loaded yet is loaded, instrumented by JaCoCo and initialized at the
first START, as other apps' classes already were. On an app with many
classes it never uses, that is more metaspace than the app alone would
take. KEPLOY_JAVA_DEDUP_WARMUP_DISABLED=true turns the warmup off.

It still runs only when the index is the app's own classes, and never
initializes a library class itself. When premain has not read the app's
classes (a JaCoCo agent attached separately), the first START's scan
still waits for that read, as it needs the class set.

Rows: a fat jar's classes warmed up through its launcher's loader;
premain's Instrumentation finding the loader of the app's loaded
classes with no thread holding it; a class the app loaded in a second
loader initialized there; of two copies of a class, the copy of the
loader that defined more initialized, whichever the JVM lists first; a
class whose static initializer throws run once, not again in a second
loader that sees it; the loaders' order; every live thread seen from
the root group; a slow static initializer holding the first START only
for the wait, the WARNING naming it, the warmup stopped after it and a
second START not waiting; a warmup thread that cannot start leaving the
START and the control server working; the WARNING naming each step; an
empty class list asking the JVM nothing.

Signed-off-by: slayerjain <shubhamkjain@outlook.com>
What a START clears ran between two tests: a health check, a scheduled
job, a cache refresh. It never counts in a test's hits, but the same
threads also run while a test does, so their classes' probes land in
some tests and not others from one run to the next. Keploy can leave
those probes out of what it compares only if it knows them.

Keploy now asks with the line BACKGROUND on the control socket. The SDK
answers ACK when it reads JaCoCo's probes in place and a receiver is on
the data socket, else IGNORED, as an older SDK answers a line it does
not know. From then on, each START that clears hits made since the last
test's END (an END answered ACK) sends one record on the data socket,
right after that START is answered, after the INDEX record when that
receiver does not have it yet:

  {"kind":"background","v":1,"build":"<the INDEX's build id>",
   "after":"<test that ended>","before":"<test that starts>",
   "executedLinesByFile":{class:[probe...]}}

plus "lost":N when the records of the N windows before it could not be
made or sent (each is tried once; one WARNING while the sends fail).
The probes are under a test's record's key and in its shape, so Keploy
reads both alike. Of two copies of one class, the copy the coverage
manifest names, else the lowest id.

- Not background, never sent: what ran before the first test (the app's
  start, the warmup), and what ran after an END that was not taken, or
  after a START whose END never came (a test's own hits).
- A test's own work that goes on after its END (an async task it
  started) lands in the next window and is reported as background; the
  record names both tests so that Keploy can weigh that.
- The ask holds for one run of the app and one receiver: an app started
  again (native keploy test does so for each test set) and a receiver
  bound anew on the data socket's path (the next docker-compose test
  set) get none until Keploy asks again, so Keploy asks after each INDEX
  record. A Keploy that never asks gets none, short of a new receiver
  bound within one file-system clock tick of the one that asked
  (receivers are told apart by the socket file's inode and modification
  time, and a socket file's modification time is its bind time).
- If the in-place read fails after the ask, the records stop (the read's
  WARNING says so): Keploy then leaves fewer probes out, never more.
- No BACKGROUND on the TCP transport used in Kubernetes.

KEPLOY_JAVA_SCAN=off (any case) turns the in-place read off: each test
takes JaCoCo's reset and dump, as when JaCoCo is reached over TCP, and
BACKGROUND is answered IGNORED.

Rows (InProcessScanTest): a window's hits reach a receiver that asked
once, and never a test; a class loaded twice is reported as a test's
record would; no record for a receiver that did not ask; none after a
START with no END, an END for another test (with or without a START
before it), or an END whose record could not be sent (skipped as root);
a receiver bound anew gets none until it asks, and the INDEX record
before its first window; with no receiver BACKGROUND is IGNORED and a
window goes nowhere; records that could not be sent are counted in the
next, said once, kept across a re-ask by the same receiver, and not
carried to a new receiver (skipped as root); with the scan off each
test takes JaCoCo's dump and BACKGROUND is IGNORED; any other value
keeps the scan.

Signed-off-by: slayerjain <shubhamkjain@outlook.com>
Keploy reads a test's probes against the classes the SDK indexed: the
class files it read from the app's dirs and jars. When a class loader
that weaves the bytes it defines (load-time weaving), or an agent
listed before JaCoCo's that does not ask to retransform, changed a
class before JaCoCo saw it, or the JVM loaded a multi-release jar's
versioned copy, JaCoCo's probes are those of other bytes, and their
indices do not belong to the indexed class. Only the SDK holds both
ids: JaCoCo's (ExecutionData's id) and the CRC64 of the indexed bytes.

A test's record now names each such class after its probes:

  {"id":"<test>","executedLinesByFile":{class:[probe...]},
   "classMismatch":[class...]}

The key is left out when there is none, so a test that ran one copy of
each class, the one the SDK indexed, sends the same record as before,
and receivers that do not know the key ignore it (Keploy's Go receivers
decode it with encoding/json). The CRC64 of a class's indexed bytes is
worked out, and kept, the first time a record reports the class.

A BACKGROUND record names such classes in a classMismatch of its own,
left out when empty: Keploy takes background out of a test's hits
probe by probe, so it must know which probes are not the indexed
class's.

Of two copies of one class (loaded twice, from other bytes) that a test
ran, the record now reports the copy the SDK indexed, else the copy the
coverage manifest names, else the lowest id compared as an unsigned
number (before: the manifest's, else the lowest id compared as a signed
long). A BACKGROUND record chooses alike. JaCoCo's dump path
(KEPLOY_JAVA_SCAN=off, or JaCoCo reached over TCP) now goes through the
same choice: before, of two copies it kept whichever the dump listed
last, and added the first to the manifest.

Every class id the SDK writes (the coverage manifest's id) is now 16
lowercase hex digits, zero-padded, as the INDEX record's build id
already hashes them. Older SDKs wrote Long.toHexString's unpadded form,
so readers parse the id as a hex number.

Rows: ids as 16 hex digits; the manifest's id over the TCP transport,
with classMismatch in its COV; a class JaCoCo saw other bytes of named,
and none for a test that ran only indexed classes; of two copies, the
indexed one over the manifest's and over a lower id, the manifest's
over a lower id (also when its id has fewer than 16 hex digits), the
lower id compared unsigned; JaCoCo's dump path choosing alike and
leaving library classes out; a BACKGROUND record naming a copy JaCoCo
saw other bytes of, and none for the indexed copy; the record's shape
with and without the key. The Java agent smoke test now fails a record
that names any class when the app ran only its own, and runs a test in
which only another copy of a class runs (other source, its own class
loader), whose record must name that class alone: on the in-place read
with the JaCoCo agent the command attaches and the one keploy-sdk.jar
starts, and on JaCoCo's dump (KEPLOY_JAVA_SCAN=off).

Signed-off-by: slayerjain <shubhamkjain@outlook.com>
keploy-sdk.jar carries JaCoCo (EPL-2.0), the ASM it depends on (BSD
3-Clause), Gson and junixsocket (Apache 2.0). Its
META-INF/THIRD-PARTY-NOTICES.txt named them, but the jar held neither
license text the notices relied on: not the Apache License 2.0, which
keploy-sdk, Gson and junixsocket are under and whose section 4(a) asks
for a copy with each redistribution, nor the EPL-2.0, named only by its
URL. junixsocket's NOTICE attribution was missing too.

- META-INF/LICENSE is the repo's LICENSE (Apache 2.0), copied in at
  build time, so the jar's and the repo's cannot differ.
- META-INF/LICENSE-EPL-2.0.txt is the EPL-2.0 as eclipse.org publishes
  it.
- THIRD-PARTY-NOTICES.txt points to both. It now says that
  keploy-sdk's own code is under the Apache 2.0 and the third-party
  software stays under its own license, where keploy-sdk's source is,
  and the build that renames the packages; names org.jacoco.agent's
  AgentJar class, which the jar carries outside jacocoagent.jar and
  under its own name; adds org.jacoco.core's sources on Maven Central
  to JaCoCo's source pointer; gives Gson's and junixsocket's source
  tags, Gson's copyright line and junixsocket's NOTICE text; and says
  which about.html covers which JaCoCo part.
- org.jacoco.core and org.jacoco.agent both hold an about.html at their
  root. The shade plugin kept the first it met, core's, and dropped the
  other with a warning. It now leaves out org.jacoco.agent's, the same
  file as the one inside jacocoagent.jar, which the jar carries whole,
  so which one lands no longer rests on the dependencies' order.

Rows: ThirdPartyNoticesTest checks that the notices name the versions
the build resolves (JaCoCo's core and agent, with AgentJar; ASM with
asm-commons and asm-tree; Gson; junixsocket), so a dependency bumped
without its notice fails CI's unit tests; that they quote the
attributions the licenses ask for (the copyright lines, ASM's three
conditions, the start and the damages sentence of its disclaimer, and
the first and last lines of junixsocket's NOTICE); and that both
license texts sit next to the notices. scripts/smoke-javaagent.sh
checks that the built jar holds the repo's LICENSE, the EPL-2.0 text
and the notices as the sources hold them, and org.jacoco.core's
about.html; that org.jacoco.agent's about.html is the one inside
jacocoagent.jar; and that each part of the jar maps onto a part the
notices name, so a new package, or a dependency from another project,
shaded in without its notice fails. Another artifact of ASM or
junixsocket maps onto its project's part, whose license the notices
already carry.

The README gets a License section.

Signed-off-by: slayerjain <shubhamkjain@outlook.com>
A server sends the response before it is done with the request: a
filter's code after the chain (a request log, an authentication
filter's cleanup) runs after Keploy has the response and has sent END.
Under load END's pass ran first, that code landed between two tests,
and the next START cleared it as background, so the same test got
different coverage from run to run.

The SDK now counts the requests of Tomcat (CoyoteAdapter.service and
asyncDispatch), Jetty (HttpChannel.handle, and on Jetty 12 and 12.1
the servlet channel's handle, matched by name) and Undertow
(Connectors.executeRootHandler). As each class loads, that method
becomes one that counts the request in and out of RequestGate around
the server's own code, which moves to a private keploy$request$<name>.
A class whose loader does not see the agent's own RequestGate is left
unchanged, and so is any class loaded before the agent started. The
JaCoCo keploy-sdk.jar starts adds its transformer first, so it ids
the server's class as the index read it, and a server in a named
module is counted too (the JVM lets a module an agent changed read the
class path).

END waits until every request that came in since the test's START is
done, for at most KEPLOY_JAVA_END_WAIT_MS (200 ms by default, 10 s at
most; 0 turns the wait and the class change off). A request already
done costs END one read; one from before the START, or one that comes
in once END has come (a probe, a scrape), is never waited for. Past the
bound, a WARNING names the test, its record ends with
"windowCut":true (after classMismatch, left out when false), and the
next START opens no BACKGROUND window, since the cut test's late hits
are in it.

Limits: code on the app's own threads (an async task) is not counted,
nor are Netty and WebFlux; the log says once when END does not wait,
and why. A JaCoCo agent attached after keploy-sdk.jar (the agents in
JAVA_TOOL_OPTIONS start before the command line's) sees the changed
server class, so the README's examples list JaCoCo first.

Rows: RequestGate's count and wait; the server entries on stand-ins of
each server's class (arguments of every type, static, synchronized, a
pre-Java 6 class, abstract/native/bridge/synthetic methods, interfaces,
loaders that do not see the counter); END's wait, cut and no-window on
the unix socket and on TCP; and real Tomcat 9, Jetty 9.4 and Undertow
2.2 in JVMs of their own with the agent, where a filter's code that runs
after the response is in the test's record, missing with the wait off,
and cut past the bound; Tomcat on the module path; and the embedded
JaCoCo measuring Tomcat's own class with no classMismatch for it.

Signed-off-by: slayerjain <shubhamkjain@outlook.com>
keploy-sdk.jar changes the method each server runs a request in
(Tomcat's CoyoteAdapter, Jetty's HttpChannel or servlet channel,
Undertow's Connectors) with a transformer that could not retransform,
so the JVM ran it in the order the agents were attached. A JaCoCo agent
attached after keploy-sdk.jar, or keploy-sdk.jar in JAVA_TOOL_OPTIONS
with JaCoCo on the command line, then instrumented and ided the changed
class, and every test's record named it in classMismatch whenever the
classes Keploy measures include the server's (an uber jar, or every
class on the class path). The requests were counted in every order.

The transformer can now retransform (Can-Retransform-Classes: true in
the jar's manifest). The JVM runs such transformers after every one
that cannot, JaCoCo's among them, so JaCoCo instruments and ids the
server's own class in any order. When another agent retransforms a
class the SDK changed as it loaded, the JVM passes the class as it was
before the SDK's change; on a redefine it passes the redefiner's bytes.
The SDK changes either again, since the JVM refuses a class that drops
a method, unless the bytes already carry the change (a copy taken from
the class as the SDK changed it), which the JVM then takes as they are.
A redefine with the server's own bytes, which the JVM refused before,
now succeeds when no other agent changed the class as it loaded
(JaCoCo does not redo its own change on a redefine). A class loaded
before the SDK started is left as it is, since the JVM refuses a method
added to a loaded class too. A jar that carries the SDK without the
manifest entry still counts the requests, as before, and when the JVM
has a JaCoCo agent of its own it says at WARNING that one attached
after it is then named in classMismatch.

KEPLOY_JAVA_END_WAIT_MS now caps at 2000 ms instead of 10000: a longer
wait per test costs more than the SDK's overhead target.

The JVM now calls the SDK's transformer in a second pass over each
class it loads: one call into the agent with a copy of the class's
bytes, then one map lookup. With KEPLOY_JAVA_END_WAIT_MS=0 no class
gets that pass, but the JVM still holds the retransform capability the
manifest asks for, as with any agent that can retransform.

Rows, on a real Tomcat in a JVM of its own: JaCoCo's own agent attached
before keploy-sdk.jar, after it, and after it with keploy-sdk.jar in
JAVA_TOOL_OPTIONS, each measuring CoyoteAdapter with no classMismatch
for it (the two later orders named it before this change); a jar
without Can-Retransform-Classes counting, with its WARNING when JaCoCo
is attached after it and none with the SDK's own JaCoCo; another
agent's retransform of CoyoteAdapter succeeding and the requests still
counted; a redefine of CoyoteAdapter with a dump of it as the SDK
changed it succeeding, the requests still counted; and a retransform of
a CoyoteAdapter loaded ahead of the SDK succeeding, the server left
uncounted. Unit rows: the change made again on a retransform for the
class changed as it loaded, and for no other; a retransform it cannot
make again refused with a WARNING; bytes that already carry the change
taken as they are on a redefine (changed twice, the JVM refuses the
class) and counted as they are when a class loads with them; the
2000 ms cap.

Signed-off-by: slayerjain <shubhamkjain@outlook.com>
The rows that attach JaCoCo's own agent after keploy-sdk.jar write an
agent jar of their own, so none read the manifest the release ships:
a keploy-sdk.jar built without Can-Retransform-Classes: true passed
every row, while a JaCoCo agent attached after it would name the
server's class in classMismatch.

scripts/smoke-javaagent.sh now runs the built jar on a real embedded
Tomcat 9.0.98, with JaCoCo's own agent attached after it and the index
holding Tomcat's classes, as an uber jar's does. Its servlet runs more
code after the response once the test's END waits for the request. The
case fails unless the INDEX record says the attached JaCoCo measures and
names Tomcat's jar among its roots, the test's record measures
CoyoteAdapter and that later code and names no class in classMismatch,
the log says the SDK counts Tomcat's requests, the END's diagnostics
show the wait for them, and no WARNING names Can-Retransform-Classes.
A failed check ends the harness's JVM with 1 (Tomcat's threads are not
daemons), its sockets and HTTP reads time out, and the case is bounded
at 180 s, so a regression fails the job instead of hanging it.
With the manifest entry set to false, the case fails on JDK 21 and on
JDK 8.

ServerEntriesTest wraps JaCoCo's instrumented copy of a stand-in of
CoyoteAdapter at class versions 52, 55, 61 and 65, as far as the JVM
can define them: JaCoCo keeps its probes in a field at 52 and reads
them through a dynamic constant from 55 on. The wrapped copy loads
through the verifier, counts its request, and reports its probes under
the CRC64 of the class's original bytes, the id the index holds, not
the id of the wrapped bytes.

Signed-off-by: slayerjain <shubhamkjain@outlook.com>
The verify job had no timeout, so a hung build, unit test or smoke case
held its runner for GitHub's default of 6 hours, up to three times over
across the JDK 8, 17 and 21 legs.

Timed on 4 CPUs (the size of a GitHub-hosted runner) with an empty Maven
repository per JDK, the cache-miss worst case, the job's three steps as
written take 116 to 121 seconds per JDK: build 25-29 s, unit tests
60-63 s, smoke 28-30 s. The smoke already stops its Tomcat case after
180 s. A job that reaches 10 minutes, about five times the slowest
measured run, is stuck rather than slow, and now fails there.

Signed-off-by: slayerjain <shubhamkjain@outlook.com>
keploy-sdk.jar carried org.jacoco.agent's AgentJar class under its own
name and jacocoagent.jar at the jar's root, and EmbeddedJacoco read the
agent jar through AgentJar. An app that ships its own org.jacoco.agent
met the SDK's copy in either order:

- the app's jar ahead of keploy-sdk.jar on the class path (a Jib
  launch): EmbeddedJacoco's AgentJar was the app's, and the SDK started
  the app's JaCoCo. With org.jacoco.agent 0.7.9 on JDK 21 that agent
  failed to start and every test's coverage was empty.
- the app's jar in a loader that asks the system class loader first
  (Spring Boot's BOOT-INF/lib): the app's AgentJar was keploy-sdk.jar's,
  and read keploy-sdk.jar's jacocoagent.jar, not the app's.

org.jacoco.agent is now a test dependency, not shaded in. The build
unpacks only its jacocoagent.jar, byte for byte, to
io/keploy/dedup/jacocoagent.jar, which EmbeddedJacoco reads from the
loader that loaded keploy-sdk and names in an IllegalStateException
when it is missing. keploy-sdk.jar holds none of that artifact's
classes, and its about.html no longer needs a shade filter. The build
unpacks it under target/jacocoagent and copies it into target/classes
only when it is missing there or older, so a cleared target/classes
gets it back and a build without clean leaves keploy-sdk.jar as it was.
THIRD-PARTY-NOTICES.txt names the jar's new place and drops AgentJar.

Rows: the carried jar is org.jacoco.agent's, byte for byte, and none
sits at the root of keploy-sdk's classes; with an app's own
jacocoagent.jar ahead of keploy-sdk, EmbeddedJacoco still reads the
SDK's; a loader without it is the named error. The notices name the
jar's place and no AgentJar. scripts/smoke-javaagent.sh checks the
carried jar against org.jacoco.agent's, maps it as a part of its own,
and adds org.jacoco.agent 0.7.9 to the app's own libraries: ahead of
keploy-sdk.jar, the INDEX record must name JaCoCo 0.8.15; in the app's
own parent-first loader, AgentJar must load from the app's jar and read
the app's jacocoagent.jar. Both smoke cases fail on the jar before this
change.

Signed-off-by: slayerjain <shubhamkjain@outlook.com>
keploy-sdk and keploy-junit5 run on Java 8, and both are released from
JDK 17 with source and target 1.8. Those set the language level and the
class file version, not the API the code is checked against: javac on
JDK 17 compiles against JDK 17's own classes. A call to a method Java 8
lacks (java.util.List.of()) then fails on Java 8 only, in the released
jar, with a NoSuchMethodError. CI's JDK 8 legs build on JDK 8 and catch
that one, but not a Java 8 method that JDK 9 overrode with a narrower
return type (ByteBuffer.position(int), flip()): that call compiles on
every JDK, and on JDK 17 it refers to ByteBuffer's own position(int),
which returns ByteBuffer and which Java 8 does not have. No CI leg runs
a jar built on JDK 17 on Java 8.

Each pom (the root's, for keploy-sdk, and keploy-junit5's, a project of
its own) gets a profile, active on JDK 9 and later, that sets
maven.compiler.release to 8: javac then checks main and test code
against Java 8's API, and a call resolves as it would on JDK 8. The
release profile does not turn it off (only activeByDefault profiles
yield to -P). JDK 8's javac has no --release, and needs none: there the
build is unchanged. Today's code has no such call: on JDK 17 both
projects' main classes come out the same, instruction for instruction,
with the profile and without it.

Java8ApiTest, in keploy-sdk's rows and keploy-junit5's, checks that a
call to ByteBuffer.position(int) refers to Buffer's method, as on
Java 8. On JDK 17 and 21 it fails with the profile off, and on JDK 17
with release 9.

Checked on JDK 8, 17 and 21, for both projects: the tree compiles to
class file version 52; a planted java.util.List.of() fails to compile,
with and without -P release; a planted ByteBuffer.position(0) compiles
to ByteBuffer.position:(I)Ljava/nio/Buffer;, Buffer's method. On JDK 17
and 21 with the profile off, both compile, and the second to
ByteBuffer.position:(I)Ljava/nio/ByteBuffer;, which Java 8 lacks. On
JDK 17 the release builds of both projects (javadoc jars included,
signing skipped) pass, and Java 8 runs scripts/smoke-javaagent.sh on
the keploy-sdk jar built there. keploy-junit5's rows pass on JDK 8 with
JUnit 5.10.5, JDK 17 with 5.14.4 and JDK 21 with JUnit 6.1.3.

Signed-off-by: slayerjain <shubhamkjain@outlook.com>
The java8-api profile sets maven.compiler.release to 8, which only
maven-compiler-plugin 3.6 and later read. Neither the root pom nor
keploy-sdk's named a version of that plugin, so keploy-sdk was compiled
with the one the running Maven binds by default: 3.15.0 on the Maven of
CI's runners today, 3.13.0 on Maven 3.9.9, and 3.1 on Maven 3.8 and
older. 3.1 has no release parameter and ignores the property without a
warning. A keploy-sdk jar built there on JDK 9 or later with -DskipTests
could again carry a call Java 8 lacks, and nothing failed. With the
tests on, Java8ApiTest fails, so CI would catch it; a package build
would not.

The root pom now pins 3.15.0 in its pluginManagement, which keploy-sdk
inherits. That is the version CI's runners, which also build the
release, bind today, so CI's and the release's compile do not change.
keploy-junit5's pom keeps its own pin, 3.13.0. Other builds of
keploy-sdk move to 3.15.0: from 3.1 on Maven 3.8, which is the fix
checked below, and from 3.13.0 on Maven 3.9.9, where its main and test
classes come out byte for byte the same from both on JDK 8, 17 and 21.

A second row in each Java8ApiTest reads the pom (keploy-sdk's, then the
root's; keploy-junit5's own) and fails when it names no compiler plugin
version, or one before 3.6 such as 3.1 or 3.5.1. It also fails when a
pom it reads is not the one it expects: run outside Maven from another
directory, it would read another project's pom and could pass on that
one's pin. On Maven 3.9 nothing else fails without the pin: the default
binding hides it.

Checked with Maven 3.8.8 on JDK 17 and 21. Without the pin, keploy-sdk
compiles with compiler 3.1: a java.util.List.of() planted in its main
code builds with -DskipTests, a planted ByteBuffer.position(0) refers to
ByteBuffer's own position(int), which Java 8 lacks, and both
Java8ApiTest rows fail. With the pin, compiler 3.15.0: the first fails
to compile, the second refers to Buffer's method, and the rows pass.
keploy-junit5 builds with 3.13.0 on Maven 3.8.8 as before.
java-agent.yml's matrix (JDK 8, 17 and 21, KEPLOY_PROVEN unset and =1,
the smoke), junit5-sdk.yml's three legs and release.yml's verify on
JDK 17 pass.

Signed-off-by: slayerjain <shubhamkjain@outlook.com>
@slayerjain slayerjain changed the title feat(dedup): java-sdk 2.2.0: faster per-test coverage, one JVM serves it feat(dedup): java-sdk 2.2.0: faster, steadier per-test coverage Nov 20, 2026
release.yml builds keploy-sdk with the release profile and smoke-tests
the jar of that clean build, then deploys to Maven Central from a
second pass over the same target/, with the profile and no clean. If
that pass changes the jar's bytes, Central gets a jar other than the
one tested, and CI does not notice: the Java Agent job never packaged
twice, and release.yml states the rule only in a comment.

The job now builds as release.yml does: with the release profile, and
with signing skipped by -Dgpg.skip, since CI holds no release key. It
records the sha256 of keploy-sdk's jar and of its sources jar. Its last
step runs that build again without clean and checks both digests.
Shade writes the jar anew on every pass, from the same entries and
times, so what the check holds is the bytes. The unit-test step between
them stops at the test phase, but what it changes in target/classes
would reach the jar in that last pass, so it is checked too. The
javadoc jar is left out: every build makes it anew, with the time in
its pages, as release.yml already says of it. The two new steps run
with `shell: bash`, so a command that fails inside a pipe fails the
step.

Checked: on JDK 8, 17 and 21 every step of the job passes, the unit
tests with KEPLOY_PROVEN unset and =1, and both jars keep their digests
through both unit-test passes, the smoke and the build without clean.
On JDK 17 the unit tests pass and the new step fails in two cases:
with copy-jacocoagent set to overwrite the file on every build, and
with a plugin under keploy-sdk's release profile that touches a file
in its classes on every build. The second passes a check run without
the release profile.

Signed-off-by: slayerjain <shubhamkjain@outlook.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant