Skip to content

Release candidate - #190

Draft
jwrosewell wants to merge 42 commits into
mainfrom
rc
Draft

jwrosewell wants to merge 42 commits into
mainfrom
rc

Conversation

@jwrosewell

@jwrosewell jwrosewell commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Releases what has collected on rc since the last release, #112, which merged on 13 November 2026 at 10:35 UTC. Maven Central refused that release's publish (run 43410313251, "Publishing blocked: your organization has reached its monthly release size limit ... Usage resets 2026-12-01"), so nothing of #112 was published and 4.5.130 is still the newest version of every package. The packages this release publishes therefore also carry every change #112 lists, and "Before you upgrade" below covers anyone moving from 4.5.130.

This release merges into main on Tuesday 17 November, and Publish runs on 1 December, when Maven Central's monthly limit resets. Only the patch number moves.

The C library. This release is built on the cxx submodule at 7992af70, the 4.5.106 release, read from the gitlink of the head. It is not built on this week's C library release. Moving Java forward is open, and What remains says so.

What this release brings into conformance with the specification

The Usage Sharing Element accepts no Evidence and reads the web request

usage-sharing-element.md lines 90 to 96 at dbaf099c says

The element accepts no Evidence, so it does not accept the session cookie either. When the track session option is on, the element MUST read the cookie named by the session cookie name option from the same two places as the values it shares, being the Evidence in the Flow Data and the web request supplied as server.http-request.

The element now accepts no Evidence and reads the values it shares from the Evidence in the Flow Data and from the web request. A test sends a header that no element reads through the web integration and finds it in the usage sharing message (#127), and further tests show that within one session a request differing only in its sequence is not shared again whilst a changed 51D_ value is, and that JA4, H2 and both billing keys added beside the web request reach the message (#136). Only the Cookie header is blocked by default, so a header named Cookies is shared (#132), and the filter compares Evidence keys in the root locale, so a Turkish or Azeri default locale no longer lets ID.EMAIL, HEADER.COOKIE or a COOKIE.<name> key past the rules that keep them out (#83). The record itself is written as the specification's Processing section sets out (#139), and closing stops sending after its wait, discards the rest and warns with the number of entries left (#147).

Usage sharing shares every query value by default, and no cookie but the 51D_ ones

usage-sharing-element.md lines 65 to 69 at 30b3741f says

It MUST also return false, whatever the configuration and even when all Evidence is shared, for any key whose prefix is cookie. and whose field does not start with 51D_.

and lines 76 to 81 say "Share all query string parameters is on by default, so every query string and HTTP form parameter is shared unless a user turns it off." Both are now the defaults (#160).

IP Intelligence reads its native results one thread at a time, types Latitude and Longitude as numbers, and processes in the cloud with the shared logic

ip-intelligence-on-premise.md lines 520 to 525 at dbaf099c says Aspect Data that reads from a ResultsIpi instance "MUST let only one thread at a time read from its results", data-model.md lines 48 and 49 give Latitude and Longitude the type float, and ip-intelligence-cloud.md lines 48 and 49 say "The majority of the processing SHOULD be handled by the shared Cloud Aspect Engine logic." IP Intelligence Element Data now holds one lock for every read of its native results, the property metadata types a native single as Float, and the IP Intelligence cloud engine leaves processing to the shared Cloud Aspect Engine logic (#177).

A property list splits at every comma, and Element Data is found whatever the case of its key

pipeline-configuration.md lines 50 to 61 at b3b2c0dd gives a list setting in a configuration file as one comma-separated string, and access-to-results.md line 11 at b3b2c0dd says "String comparisons MUST be case insensitive." An engine builder's setProperties(String) now splits the list at every comma and trims each name, where it split only at a comma with one white space character on each side (#166), and Flow Data finds Element Data by its key whatever the case, so flowData.get("Device") finds what is stored under device (#167).

A missing Property gives the specification's message and reason

properties.md lines 116 to 118 at dbaf099c says

When the user attempts to access a Property that is defined for an Engine, but does not exist in the result set for some reason, the system MUST throw an error/exception with a message explaining why the Property is not available.

and the table under it gives each reason's message. The missing Property messages are now that table's text, the reason is decided in the order the specification gives, a failed cloud request has its own reason, CloudRequestFailed, and the on-premise Device Detection and IP Intelligence data read each Property by its position rather than asking the native results by name (#175). The same change turns usage sharing off in the IP Intelligence console examples with the comment the specification asks for, adds a programmatic data update example, ManualDataUpdate, declares what FlowData throws, and deprecates the test utility classes FileUtils and KeyUtils in the published device-detection.shared jar.

The JSON Builder writes errors as a list of messages

json-builder.md lines 106 to 107 at dbaf099c says

Add any errors from the Flow Data errors collection to the result. This is structured as a string array of the error messages under the top-level key errors.

errors is now a list of messages in the order they were recorded, where it was an object keyed by element (#128). A Property assigned no value is written as null beside its nullreason, at the top level and in each item of a list of Element Data, and the JavaScript Builder's public constructors refuse a protocol that is neither http nor https (#145).

The JavaScript Builder minifies its script, with a library, and its cost is tested

javascript-builder.md lines 105 to 114 at dbaf099c says

Finally, the resulting JavaScript will be minified (or not, based on the configuration provided). Minification SHOULD be performed using an existing third-party library.

Performance tests MUST be in place to verify the performance impact of this feature.

and line 350 gives SetMinify the default True. The script was never minified, whatever SetMinify said. It is now minified with net.logicsquad:minifier once for each form of the template rather than on each request, with each request's values put into the minified form afterwards so that the minifier never touches them, a performance test holds a minified render to within twice the cost of an unminified one, and a test reads the JSON endpoint's answer and the JSON in the script and holds them to the same form (#178).

A cloud engine works with empty Property metadata

cloud-aspect-engine.md lines 57 to 61 at dbaf099c says

Where the Property metadata is empty, for example before discovery has completed, a Cloud Aspect Engine MUST still work, reading the response JSON and inferring the type of each value from the value itself, as it already does, and anything that reads the metadata to decide what the Engine can produce MUST tolerate finding nothing there.

The Device Detection cloud engine gave none of the response's values when its metadata was empty, and reading one threw PropertyMissingException. It now reads them, with a number given the type its typed accessor declares, and the Cloud Request Engine logs each entry of a response's warnings (#172).

The client-side endpoints answer a revalidation without running the Pipeline

web-integration.md lines 214 to 222 at dbaf099c says

The Pipeline Evidence key filter is the union of the accepted Evidence of the Pipeline's Flow Elements, and a Flow Element accepts only the Evidence that can change its results, so the ETag and the Vary header cover only those keys. The Usage Sharing Element accepts none. If-None-Match changes no result, so no Flow Element accepts it, and a revalidation's ETag is the same as the one it sends.

The JavaScript and JSON endpoints send a weak ETag of the Evidence the request supplied and answer a matching If-None-Match before the Pipeline runs, in the servlet filter and in the Spring interceptor, with a 304 to a GET or HEAD and a 412 to any other method but CONNECT, OPTIONS and TRACE. The 304 carries the Vary, Cache-Control, ETag and CORS headers of a good response, a response built whilst processing recorded errors sends Cache-Control: private, no-cache and no ETag, and the maximum age can be set, with 1800 seconds still the default (#67). The web integration also supplies server.request-query, the query string as sent and empty when there is none, which usage sharing never shares (#144).

A data update failure is logged as its row of the Errors table

data-updates.md line 481 at dbaf099c says of the Errors table's messages

These messages MUST also be consistent across languages to make life easier for 51Degrees support.

Every HTTP failure of the data file download, one cut short included, is logged as its row of that table, and a failed check is logged at Error with the reason in the attached error (#129). Four more rows now match the table: a response with no Content-MD5 header is a warning and the data is applied (data-updates.md line 505), a failed integrity check gives AUTO_UPDATE_ERR_MD5_VALIDATION_FAILED with both hashes in its message, a data file with no temporary file path gives the new AUTO_UPDATE_TEMP_PATH_NOT_SET, and a failure writing data passed in from memory is a warning with the engine still refreshed (#186).

A cloud Property the response leaves out is reported, not guessed

The Device Detection and IP Intelligence cloud engines now throw PropertyMissingException for a Property the response leaves out, as Location already did, and a Property the response gives only as a nullreason keeps that reason (#143). The cloud Pipeline builders take the properties wanted, which a caller authenticating with a licence key alone needs, and a live test authenticates on a licence key alone (#131).

51Did factor words are read whatever their case

Factor words are compared ignoring case, so a redeem answer's Verified or NotRecorded reads as that outcome where it read as Mismatch, and a payload of one to four bytes naming version 1, 2 or 3 is refused as UNSUPPORTED_PAYLOAD_VERSION naming that version, where it was PAYLOAD_TOO_SHORT (#141).

Before you upgrade

  1. Each package now has its own version. Packages are released only when a file under them changed since their last release, or a package they depend on is released, and each is tagged <package>/<version> (#63). A project that sets one version for every 51Degrees artifact needs each artifact's own version from now on. Each published module requires the versions it was built with, so Maven resolves a consistent set from any one of them. The Build section gives the plan the Publish dry run printed, package by package.
  2. The JSON Builder writes errors as a list of messages, in the order the errors were recorded, where it wrote an object keyed by element. An application that reads the JSON Builder's output and treats errors as an object must read it as a list, which is what the specification asks for, the .NET builder writes and the client-side script reads (#128).
  3. The JSON Builder writes a Property assigned no value as null, beside its nullreason, at the top level and in each item of a list of Element Data, where it wrote only the reason. Code that took a missing key to mean no value now finds the key with a null value (#145).
  4. A JavaScriptBuilderElement made with a public constructor refuses a protocol that is not http or https, white space included, with PipelineConfigurationException when it is made. Before, the element was made and gave a script with no callback URL on each request. The builder already refused such a protocol (#145).
  5. An application that builds its own Evidence has to give Usage Sharing the web request. This applies to an application that adds a value to Evidence only when Pipeline.getEvidenceKeyFilter() includes its key. An application that uses the web integration has nothing to change. Up to 4.5.130 Usage Sharing accepted the headers it shares, so that check kept them. Usage Sharing now accepts none, so the same check drops every header that no other Flow Element reads, and those headers stop being shared. Add the request with flowData.addEvidence(ShareUsageEvidence.HTTP_REQUEST_KEY, request). ShareUsageEvidence is in fiftyone.pipeline.engines.fiftyone.data, and the key is server.http-request. The value is a java.util.Map that holds each value of the request under the key it would have as Evidence, such as header.user-agent, cookie.51D_Profile, query.q and server.client-ip. Usage Sharing shares every Evidence value in the Flow Data, then each value of that map whose key the Evidence does not hold. Its rules still apply, so the Cookie header, a blocked header and a cookie whose name does not start 51D_ are left out. The map itself is never shared and is not sent to the cloud service. In a servlet application, WebRequestEvidenceServiceCore.addEvidenceFromRequest in pipeline.web adds the request for you (#68).
  6. A request header named Cookies is now shared by Usage Sharing by default, as every other header but Cookie is. To keep it out, add it to the blocked headers with setBlockedHttpHeader("Cookies") on the usage sharing builder (#132).
  7. The client-side endpoints send a weak ETag of the Evidence the request supplied, and answer a matching If-None-Match before the Pipeline runs. An ETag kept from an earlier version does not match, so each browser gets one full response after the upgrade. An application that wires the web integration itself, rather than through PipelineFilter or FiftyOneInterceptor, calls the new serveNotModified before processing to answer revalidations the same way (#67).
  8. A failed check for a data update is logged at Error as "Error during check for data update", with the reason in the attached error, where it was a warning. An application that alerts on Error entries sees these (#129).
  9. A cloud Property the response leaves out throws PropertyMissingException from the Device Detection and IP Intelligence cloud engines, where it gave a value of Unknown or null. Location behaved this way already (#143).
  10. Usage sharing's close gives up waiting after its limit and discards what is left in the queue, with a warning giving the number of entries left, rather than holding the process that closes it (#147).
  11. The usage sharing record changes form. Its root no longer carries version="1.1", it is sent as text/xml where it was text/xml; charset=utf-8, FlowElement holds each element's fully qualified class name, and SessionId, Version, LanguageVersion, ClientIP, ServerIP and Platform are written empty where they were left out, each as the specification's Processing section sets out (#139). Anything that reads the record itself, rather than the usage sharing service, sees the new form.
  12. Missing Property reasons and messages change (#175). The messages are the specification's text. A Property missing because the cloud request failed gives the new reason CloudRequestFailed. A Cloud Engine no longer gives DataFileUpgradeRequired, and one whose metadata has not loaded gives Unknown or CloudRequestFailed where it gave ProductNotAccessibleWithResourceKey. Code that matched on a message's text or on those reasons needs to follow.
  13. A Property named when the on-premise engine was built, and missing from the data file, now reads as a value with hasValue() false and the native message, where reading it threw PropertyMissingException (#175). This follows from reading each Property by its position. It is read from the code and has no test of its own, and whether such a read should throw with the upgrade message instead is an open question on the specification.
  14. FlowDataDefault.get(String) before processing throws IllegalStateException, a subclass of the RuntimeException it threw, so a caller that catches RuntimeException sees no change (#175).
  15. FileUtils and KeyUtils in device-detection.shared are deprecated. They are test utility classes and will leave the published jar in a later release. The examples use their own DataFileHelper and KeyHelper (#175, #187).
  16. The client-side script is now minified by default. Up to 4.5.130 setMinify had no effect and the script was served as the template produced it. Anything that compares the script byte for byte, or reads it expecting the template's layout, sees minified output. Set setMinify(false) on the JavaScript Builder to keep the old output (#178).
  17. pipeline.javascriptbuilder has a new dependency, net.logicsquad:minifier 1.0, under the 2-Clause BSD licence, with its log4j binding for SLF4J left out so that it does not take over an application's logging. A project that pins its own transitive versions needs it on the path (#178).
  18. Usage sharing shares every query string and HTTP form parameter by default. To share only the 51D_ parameters and those named, as before, set setShareAllQueryStringParameters(false) on the builder, or ShareAllQueryStringParameters to false in the options file (#160).
  19. With share all Evidence on, a cookie whose name does not start with 51D_ is no longer shared, the session cookie included. There is no setting to share it (#160).
  20. Latitude and Longitude are Float values where they were String values wherever the type is taken from the metadata: getType(), get("latitude"), get("longitude") and asKeyMap() on the on-premise data, and so the JSON the JSON Builder writes for them, which now carries numbers where it carried text (#177).
  21. The IP Intelligence cloud engine behaves as the Device Detection one does when the cloud request fails. With no response it logs the shared warning and gives no values, where it failed reading the response, and with no Cloud Request Engine before it in the Pipeline it throws PipelineConfigurationException. A subclass that overrode processEngine overrides processCloudEngine instead (#177).
  22. A data update response with no Content-MD5 header is now applied, with a warning, where it stopped the update with AUTO_UPDATE_ERR_MD5_VALIDATION_FAILED. A data file with no temporary file path reports AUTO_UPDATE_TEMP_PATH_NOT_SET, a new member at the end of AutoUpdateStatus, where it reported AUTO_UPDATE_REFRESH_FAILED, and a failure writing data passed in from memory is logged at Warning, not Error (#186).
  23. setProperties("IsMobile,BrowserName") now asks for two properties, where it asked for one named IsMobile,BrowserName (#166).
  24. Element Data keys are compared ignoring case. flowData.get("Device") finds the Element Data stored under device, where it threw NoSuchElementException, and adding Element Data under a key that differs only in case finds what is already there (#167).

New features

  • A programmatic data update example, ManualDataUpdate, which asks the data update service for an update now (#175).
  • Package descriptions say what each package does in the words people search for, and the parent poms' source links point at this repository (#27).
  • The web integration supplies server.request-query (#144).
  • The builder defaults that were literals are named constants, with no value changed, and the lazy loading Javadoc gives the real default of 1 second (#130).
  • Every Device Detection and IP Intelligence example ends with a Find out more section, tagged for the example, in the console and on the web pages, an unknown performance profile ends the same way, the Device Detection web cloud options file uses https, and the IP Intelligence web examples no longer fall back to the Enterprise file (#133). The Pipeline developer examples and the Location console examples do the same (#134), and the IP Intelligence API example says at start-up what the Enterprise data file adds, with the contact link (#159).

Performance and test coverage

  • The share usage overhead test measures a median over warmed up rounds, with limits set from every runner's figures (#91).
  • Chrome starts once before the Selenium contract tests (#124), and so does the first Chrome driver session, timed, so the first test no longer carries the slow first start on the Intel macOS runners (#138).
  • A test checks the positions of the elements the web integration adds (#155), tests keep FodId's byte layout, payload version and field-setting constructor out of the public surface (#157), and the configured interval data update test no longer races when its random delay comes out as zero (#161).
  • The usage sharing builder tests, which JUnit 5 passed over because they were written for JUnit 4, now run (#142).
  • A workflow builds and runs the getting started examples from a fresh clone against the packages on Maven Central, on each runner's own JDK, downloading each free data file once a run (#137).

Housekeeping

  • Each pull request builds on the configurations its changes need, and the CI script tests run on pull requests (#43).
  • Configuration page links point at https://51degrees.com/configure (#115), AGENTS.md points at common-ci's one list of environment variable names (#118), AGENTS.md records that a test failing on timing is fixed rather than only re-run (#122), and AGENTS.md and the example READMEs describe the examples as they run today (#162).

Fixed

Issues this closes

Build

The four release checks run on the final head between 02:00 and 05:00 UTC on Tuesday 17 November, being the release build with macOS, the Publish dry run, the Nightly Pipeline dry run and the timing against main. This section gives each run and its result then, and nothing in this body is reported as checked until it does.

Independent review

Every change in this release was reviewed as its own draft into rc at the time it merged. This release body has not yet been independently reviewed at its head. A review of rc 20f9089 against main f212dfc on 16 November left the release not ready on three findings, which were that no release draft existed, that no release check had run on the head, and that the examples did not build from a fresh clone. This draft answers the first, the Build section answers the second once the checks run, and #187 fixed the third.

What remains

  1. The C library. This release is built on cxx 7992af70, the 4.5.106 release, so the weighted Latitude and Longitude are not rounded to 3 decimal places, which this week's C library release does. Whether Java moves to it is open.
  2. The package build on the runner's newest JDK. The scheduled Nightly Pipeline on main fails in its Build job on ubuntu-26.04 with "Unsupported class file major version 69", which is a Java 25 class file read by PMD 3.26.0. The fix, #140, is not in this head, so until it is the dashboard badge stays red and the Publish and nightly dry runs stop in Build.
  3. Examples. A cloud example run with no resource key ends with an exception (#163), Comparer needs the Enterprise data file (#158), and a missing Property is answered with an HTML link printed to the console (#193). None of those is in this head.
  4. Merging this release into main starts a real publish at the next 03:00 UTC Nightly Pipeline, because its Publish job runs with dryrun false on the schedule. Until #140 is in, that job stops in Build and uploads nothing. Once it is in, each night's run tries the upload, which Maven Central refuses until its monthly limit resets on 1 December, so the nightly fails until then, and the first scheduled run after the reset publishes whatever main holds.
  5. The configuration page link in two missing Property messages. pipeline.engines' Constants still gives https://configure.51degrees.com in the messages for a Property not in the resource key, where #115 moved every other link to https://51degrees.com/configure, because #175 copied the specification's table, which still gives the old address.
  6. The issues left open and why each cannot be done for this release are listed with the release.

* DOC: Point at the one list of environment variable names

AGENTS.md gains a section linking common-ci's list of environment variable
names (environment-variables.md, made from environment-variables.json) at
commit 18f9a570, and saying the list decides.

Documentation only.

* DOC: Link the list of environment variable names at d8a9c596

The list at that commit holds the names review found missing.
The share usage overhead test now measures the median of warmed-up rounds of 250 calls, timed with System.nanoTime, and fails at about ten times the highest median any runner shows at usual speed: 0.07 ms a call with one Evidence value and 40 ms with 1,000, both set from 82 medians across Ubuntu, Ubuntu ARM, Windows, macOS arm64 and macOS x64. The old single-pass test failed by chance on Intel macOS (0.1083 ms against 0.1 ms in release build 43364257944). Full build 43499659322 passed all 15 jobs with macOS.
…#127)

The Usage Sharing Element accepts no Evidence. It finds the values to share in the web request, which the web integration adds to Evidence as `server.http-request` on every request ([`WebRequestEvidenceServiceCore.java` line 73](https://github.com/51Degrees/51degrees-java/blob/30e4886708974308d4b70e1804424f55560824a7/pipeline/pipeline.web.packages/pipeline.web/src/main/java/fiftyone/pipeline/web/services/WebRequestEvidenceServiceCore.java#L73), with every value recorded at [line 150](https://github.com/51Degrees/51degrees-java/blob/30e4886708974308d4b70e1804424f55560824a7/pipeline/pipeline.web.packages/pipeline.web/src/main/java/fiftyone/pipeline/web/services/WebRequestEvidenceServiceCore.java#L150)). A header that no element reads is therefore not in Evidence at all, and reaches the usage sharing message only through that web request.
…s point at this repository (#27)

- **Package descriptions.** The `<description>` in six pom files now says what each package does, in the words developers type when they look for a package like it, and ends by naming the products it is an alternative to. The detail, and a before and after table for each artifact, follow below.
- **Source links.** The `<scm>` `<url>`, `<connection>`, and `<developerConnection>` in the four parent poms ([`pipeline/pom.xml`](https://github.com/51Degrees/51degrees-java/blob/7d8b2091582e3f77b8368e27a34fbffe9a7da1b7/pipeline/pom.xml#L374-L379), [`device-detection/pom.xml`](https://github.com/51Degrees/51degrees-java/blob/7d8b2091582e3f77b8368e27a34fbffe9a7da1b7/device-detection/pom.xml#L327-L332), [`ip-intelligence/pom.xml`](https://github.com/51Degrees/51degrees-java/blob/7d8b2091582e3f77b8368e27a34fbffe9a7da1b7/ip-intelligence/pom.xml#L389-L394), and [`location/pom.xml`](https://github.com/51Degrees/51degrees-java/blob/7d8b2091582e3f77b8368e27a34fbffe9a7da1b7/location/pom.xml#L312-L317)) pointed at the earlier repositories [pipeline-java](https://github.com/51Degrees/pipeline-java), [device-detection-java](https://github.com/51Degrees/device-detection-java), [ip-intelligence-java](https://github.com/51Degrees/ip-intelligence-java), and [location-java](https://github.com/51Degrees/location-java). Those repositories are archived, so the source link on each package page was a dead end. Two changes fix that.
  1. All three elements now point at this repository in https form, `https://github.com/51Degrees/51degrees-java` for `<url>` and `scm:git:https://github.com/51Degrees/51degrees-java.git` for `<connection>` and `<developerConnection>`. The `<connection>` used git://, which GitHub [permanently turned off on March 15, 2022](https://github.blog/security/application-security/improving-git-protocol-security-github/). The `<developerConnection>` named `info@51degrees.com` as its host rather than GitHub.
  2. Each parent's `<scm>` now sets `child.scm.connection.inherit.append.path`, `child.scm.developerConnection.inherit.append.path`, and `child.scm.url.inherit.append.path` to `"false"`. Child packages inherit these links, and without the attributes Maven adds each child's artifactId to them. The [Maven model reference](https://maven.apache.org/ref/3.10.0/maven-model/maven.html) gives the default for `<scm>` `<url>` as "parent value [+ path adjustment] + (artifactId or project.directory property), or just parent value if scm's child.scm.url.inherit.append.path="false"", and says the same for the two connections. That is why the Maven Central page for [`com.51degrees:device-detection` 4.5.130](https://central.sonatype.com/artifact/com.51degrees/device-detection/4.5.130) links its source to https://github.com/51Degrees/device-detection-java/device-detection, which GitHub answers with 404. With the attributes, a child's links are the repository itself. The attributes need Maven 3.6.1 or later, and every runner image this repository's CI uses lists Maven 3.9.12 or later (see "How it was checked").
In [full build 43411790985, attempt 1](https://github.com/51Degrees/51degrees-java/actions/runs/43411790985/attempts/1), the job [MacOS_X64_Java_11](https://github.com/51Degrees/51degrees-java/actions/runs/43411790985/job/131839970848) failed in the Selenium contract tests against the cloud web example, and that cancelled MacOS_X64_Java_17, MacOS_X64_Java_21 and MacOS_Java_11.
A pull request into `rc` runs all 15 configurations in `ci/options.json`, whatever it changes. Six of them are macOS, and only five macOS jobs run at once across the whole organisation. Every configuration runs the three IP Intelligence tests that load the multi-gigabyte Enterprise data file, every Windows configuration installs WMIC, which is only there for one of those tests, and fourteen configurations run the Selenium contract, the flakiest check in the window.
Step 3.5 of the run book in [common-ci#306](51Degrees/common-ci#306). [`ci/build-package.ps1` line 31](https://github.com/51Degrees/51degrees-java/blob/7b5c70fcadf02052b9bb3a5b3bbc298b5571af0a/ci/build-package.ps1#L31) ran `mvn versions:set` in each package tree, so every one of the 45 artifacts this repository publishes to Maven Central got a new version whenever anything changed. The last release, 4.5.130 on 5 November, published all 45 at that version.
[`EvidenceKeyFilterShareUsage`](https://github.com/51Degrees/51degrees-java/blob/30e4886708974308d4b70e1804424f55560824a7/pipeline/pipeline.engines.fiftyone/src/main/java/fiftyone/pipeline/engines/fiftyone/data/EvidenceKeyFilterShareUsage.java#L166-L196) decides which Evidence the usage sharing element sends. On `rc` at 30e4886 it lower-cases each key with `key.toLowerCase()` at line 192, and the blocked headers, the query string parameters and the session cookie name at lines 170, 172 and 178, all in the default locale of the JVM. Under a Turkish or Azeri default locale a capital I lower-cases to a dotless i (U+0131), so `ID.EMAIL` becomes `ıd.emaıl`, does not match the never-shared key `id.email`, and is shared. `HEADER.COOKIE` and `ID.SALT` get through the same way, and so does a blocked header written with a capital I.
The configuration page has moved from `configure.51degrees.com` to https://51degrees.com/configure. The links and mentions that people read (READMEs, code comments, example output, and the messages that tell someone where to create a Resource Key) now use the new address, and the text that names the page calls it the configuration page.
…web request (#136)

Two of James Rosewell's usage sharing rules of 13 and 14 November had no test in Java, found in the cross-language check of usage sharing on 14 November.
… license key alone against the live service (#131)

Closes #12.
… blocked (#132)

The usage sharing filter blocks two headers by default where the specification blocks one. `EvidenceKeyFilterShareUsage` adds both `cookies` and `cookie` to its blocked headers whatever it is given ([lines 165 to 168 at rc 1c99b72](https://github.com/51Degrees/51degrees-java/blob/1c99b72f975209ab30082985f6cceeaf71d9aca1/pipeline/pipeline.engines.fiftyone/src/main/java/fiftyone/pipeline/engines/fiftyone/data/EvidenceKeyFilterShareUsage.java#L165-L168)), so a request header named `Cookies` is never shared unless all Evidence is shared. `Cookies` is not the Cookie header, so it is an ordinary header that the specification shares by default.
… 9110 ask (#67)

Closes #11. This brings the client-side JavaScript and JSON endpoints of the web integration, served by both the servlet filter and the Spring interceptor, in line with the [Cache header detail](https://github.com/51Degrees/specifications/blob/6570727d2af6967390d3c4fbec77b9cf0540d3f3/pipeline-specification/features/web-integration.md#cache-header-detail) section of the web integration specification, quoted below at [6570727d](51Degrees/specifications@6570727), the head of the specifications `main` branch. That section was written by [51Degrees/specifications#53](51Degrees/specifications#53), [#81](51Degrees/specifications#81), [#115](51Degrees/specifications#115), [#121](51Degrees/specifications#121) and [#136](51Degrees/specifications#136), all merged.
…s the specification's Errors table rows (#129)

Closes #125.
…t more section (#133)

Three gaps in the Device Detection and IP Intelligence examples, found in the audit of the examples on 13 November.
… out more section (#134)

No Pipeline developer example and no Location example ended with a "Find out more" section, found in the audit of the examples on 13 November. The 51Degrees rule for examples is that every example ends with a clearly separated "Find out more" section that gives the person running it links to follow up, being the relevant repositories as plain addresses and 51degrees.com pages tagged with the campaign convention that common-ci's [`scripts/utm-lint.ps1`](https://github.com/51Degrees/common-ci/blob/1c5c0e0b09a1f77722e20ba2d10cbd1031caca9e/scripts/utm-lint.ps1) checks.
The web integration supplies the method and the path of the request line as Evidence, and not the query string. [web-integration.md lines 95 to 114 at 706de1f0](https://github.com/51Degrees/specifications/blob/706de1f0cb3edd560eb7fd7ef3bb19b93a4883dc/pipeline-specification/features/web-integration.md?plain=1#L95-L114) list "Request query string, whole and undecoded. Key is `server.request-query`" beside the method and the path, and say:
…col in the JavaScript Builder's constructor (#145)

Two places where the JSON Builder and the JavaScript Builder differ from the specification, found by a conformance check of this repository against [specifications at 706de1f0](https://github.com/51Degrees/specifications/tree/706de1f0cb3edd560eb7fd7ef3bb19b93a4883dc).
…hort payload (#141)

Two places where the 51Did package differs from the specification, found by a conformance check of this package against [specifications at 706de1f0](https://github.com/51Degrees/specifications/tree/706de1f0cb3edd560eb7fd7ef3bb19b93a4883dc).
…#137)

Two gaps against the examples rule, that every example builds and runs both from a fresh clone and against the published packages, found in the audit of the examples on 13 November.
Tests only. The web integration adds the Sequence Element, the JSON Builder Element, the JavaScript Builder Element and the Set Headers Element where the application's Pipeline lacks them, in the positions the specification gives, but the only test checked that each one is there, not where. This adds a test of the positions.
…nstructor out of reach (#157)

Tests only. Three rules of the 51Did specification limit what `FodId` offers a caller, and the Java package meets all three, but no test would fail if a later change broke one of them. This adds those tests. It is the follow-up that [#141](#141) names under What remains.
…ence API example (#159)

The IP Intelligence API example answers in JSON, so on the free Lite data file a caller saw the three registration properties and was never told what the Enterprise data file adds, where to ask for it, or where to find out more.
…its verify (#161)

Tests only. `DataUpdateServiceTests.DataUpdateService_Register_AutoUpdateConfiguredInterval` fails about once in 600 runs, and it failed `Ubuntu_Java_17` in full build [43667464820](https://github.com/51Degrees/51degrees-java/actions/runs/43667464820) of [#156](#156), which changes nothing in `pipeline.engines`. This fixes the cause rather than leaving it to a re-run, which is the rule for a test that fails on timing. [#122](#122), which is open and not yet merged, adds that rule to AGENTS.md.
…143)

The Device Detection and IP Intelligence cloud engines add an entry to the Element Data for every property in their metadata, whether or not the cloud response holds it ([`DeviceDetectionCloudEngine` line 148](https://github.com/51Degrees/51degrees-java/blob/b33daa0f32964dbf5b5f77ba5e947ef1ac75c55d/device-detection/device-detection.cloud/src/main/java/fiftyone/devicedetection/cloud/flowelements/DeviceDetectionCloudEngine.java#L148) and [`IPIntelligenceCloudEngine` line 160](https://github.com/51Degrees/51degrees-java/blob/b33daa0f32964dbf5b5f77ba5e947ef1ac75c55d/ip-intelligence/ip-intelligence.cloud/src/main/java/fiftyone/ipintelligence/cloud/flowelements/IPIntelligenceCloudEngine.java#L160) at `rc` b33daa0). A property the response leaves out therefore reads as a value that "has no value" with a null message, where it should throw `PropertyMissingException` with a message saying why the property is not there.
…ion sets out (#139)

Part of [#28](#28), which stays open for one point only, the order of the Evidence entries (point 9 below).
…5 seconds (#147)

Closing the Usage Sharing Element waits up to 5 seconds for the data in its queue to be sent. When that time runs out, `managedResourcesCleanup` logs "Could not send final data on close down" and calls `shutdownNow` on the element's executor ([`ShareUsageBase` lines 753 to 755 at rc b33daa0](https://github.com/51Degrees/51degrees-java/blob/b33daa0f32964dbf5b5f77ba5e947ef1ac75c55d/pipeline/pipeline.engines.fiftyone/src/main/java/fiftyone/pipeline/engines/fiftyone/flowelements/ShareUsageBase.java#L753-L755)). `shutdownNow` only interrupts the sending thread. A send that is waiting on the network carries on, and the thread then goes back to its loop, which keeps sending while the element is stopping and the queue is not empty ([`ShareUsageElement` line 489](https://github.com/51Degrees/51degrees-java/blob/b33daa0f32964dbf5b5f77ba5e947ef1ac75c55d/pipeline/pipeline.engines.fiftyone/src/main/java/fiftyone/pipeline/engines/fiftyone/flowelements/ShareUsageElement.java#L489)). So after `close()` has returned, the element can go on sending everything that was queued, for as long as the remote service takes to answer each message, and nothing says how much was left.
Tests that fail on a time limit, a performance threshold or a timeout, and pass when run again, keep coming back at random, and each time someone has to look into the failure before running it again. [`AGENTS.md`](https://github.com/51Degrees/51degrees-java/blob/30e4886708974308d4b70e1804424f55560824a7/AGENTS.md) says nothing about them, so a contributor who meets one runs it again.
The Java packages differed from seven SHOULD sentences of the specification.
`ShareUsageBuilderBaseTest` holds 32 tests of the usage sharing builder's setters, and none of them has ever run. The class uses JUnit 4 (`org.junit.Test`, `org.junit.Before`, `@Test(expected = ...)`), while its module, `pipeline/pipeline.engines.fiftyone`, depends on JUnit 5 alone ([pom.xml lines 61 to 64 at rc 23e0fc6](https://github.com/51Degrees/51degrees-java/blob/23e0fc649e97b290436f22eb0f6894002a9375f6/pipeline/pipeline.engines.fiftyone/pom.xml#L61-L64)). Surefire then runs the JUnit 5 engine only, and a JUnit 4 class there is passed over without a word. [#62](#62) noted that the class "did not run in any of the runs above".
…READMEs (#162)

Documentation only. AGENTS.md and the two example READMEs described how the examples behaved before the free data file and Find out more changes, so an agent or a person following them was told to pass the Lite file as an argument, to run from the repository root only, and to expect a pricing link.
…y, and log a cloud response's warnings (#172)

The Cloud Request Engine and the Device Detection cloud engine differed from the specification in two places, and two more rules had no test.
…when all Evidence is shared (#160)

Usage sharing now shares every query string and HTTP form value by default, and shares no cookie but the `51D_` ones even when all Evidence is shared, as [specifications#155](51Degrees/specifications#155) now says.
…Longitude as Float, and process cloud IP Intelligence with the shared engine logic (#177)

The IP Intelligence packages differed from the specification in three places, and eight more rules had no test.
An engine builder's `setProperties(String)`, the setter for a property list given as one string, now splits the list at every comma and trims the spaces round each name. Until now it split only where a comma had one white space character on each side, so `IsMobile,BrowserName`, the form the specification gives, was passed on as one property of that name.
Element Data keys are now compared ignoring case, so `flowData.get("Device")` finds the Element Data stored under `device`, and adding Element Data under a key that differs only in case finds the Element Data already there. Until now the lookup compared keys exactly, so `flowData.get("Device")` threw `NoSuchElementException`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment