Repository navigation
Nine fixes the specification asks for, in usage sharing, data updates, Evidence keys and the JavaScript Builder - #170
Draft
jwrosewell wants to merge 18 commits into
Draft
jwrosewell wants to merge 18 commits into
jwrosewell wants to merge 18 commits into
Conversation
…ry string parameters is on by default (US-5)
… string parameter is shared unless a user turns the option off (US-5)
…D_, the session cookie among them, is still not shared (US-35, US-36)
… cookie among them, even when all Evidence is shared (US-35, US-36)
…eported, and does not stop later checks (DU-13, DU-14)
… unexpected one as AUTO_UPDATE_UNKNOWN_ERROR, and keep later checks running (DU-13, DU-14)
…e for MD5, temporary path, other errors and writes from memory (DU-15)
…on when Content-MD5 is missing, add AUTO_UPDATE_TEMP_PATH_NOT_SET and AUTO_UPDATE_UNKNOWN_ERROR, and warn on a failed write from memory (DU-15)
… whitelist and the JavaScript builder's query parameters (EV-5)
…t, and read query Evidence in the JavaScript builder without regard to case (EV-5)
…'s script, which is on by default (JB-7)
…ify option, using a cut-down single pass minifier that keeps line breaks (JB-7)
… space is around each comma (CF-8)
…t is read the same whatever white space is around the commas (CF-8)
…stored under device is found as Device (PR-3)
…ed key map (PR-3)
…th its weighting (IP-32)
…th its weighting, read through a new weightedValues helper (IP-32)
This was referenced Nov 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The Java packages differed from the specification in nine places.
51D_or was named in the included list, where share all query string parameters is on by default.51D_cookies may ever be shared.Content-MD5header failed the update, and a missing temporary path and an unexpected error both gaveAUTO_UPDATE_REFRESH_FAILED. A failed write of data passed in from memory was not logged as a warning.setPropertiessplit a property list only at a comma with white space on both sides, so "IsMobile,BrowserName" was read as one property.devicewas not found asDevice.What the specification says
All at 51Degrees/specifications dbaf099c.
usage-sharing-element.md lines 80 to 83
usage-sharing-element.md lines 65 to 69 and line 99
data-updates.md lines 474 to 477
data-updates.md lines 481 to 483, with the rows of the Errors table at lines 505 to 507, 511 and 515
evidence.md line 10
javascript-builder.md lines 105 to 114, and the
SetMinifyrow, default true, at line 350pipeline-configuration.md line 50
access-to-results.md lines 5 and 11
ip-intelligence-specification/required-examples.md line 107
What changed
Links are to the head, 0014283.
shareAllQueryStringParametersstarts true inShareUsageBuilderBaseline 69, with its@DefaultValueand Javadoc to match. Three tests inShareUsageOptionsTestspinned the old default. Two now expect the new one, and the third turns the option off to test the other options.51D_cookies are shared.EvidenceKeyFilterShareUsage.includelines 204 to 212 applies the cookie rule before the share all Evidence check. The session cookie is still included for the tracker, which compares it and never sends it.ShareUsageOptionsTestsasserted thatcookie.plainis shared when all Evidence is shared, and now expects it not to be.DataUpdateServiceDefaulteach polling and watcher task catches a failure and logs it at Error as "An unhandled error occurred while checking for automatic updates for engine '<type>'", with the error attached. An error of a kind the service does not expect is reported to the callbacks as the newAUTO_UPDATE_UNKNOWN_ERROR, and the next polling check is still scheduled. The watcher task always resets its watch key (line 743), so it keeps running. A programmatic check and the update on start-up still pass a callback's error to their caller.Content-MD5header is logged as a warning and the update goes on (line 1138). A data file with no temporary path gives the newAUTO_UPDATE_TEMP_PATH_NOT_SET(line 1285). A failed write of data passed in from memory is logged as a warning and the engine is still refreshed (line 577). The two new statuses are added at the end ofAutoUpdateStatus.DataUpdateServiceTests.DataUpdateService_UpdateFromUrl_MD5Invalidallowed one warning and now allows the one Error the table gives.EvidenceKeyFilterWhitelistconstructors that take no comparator now useString.CASE_INSENSITIVE_ORDER. The constructors that take a comparator are unchanged, andEvidenceKeyFilterAggregatorTestsnow passes its case-sensitive comparator explicitly. The JavaScript Builder reads query Evidence whatever the case of its prefix (JavaScriptBuilderElementline 458).setMinifyoption onJavaScriptBuilderElementBuilder, default true as theSetMinifyrow gives, and a newJavaScriptMinifier. The minifier makes one pass over the script, removing comments, indentation and empty lines. It keeps line breaks, so the way the browser inserts semicolons is unchanged, and it keeps strings, template literals and regular expressions as they are. It is the cut-down process the specification allows, so the published package gains no new dependency. TheJavaScriptBuilderElementconstructor gains aminifyargument, and the constructor without it is kept and minifies. Tests that read the rendered text (JavaScriptObjectNameTests,SessionIdAndSequenceTests,JavaScriptUrlTestsand three inJavaScriptBuilderTests) setsetMinify(false).AspectEngineBuilderBase.setPropertiessplits at each comma, trims each name and skips an empty one.TypedKeyMapBuilderfinds a key withequalsIgnoreCase.PropertyHelpergainsweightedValues, which gives a single value a weighting of 1, andasWeightedString.GettingStartedOnPremandGettingStartedCloudread each value and its weighting through them and print, for example, "gb (weighting 1.00)".No public name is removed or renamed.
Before you upgrade
51D_and those named in the included list. To keep the old behaviour, callsetShareAllQueryStringParameters(false)on the usage sharing builder, or setShareAllQueryStringParametersto false in the configuration file.51D_is no longer shared, and nor is the session cookie. There is no setting to share them.EvidenceKeyFilterWhitelistbuilt with a list or a map and no comparator now matchesQuery.UserAgenttoquery.useragent. To keep matching with case, pass a case-sensitive comparator to the constructor that takes one.setMinify(false)on the builder, or setMinifyto false in the configuration file.Content-MD5header no longer fails the update withAUTO_UPDATE_ERR_MD5_VALIDATION_FAILED, and is logged as a warning instead. A data file with no temporary path givesAUTO_UPDATE_TEMP_PATH_NOT_SET, and an unexpected error givesAUTO_UPDATE_UNKNOWN_ERROR, where both gaveAUTO_UPDATE_REFRESH_FAILED. Code that switches onAutoUpdateStatusshould handle the two new values. An integrity check failure is now logged at Error rather than Warning.Tests
Each item has a test commit followed by its fix, and each test commit was run on a runner before its fix.
Red, run 43772345068. The run's commit, c2ec1b9, is the head with only a temporary workflow added. Each of its eleven jobs checks out one test commit, builds the module it tests with the modules it needs on Java 17, runs only that commit's new or changed tests, and fails.
ShareUsageOptionsTestsdefaults_ShareEveryQueryStringParameterexpected true but was falseEvidenceKeyFilterShareUsageTestsDataUpdateServiceCheckFailureTestsDataUpdateServiceLoggingTestsEvidenceKeyFilterWhitelistCaseTestsJavaScriptUrlTestsParameters_QueryPrefixIsCaseInsensitive, "Query evidence with prefix 'Query.' was dropped"JavaScriptMinifyTests,MinifyOverheadTestsJavaScriptMinifierandsetMinifydo not existAspectEngineBuilderPropertiesTestsFlowDataTestsPropertyHelperWeightingTestsweightedValuesandasWeightedStringdo not existWeightingShownTestThe
WeightingShownTestjob removesPropertyHelperWeightingTestsbefore it builds, because that test does not compile at this commit and the console module's build needs the shared module.Green, the full build below, in which every one of these tests ran on the head.
Build
Run 43771872937 built 0014283 merged with
rc(e2472ee) on nine configurations, Ubuntu x64 with Java 8, 11, 17 and 21, Ubuntu arm64 with Java 17, and Windows with Java 8, 11, 17 and 21, and every job passed. Each of its tenSetting 'PullRequestSha'lines names 0014283, and its Configure job printed "Left out 6 macOS entries". Every test class named in the red table ran and passed in all nine jobs,MinifyOverheadTestsamong them.macOS was left out of this draft's full build under the 9 November decision. The rc head is built with macOS before it merges into main.
Merging with #186
#186 also brings the data update Errors table rows into line, and a trial merge of the two heads conflicts in
DataUpdateServiceDefault.java,DataUpdateServiceLoggingTests.javaandDataUpdateServiceTests.java. Both also addAUTO_UPDATE_TEMP_PATH_NOT_SETtoAutoUpdateStatus, this draft at the end and #186 in the middle, and git mergesDataUpdateService.javawithout a conflict, which would declare the constant twice and stop the build. Whichever of the two merges second must first mergerc, after the other has merged, and keep one declaration, at the end of the enum.What remains
AUTO_UPDATE_UNKNOWN_ERROR, as data-updates.md lines 477 to 479 asks.MinifyOverheadTeststhree times on each kind of runner the release builds on, with Java 17, and printed the medians the test records. The slowest runner is macOS Intel (macos-15-intel), at 0.30 to 0.36 ms for one minification and 0.35 to 0.49 ms of extra time for each Process call, so each limit is about ten times the slowest runner's figure. The others gave 0.13 to 0.18 ms and 0.13 to 0.30 ms (Ubuntu x64 and arm64, Windows, macOS on Apple silicon).Ready for review, head 0014283, build https://github.com/51Degrees/51degrees-java/actions/runs/43771872937