Skip to content

Add the Metrics component and the profiles for a property value, and read Device Detection results by position under one lock - #174

Draft
jwrosewell wants to merge 31 commits into
rcfrom
remote/java-11
Draft

jwrosewell wants to merge 31 commits into
rcfrom
remote/java-11

Conversation

@jwrosewell

@jwrosewell jwrosewell commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

The specification this brings the code into conformance with

  • See the commits below, each of which names the specification item it answers.

Breaking changes

None known. Any change of default behaviour is named in its Fix commit below.

Commits (21, on rc)

  • Test: only one thread at a time reads the native results of finished device data
  • Fix: hold one lock for every read of the native results in device data
  • Test: the engine adds metadata for each match metric, as the table gives it
  • Fix: give each match metric the Metrics component, and Method a single value
  • Test: the engine's components include Metrics
  • Fix: add the Metrics component to the engine's components
  • Test: the engine gives the ids of the profiles that hold a value of a property
  • Fix: give the profiles for a property value, and an Indexed Properties option
  • Test: every Device Detection engine populates Aspect Data
  • Test: on-premise and cloud engines return interface compatible data
  • Test: the engine reads the evidence keys again when the data is refreshed
  • Test: device data reads a property from the native results only when asked
  • Test: an engine configured with no property names is created with none
  • Test: the published Device Detection packages hold no tests or examples
  • Fix: keep the test helpers out of the published device-detection.shared jar
  • Test: a read of a property the engine was created with goes by position only
  • Fix: find a property by its position, not by a native lookup of its name
  • Test: the engine reads what changes only on refresh from native code once
  • Test: the cloud Device Detection engine leaves processing to the shared logic
  • Fix: let the shared Cloud Aspect Engine logic process for cloud Device Detection
  • Test: the hardware profile cloud engine processes through the shared logic

What remains

This draft is not ready for review yet. Its description, with the runs that show each test failing and then passing, is written once its full build has finished.

…device data

device-detection-specification/pipeline-elements/device-detection-on-premise.md
line 327 at 706de1f0: Aspect Data that reads its values from a ResultsHash
instance MUST let only one thread at a time read from its results.

Covers a match metric (DeviceId), a typed property read (IsMobile) and the
public getValues. DD-18.
device-detection-specification/pipeline-elements/device-detection-on-premise.md
line 327 at 706de1f0: Aspect Data that reads its values from a ResultsHash
instance MUST let only one thread at a time read from its results.

The lock in DeviceDataBaseOnPremise.tryGetValue covered only the typed
getters. The check that the results have a property, the public getValues,
the match metrics (which can walk every graph not yet walked) and close read
the results with no lock. DeviceDataHashDefault now holds one lock of its own
for each of them. DD-18.
…ves it

device-detection-specification/pipeline-elements/device-detection-on-premise.md
line 452 at 706de1f0: Metadata for these Properties MUST be added, as they will
not be included in the metadata exposed by the native code.

Checks the order, Category, Available With, Component, type and default of
each metric, over a native engine that is a mock with no properties. The
default of the list-valued UserAgents is left out, as how it is described is
an open question. DD-20.
…e value

device-detection-specification/pipeline-elements/device-detection-on-premise.md
line 452 at 706de1f0: Metadata for these Properties MUST be added, as they will
not be included in the metadata exposed by the native code.

The table gives Component = "Metrics" for every match metric, where each had
no component, and gives Method the type string, where it was marked as a
list. The engine now makes a Metrics component once, holding the seven
metrics, and each metric names it. The default of UserAgents is unchanged
while how a list-valued metric is described is an open question. DD-20.
device-detection-specification/pipeline-elements/device-detection-on-premise.md
line 464 at 706de1f0: Component = "Metrics" - This component MUST also be
added to the list of components returned by the Engine.

Over a native engine that is a mock with no components. DD-21.
device-detection-specification/pipeline-elements/device-detection-on-premise.md
line 464 at 706de1f0: Component = "Metrics" - This component MUST also be
added to the list of components returned by the Engine.

getComponents gave only the data file's components. It now gives the Metrics
component after them, so getComponent("Metrics") finds it too. The metadata
hash test no longer leaves out the metrics' component, which they now have,
and skips the default profile of Metrics, which has no profiles. DD-21.
… property

device-detection-specification/pipeline-elements/device-detection-on-premise.md
line 479 at 706de1f0: The Engine MUST be able to give the ids of the profiles
that hold a given value of a given Property, such as a TAC or a native model
name, through the native code's EngineHash::getProfileIdsForPropertyValue().

One test over a mock native engine. One over the free data file, skipped
where the file is not there, checks the ids are ascending and distinct, a
value in another case matches nothing, and indexing the property with the
Indexed Properties option gives the same ids. DD-22.
…s option

device-detection-specification/pipeline-elements/device-detection-on-premise.md
line 479 at 706de1f0: The Engine MUST be able to give the ids of the profiles
that hold a given value of a given Property, such as a TAC or a native model
name, through the native code's EngineHash::getProfileIdsForPropertyValue().

The native function was in the SWIG wrapper, but the engine kept its native
engine private and nothing called it. DeviceDetectionHashEngine now has
getProfileIdsForPropertyValue, which copies the native ids and releases the
native vector. The builder has setIndexedProperties, the Indexed Properties
option of the configuration table, so a lookup of an indexed property does
not read every profile. DD-22.
device-detection-specification/data-model.md line 10 at 706de1f0: All Engines
MUST populate data objects that implement Aspect Data as defined in the
Pipeline specification.

Checks the data type each engine is declared with, and the classes made to
hold each engine's data. DD-2.
device-detection-specification/data-model.md line 15 at 706de1f0: Each of
these Engines MUST return data that is interface compatible with the others.

Checks the on-premise and cloud data are both DeviceData with the same
accessors and types, and the cloud hardware lookup gives DeviceData. DD-3.
…shed

device-detection-specification/pipeline-elements/device-detection-on-premise.md
line 86 at 706de1f0: This MUST be done at start-up and any time the data is
refreshed.

Over a native engine that is a mock whose keys change between refreshes, so
a key only the new data has is accepted after the refresh. Also checks the
keys are stored, not read from the native code for each use. DD-11.
…asked

device-detection-specification/pipeline-elements/device-detection-on-premise.md
line 152 at 706de1f0: The Element Data MUST read a Property value from
ResultsHash only when that Property is requested, and MUST NOT copy Property
values out of ResultsHash while the Engine processes the Flow Data.

Processing, over a native engine that is a mock, makes no call to the results
it stores. That test makes native evidence, so it needs the native library,
though no data file. Storing the results reads nothing, and a request for
IsMobile reads IsMobile and nothing else. DD-13.
device-detection-specification/pipeline-elements/device-detection-on-premise.md
line 189 at 706de1f0: An implementation configured with no names MUST create
the Engine with no names, and MUST NOT pass the data set's names itself.

Reads the names held by the native configuration the engine was created
with, which the property indexes cannot show, as they are the same either
way. Needs the free data file, and is skipped where it is not there. DD-14.
device-detection-specification/packaging-and-structure.md line 16 at
706de1f0: Examples and tests SHOULD NOT be included in any published packages.

Lists what each published package's main jar holds and looks for test
helpers, tests and examples. DD-7.
…ed jar

device-detection-specification/packaging-and-structure.md line 16 at
706de1f0: Examples and tests SHOULD NOT be included in any published packages.

FileUtils and KeyUtils move from the shared module's main sources to its test
sources, so the tests of the other modules still reach them through the
test jar. The examples used them too, so the examples' DataFileHelper now
has the data file names and the search for the best data file, and their
KeyHelper reads and checks keys. DD-7.
…on only

device-detection-specification/pipeline-elements/device-detection-on-premise.md
line 205 at 706de1f0: Every read SHOULD use the position.

Reading IsMobile, at position 1, makes one call to the native results, the
read by position, and none by name. DD-15.
device-detection-specification/pipeline-elements/device-detection-on-premise.md
line 205 at 706de1f0: Every read SHOULD use the position.

The values were read by position, but each read first asked the native
results whether they held the property, by name. A property the engine was
created with has a position, so it is now taken as available without that
call, and a property the data set lacks still reads as having no value. A
name with no position, and results from more than one engine, still use the
name. DD-15.
…once

device-detection-specification/pipeline-elements/device-detection-on-premise.md
line 339 at 706de1f0: As such, steps SHOULD be taken to minimize this as much as
possible.

The last of those steps stores in the engine the values that change only on
a data refresh. Counts the calls to a mock native engine for the property
metadata and for the evidence keys, before and after refreshes. DD-19.
…ed logic

device-detection-specification/pipeline-elements/device-detection-cloud.md
line 41 at 706de1f0: The majority of the processing SHOULD be handled by the
shared Cloud Aspect Engine logic.

The engine should give only the parsing of its part of the response. The
shared logic passes over an empty response, whose error the Cloud Request
Engine has reported, and refuses a Pipeline where that engine has not run.
DD-9.
…e Detection

device-detection-specification/pipeline-elements/device-detection-cloud.md
line 41 at 706de1f0: The majority of the processing SHOULD be handled by the
shared Cloud Aspect Engine logic.

DeviceDetectionCloudEngine replaced the shared processEngine with its own, so
an empty response, after an error the Cloud Request Engine had reported,
threw a JSON error, and a Pipeline where that engine had not run was not
refused. The engine now gives only processCloudEngine, which parses its part
of the response, and the shared processEngine does the rest. Two test files
that called the engine's own processEngine now go through the shared one, or
call processCloudEngine with the response. DD-9.
…logic

device-detection-specification/pipeline-elements/hardware-profile-lookup-cloud.md
line 56 at 706de1f0: The majority of the processing SHOULD be handled by the
shared Cloud Aspect Engine logic.

The engine gives no processing of its own. Each profile in its part of a
canned response becomes DeviceData with the values and no-value reasons the
response gives, and an empty response gives no profiles. DD-28.
@jwrosewell jwrosewell added the remote-conformance Specification conformance work from the remote packs label Nov 15, 2026
rc already moved the example tests and the example data file helper off the
packages' test helpers, in #175, so the merge keeps rc's version of those
and of the TacCloud key check. The two engine files keep this draft's
version, which adds the lock around every read of the native results and
only takes the position shortcut with one set of results. The device
detection cloud request failed test processes through the engine, as the
IP Intelligence one does, now that the engine leaves processing to the
shared logic.
@jwrosewell jwrosewell changed the title Specification conformance, 51degrees-java, section 11 Add the Metrics component and the profiles for a property value, and read Device Detection results by position under one lock Nov 16, 2026
# Conflicts:
#	device-detection/device-detection.cloud/src/main/java/fiftyone/devicedetection/cloud/flowelements/DeviceDetectionCloudEngine.java
…ts, now the engine leaves processing to the shared logic
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

remote-conformance Specification conformance work from the remote packs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant