Skip to content

Release candidate - #36

Merged
jwrosewell merged 39 commits into
mainfrom
rc
Nov 16, 2026
Merged

jwrosewell merged 39 commits into
mainfrom
rc

Conversation

@jwrosewell

@jwrosewell jwrosewell commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

This test suite has no specification requirements to report.

Before you upgrade

From this release the suite reads the demo's resource key from _51DEGREES_RESOURCE_KEY, then _51DEGREES_RESOURCE_KEY_51DID, then the older 51DEGREES_RESOURCE_KEY, and the cloud endpoint from _51DEGREES_CLOUD_ENDPOINT, then the older 51DEGREES_CLOUD_ENDPOINT (#35). A value taken from an older name is still used, and is reported once as deprecated. A job that sets both an older name and a current name, meaning different keys, now gets the current name's key, so such a job should set the _51DEGREES_ names to the key and endpoint it means before taking this release.

What is in

Nine pull requests since the last release, #27, which merged rc into main at 15:06 UTC on 11 November 2026.

  1. #28, "Start Chrome and Firefox once before the first test that uses them". A browser's first start in a run can take far longer than the starts after it, and with Selenium's 60 second limit a slow first start could fail a test that had nothing wrong with it. BrowserDrivers.StartOnce makes that first start, with 300 seconds for each command, before the first test that asks for the browser.
  2. #34, "Give every Chrome and Firefox session the first start's time limit". Every later local Chrome and Firefox session gets the same 300 seconds, because a later start can run past 60 seconds too.
  3. #29, "Wait for the shared store read in the shared choice test". The shared choice test, and the common path and consent platform tests, waited for a request to finish but read its body before the page had recorded it, which could fail them against a cloud reached over the internet. They now wait for the body, and say the status and any error where there is none.
  4. #32, "Start the .NET demo with the _51DEGREES_ names as well as the older ones". The .NET demo is started with _51DEGREES_RESOURCE_KEY and _51DEGREES_CLOUD_ENDPOINT as well as the older names, so the key the suite hands it is the one it uses.
  5. #35, "Read the _51DEGREES_ names for the demo's key and endpoint, before the older ones", as described under "Before you upgrade".
  6. #33, "Merge main into rc". It changes no file. It makes the two squashed release commits on main part of rc's history, so rc is ahead of main and none behind, and this release merges without conflict.
  7. #37, "Remote session time limit, Chromium arguments in one place and quiet driver services, from FIX: Harden WebDriver creation so browser tests stop timing out #7". Three parts of James Rogers's FIX: Harden WebDriver creation so browser tests stop timing out #7. Remote Selenium sessions get the same 300 second limit for each command as local Chrome and Firefox sessions. CreateChrome gives --no-sandbox and --disable-dev-shm-usage in one place, in place of the copies tests added by hand. The Chrome and Firefox driver services print no start up banner, open no console window and write no verbose log. The 51Did harness, which starts its own browsers, gets all three as well.
  8. #38, "Say which browsers the README's browser paragraph describes". The paragraph now says it describes Helpers/BrowserDrivers.cs, and that the 51Did harness starts its own Chrome and Firefox with the same time limit, arguments and quiet drivers, but leaves Selenium Manager to find or fetch the driver and makes no first start. No code changes.
  9. #40, "Edge hardened and back on, failing grid sessions that fail, and one preparation for every driver, from FIX: Harden WebDriver creation so browser tests stop timing out #7". The rest of James Rogers's FIX: Harden WebDriver creation so browser tests stop timing out #7. BrowserDrivers.Prepare runs every browser headless and accepting insecure certificates and gives Chrome and Edge the two Chromium arguments, in place of the copies in the tests. Edge gets a quiet driver service and the 300 second limit, and its tests are switched back on. The Chrome, Firefox and Edge test classes fail, rather than report inconclusive, when a Selenium grid cannot hand out a browser.

Pull requests still open, not in this release

Closed without merging

Issues this release closes

Closes #31.

Issues still open

  • #12, "A page served through the test relay reports "fod is not defined" against a cloud reached over the internet". The cause is not yet found.
  • #26, "Remove the old variable names the harness sets for the .NET example". This waits until the deprecated names may go.
  • #9, "Run the browser suite from the repositories it tests, not only from the cloud".

One test is still skipped with [Ignore], being LocationClientSideOverrideTests.JavaScript_ClientSideOverrides (ClientSideOverrides/LocationClientSideOverrideTests.cs line 83), whose reason reads "--headless doesn't work". The Edge tests are back in use, and on ARM64 Linux, where Microsoft ships no Edge, they skip with that reason.

Build

Run 43898770539, on head 68691f9, passed. build-and-test ran 72 tests, of which 57 passed and 15 skipped, being the tests that need a browser, the cloud or keys, which that job leaves out. The browser job passed all 10 on ubuntu-latest, both Edge tests included, and 8 of 10 on each of ubuntu-22.04-arm and ubuntu-24.04-arm, where the two Edge tests skip because Microsoft ships no Edge for ARM64 Linux. The Only rc into main check passed (run 43898771384).

Version

This repository is not published as a package. No version change, and none of the 39 commits carries a version directive.

ben51degrees and others added 28 commits November 3, 2026 18:43
device-detection-dotnet-examples has moved into 51degrees-dotnet, under
examples/device-detection. Both dotnet demos, pmp-web and GettingStarted-Web,
are found there as siblings, and the docs name the monorepo.
* Find the Node example in 51degrees-node

* Copy the monorepo packages in when installing the Node example
* Let Java use the default JSON endpoint and still accept its old one

* Say once which JSON path each language uses
…d ones (#25)

* Test that the .NET example is given the underscore variable names

The .NET cloud web example reads the resource key from
_51DEGREES_RESOURCE_KEY and the cloud endpoint from
_51DEGREES_CLOUD_ENDPOINT, with no other name, once its rename is
released.

* Give the .NET example the _51DEGREES_ variable names

The .NET cloud web example now reads the resource key from
_51DEGREES_RESOURCE_KEY and the endpoint from _51DEGREES_CLOUD_ENDPOINT,
with no other name. Released versions of it read SUPER_RESOURCE_KEY and
FIFTYONE_CLOUD_ENDPOINT, so the harness sets both pairs until every
language has released with the new names.
Each commit main holds that rc does not is the squashed merge of an rc into main release whose content rc already holds, so this merge keeps rc's tree unchanged and only records main as a parent. Without it, merging main into a later release head conflicts in every file rc has changed since that release.
Each test starts three Chrome or Firefox sessions through BrowserDrivers,
writes how long each start took, and checks that Selenium gives each
session's commands at least FirstStartTimeoutSeconds. Every session after
a browser's first start is made with the driver's 60 second default, which
a later start on a busy runner has run past, so these fail until every
session is given the longer limit (#31).
Every session after a browser's first start was made with the driver's
default command limit of 60 seconds, and a later Chrome start on a busy
runner ran past it. CreateChrome and CreateFirefox now pass
SessionTimeoutSeconds, which is FirstStartTimeoutSeconds, to every
session, on the driver the machine provides and on the one Selenium
Manager fetches. The no-timeout constructors used before are the same
constructors with the 60 second default, so nothing else changes.

Edge keeps the default, because headless Edge hangs on the Ubuntu runners
and every Edge test is ignored, so no Edge start has been measured and a
hang would hold each command for five minutes. Sessions on a Selenium grid
are left as they were. Refs #31.
Start Chrome and Firefox once before the first test that uses them
Give every Chrome and Firefox session the first start's time limit
The page's recorder marks a fetch done when its headers arrive and records
the body afterwards, so waiting for done alone could read an empty answer.
The read wait now ends on the body, a failed request or a status other than
200, and the assertion reports the status and any error.

PMP shows the floating button once it has parsed its own copy of the same
answer, so the bubble check is now a wait rather than a single look.

Harness.Until takes the failure message as a function as well as a string,
so the requests and console it reports are those at the moment the wait
gives up. The shared choice test builds every message that way.
… name

The harness's comments, its not-configured message and both READMEs still
said the demo is handed only the older names. They now say it is given the
key and the endpoint under each name it reads, which for the dotnet demo is
the _51DEGREES_ names and the older ones.

The comment on the older names says older versions read only those, as the
latest release reads the _51DEGREES_ names first. The test now asserts the
older endpoint name as well, so it checks all four names it sets.
…e older ones

The suite read the demo's resource key from 51DEGREES_RESOURCE_KEY and then
_51DEGREES_RESOURCE_KEY_51DID, and its cloud endpoint only from
51DEGREES_CLOUD_ENDPOINT. A job that set only _51DEGREES_RESOURCE_KEY and
_51DEGREES_CLOUD_ENDPOINT therefore had every browser test skip as not
configured, and still passed.

The key is now read from _51DEGREES_RESOURCE_KEY, then the CI name
_51DEGREES_RESOURCE_KEY_51DID, then the older 51DEGREES_RESOURCE_KEY. The
endpoint is read from _51DEGREES_CLOUD_ENDPOINT, then the older
51DEGREES_CLOUD_ENDPOINT. Every current name comes before any older one,
which is the order the .NET demo reads its own names in, and a value taken
from an older name is reported once on the error stream with a sentence
naming the name that replaces it.

RequireFirst had no other caller, so RequireCurrentOrOld replaces it. The
settings tests cover each order and the deprecation sentence, and the
harness messages, comments and both READMEs name the current names.
The common path and the consent platform tests asserted on the response
body of a request copied out of the page earlier. The recorder marks a
fetch done when its headers arrive and records the body afterwards, and a
copy is never refreshed, so either test could read an empty body against a
remote cloud. Visitor.ResponseOf reads the request again and waits for its
body, a failure or a status other than 200, and both tests use it.
… build Visitor's wait messages at failure

Visitor.ResponseOf returned only the body, so where the wait ended on an
error or a status other than 200 the caller's message left both out. It is
now Visitor.Settled, returning the request read again, and
RecordedRequest.Outcome gives the status and any error, which the common
path, consent platform and shared choice assertions report.

Go, WaitForPmp, WaitForCard, WaitForClientRounds and WaitForClientObject
now pass Harness.Until a function for their message, so the console and
what was served are read when the wait gives up.
…urrent names start with an underscore

NoEndpointNameSet_TheFailureNamesBoth pins the message when neither endpoint
name is set, so the category's README now counts seven settings tests. The
TestConfig comment said every name starts with an underscore, which the two
older ones do not, and the Demo.cs summary line is wrapped near 80 columns.
Wait for the shared store read in the shared choice test
Start the .NET demo with the _51DEGREES_ names as well as the older ones
Read the _51DEGREES_ names for the demo's key and endpoint, before the older ones
@jwrosewell
jwrosewell marked this pull request as ready for review November 16, 2026 11:56
…sion limit

DriverSettingsTests checks that BrowserDrivers adds --no-sandbox and
--disable-dev-shm-usage to Chrome's options once each, keeping the caller's
own, that CreateChrome gives them, that the Chrome and Firefox driver
services are quiet, and that a remote session gets the session time limit.
The remote test answers the session's two requests itself, so it needs no
browser. The members are reached by name, so these fail here rather than
stopping the build.
…ne place, and quiet driver services

From #7, by James Rogers.

- CreateRemote creates every remote session with SessionTimeoutSeconds for
  each command, as local Chrome and Firefox sessions already have, in place
  of Selenium's 60 seconds.
- CreateChrome adds --no-sandbox and --disable-dev-shm-usage through
  AddChromeArguments, skipping any the caller already set. The copies four
  tests added by hand go, and the remote path no longer adds Chrome's twice.
- ChromeService and FirefoxService make every Chrome and Firefox driver
  service print no start up banner and open no console window, and the
  Chrome one write no verbose or appended log.

The Browser51Did harness keeps its own two arguments, because it creates
its driver directly rather than through CreateChrome. Edge is unchanged.
The harness starts its browsers itself rather than through CreateChrome,
and on a grid it made its own RemoteWebDriver with Selenium's 60 seconds.
The test points SELENIUM_URL at the stand-in remote for its own duration
and checks the harness's remote Chrome and Firefox sessions. It fails
here with 1:00 against the expected 5:00.
…nd Chrome's arguments

Harness.Start made its own RemoteWebDriver with Selenium's 60 seconds, and
its local ChromeDriver and FirefoxDriver with the default services and the
same 60 seconds. It now creates remote sessions through
BrowserDrivers.CreateRemote, and local ones with BrowserDrivers.ChromeService
and FirefoxService and SessionTimeoutSeconds. NewChrome gives Chrome's two
arguments through BrowserDrivers.AddChromeArguments in place of its own
copy, so every Chrome in the suite gets them from one place.

HarnessLocalSessions_StartWithTheSessionLimit starts the harness's local
Chrome and Firefox on every runner, which nothing else in this repository's
CI does, as the 51Did tests skip without a cloud.

The ChromeArguments summary says what the two arguments do, and the remote
helper's comment holds for both of its callers.
Remote session time limit, Chromium arguments in one place and quiet driver services, from #7
The paragraph described BrowserDrivers as if it started every browser, but
the 51Did harness starts its own Chrome and Firefox. It never looks for a
driver the machine provides and never makes a first start. The paragraph
now names BrowserDrivers, and one sentence says what the harness shares
and what it does differently.
Say which browsers the README's browser paragraph describes
… and grid failures that fail

DriverSettingsTests checks that BrowserDrivers.Prepare gives Chrome and
Edge options headless, insecure certificates accepted and both Chromium
arguments, each once, keeps a headless argument the caller chose, and
gives Firefox options insecure certificates accepted. It checks that the
Edge driver service is as quiet as Chrome's, that a local Edge session has
the Chromium arguments and the session limit, and that the Chrome, Firefox
and Edge browser test classes fail, rather than report inconclusive, when
SELENIUM_URL names a Selenium that cannot be reached. Here the last fails
with AssertInconclusiveException and the others find no Prepare or
EdgeService.
…hand out a browser, and prepare every driver in one place

The rest of #7, by James Rogers, apart from WebDriverFactory.cs and
--disable-gpu.

- BrowserDrivers.Prepare runs every browser headless and accepting insecure
  certificates, and gives Chrome and Edge --no-sandbox and
  --disable-dev-shm-usage, each once, keeping a headless argument the caller
  chose. CreateChrome, CreateFirefox, CreateEdge and the 51Did harness call
  it, and the copies of those settings in the tests go.
- CreateEdge gives Edge a quiet driver service and SessionTimeoutSeconds
  for each command, and the remote helper adds Edge's two arguments only
  where they are missing.
- EdgeTests, CacheTests.JavaScript_BrowserCache_Edge and
  DriverSmokeTests.Edge_RunsThePage are no longer ignored. On ARM64 Linux,
  where Microsoft ships no Edge, they still skip with CreateEdge's reason.
- ChromeTests, GeckoTests and EdgeTests report inconclusive only when no
  SELENIUM_URL is set, so a grid that cannot hand out a browser fails the
  class rather than hiding.
JavaScript_BrowserCache_Edge lost its Ignore without a catch around
CreateEdge, so on ARM64 Linux, where Microsoft ships no Edge, it would
fail on CreateEdge's WebDriverException rather than skip. It now reports
inconclusive with that reason there, as Edge_RunsThePage does.
Edge hardened and back on, failing grid sessions that fail, and one preparation for every driver, from #7
@jwrosewell
jwrosewell merged commit 1dbb087 into main Nov 16, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Every browser session after the first keeps Selenium's 60 second command limit

4 participants