[SYSTEMDS-3959] Report an error when a federated worker cannot bind its port - #2586
gaturchenko wants to merge 1 commit into
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #2586 +/- ##
============================================
- Coverage 71.44% 71.38% -0.07%
- Complexity 50437 50550 +113
============================================
Files 1629 1633 +4
Lines 195513 196270 +757
Branches 38118 38208 +90
============================================
+ Hits 139686 140101 +415
- Misses 44867 45144 +277
- Partials 10960 11025 +65 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Waiting for the review by @ywcb00 |
0612897 to
1e8037d
Compare
| # verify that $1 is a port that a server can be bound to, otherwise abort with an error. | ||
| # $2 names the server the port is meant for. | ||
| function checkPort { | ||
| local port=$1 | ||
| local target=$2 | ||
| local re='^[0-9]+$' | ||
| if ! [[ $port =~ $re ]] ; then | ||
| echo "error: Port '$port' for the $target is not a number" | ||
| printUsage | ||
| exit 1 | ||
| fi | ||
| # drop leading zeros, so that the length check below is not fooled by e.g. 0000080505 | ||
| local num=$port | ||
| while [[ ${#num} -gt 1 && $num == 0* ]] ; do num=${num#0} ; done | ||
| # more than 5 digits is out of range by definition, and comparing it would silently | ||
| # overflow the 64 bit integers of the shell for very long inputs | ||
| if [ ${#num} -gt 5 ] || [ "$num" -lt 1 ] || [ "$num" -gt 65535 ] ; then | ||
| echo "error: Port $port for the $target is out of range, expected a port in [1, 65535]" | ||
| printUsage | ||
| exit 1 | ||
| fi | ||
| if [ "$num" -lt 1024 ] ; then | ||
| echo "warning: Port $port for the $target is a reserved system port, binding it requires elevated privileges" | ||
| fi | ||
| } | ||
|
|
There was a problem hiding this comment.
Only criticism. I am strongly against adding more logic to the systemds file. Can we not just have the error message and verification inside Java. This gives a single point of failure, rather than splitting the logic in two.
While there was some logic already verifying basic number compatibility, i think that should also just be cleaned up and moved into the java PortUtils you have defined.
The command
systemds WORKER 80505terminated silently with exit code 0. There are 2 causes:-w <port>, neither inbin/systemdsnor when parsing the optionFederatedWorker.run()captured every exception in a catch-all and logged onlye.getMessage(), which is empty or ambigous for a bind failure. The same held for occupied and reserved ports, and for the-fedMonitoringbackendThe following changes were implemented:
PortUtils(new): the shared range constants and validation, as well as explicit port and cause naming in case of errors;DMLOptions:-wand-fedMonitoringreject non-integers and ports outside[1, 65535]at parse time, naming the incorrect option. A missing (optional)-wargument now falls back to the default port instead of throwingNumberFormatException.FederatedWorker/FederatedMonitoringServer: a failed bind is logged throughexplainBindFailureand re-thrown asDMLRuntimeException. The monitoring backend now separatesInterruptedException(a normal shutdown) from a real failure.bin/systemds: adds a range check, a warning for reserved ports, andexit 1. Previously, an error was printed and the JVM was launched regardless.FederatedWorkerPortTest(new): 14 tests covering the range predicates, option parsing, and the two worker paths (out-of-range and occupied port).Resulting messages: