refresh-policy-routes@<if>.timer periodically removes custom/manually configured static routes on secondary ENI

[neichmann]

Environment

• Amazon Linux 2023, EC2 instance with two ENIs (ens5 primary, ens6 secondary)
• amazon-ec2-net-utils upgraded 2.7.1 → 2.7.6 via AMI update
• Network backend: systemd-networkd (NetworkManager not installed)

Expected behavior

Per the project README ("Usage" section):

• "Custom configuration should be respected."

Actual behavior

A static route added via ip route add dev ens6, and additionally persisted as a proper drop-in at /etc/systemd/network/70-ens6.network.d/*.conf with a [Route] section (i.e. using the documented override mechanism), is silently removed shortly after being added even though it is declared configuration, not just an imperative runtime change.

Root cause appears to be: refresh-policy-routes@ens6.service runs periodically (not just at boot/DHCP-renewal) via an active refresh-policy-routes@ens6.timer (~60–90s interval), and each run appears to trigger a networkctl reload that removes the custom route regardless of whether it was added imperatively or declared via the documented /etc/systemd/network drop-in mechanism.

Evidence

Jul 31 12:36:07 refresh-policy-routes@ens6.service: Starting...
Jul 31 12:37:29 refresh-policy-routes@ens6.service: Starting...
Jul 31 12:39:07 refresh-policy-routes@ens6.service: Starting...
Jul 31 12:40:45 refresh-policy-routes@ens6.service: Starting...

$ systemctl list-timers --all | grep policy-routes
Fri ... 12:43:10 CEST  55s left   ... refresh-policy-routes@ens6.timer refresh-policy-routes@ens6.service
Fri ... 12:43:12 CEST  57s left   ... refresh-policy-routes@ens5.timer refresh-policy-routes@ens5.service

Reproduction steps

1. On an AL2023 instance with a secondary ENI, add a custom static route: ip route add dev ens6, and additionally persist it as /etc/systemd/network/70-ens6.network.d/99-custom.conf with a [Route] section.
2. Wait for the next refresh-policy-routes@ens6.timer cycle (≤ ~90s).
3. Observe the route has been removed from ip route list, even though it does not conflict with any VPC-assigned prefix, secondary IP, or policy-routing rule for that ENI, and was declared via the documented override mechanism.

Why we believe this is a bug, not intended behavior

• 2.7.1 → 2.7.6 are patch releases; none of the changelogs describe an intentional behavior change around discarding externally-added or drop-in-declared routes.

Current workaround

# /etc/systemd/system/refresh-policy-routes@ens6.service.d/99-custom-routes.conf
[Service]
ExecStartPost=/path/to/our-route-reapply-script.sh

Appreciate your support!! :-)

[joeysk2012]

Can you post the full logs of when you think the exact moment this happens?