Skip to content

esp32 network: Add PPP over a UART - #2461

Merged
bettio merged 1 commit into
atomvm:release-0.7from
iobee-io:esp32-ppp
Nov 6, 2026
Merged

bettio merged 1 commit into
atomvm:release-0.7from
iobee-io:esp32-ppp

Conversation

@gabrielmancini

Copy link
Copy Markdown

Adds PPP over a UART to the ESP32 network module, for boards with a cellular modem on a UART (LilyGO T-Internet-COM, T-PCIE and T-Vending with a SIM7600 Mini PCIe module, T-Call with a SIM800L).

network:start/1 accepts {ppp, [...]}, alone or next to sta/ap/eth: the UART number, TX and RX pins, speed and route_priority, and the got_ip and disconnected callbacks. The other end must already be ready for PPP: the application powers the modem and dials it on the same UART (ATD*99# answered CONNECT) before starting the network, so the driver stays modem-agnostic. The link negotiates its address and DNS servers, and its netif becomes the default once it is up. With its default route_priority of 20 it carries the traffic only when Ethernet and the STA are down. network:stop/0 closes the link with an LCP terminate, so the modem hangs up, and frees the UART.

PPP is opt-in with CONFIG_AVM_ENABLE_PPP, which needs CONFIG_LWIP_PPP_SUPPORT; otherwise a ppp entry returns {error, ppp_not_supported}.

Built on top of #2419 (it shares start_network with the Ethernet entry). The ESP32 build matrix gets a PPP build, and the network guide a section.

Tested on hardware, with an application that dials the modem and falls back to WiFi: a SIM7600 on a T-Internet-COM and a T-PCIE (ESP32) and on a T-Vending (ESP32-S3), and a SIM800L (2G) on a T-Call (ESP32). A TLS session over the link for 11 hours overnight (no LCP errors, same address), link drops and redials, network:stop/0 with the link up and after the modem hung up, and OTA updates of the application over 2G (479 KB, with resumed downloads). No QEMU test: the suite has no peer for a PPP link on a UART.

These changes are made under both the "Apache 2.0" and the "GNU Lesser General Public License 2.1 or later" license terms (dual license).

SPDX-License-Identifier: Apache-2.0 OR LGPL-2.1-or-later

@petermm

petermm commented Nov 1, 2026

Copy link
Copy Markdown
Contributor

Amp code with this promp:
review last commit, ask the oracle, write findings to PR_REVIEW.md in markdown, include code diffs for small fixes/suggestions

Not sure how hypothetical the teardown issue is, but the priority override seems nice to fix, your call:

Review: ESP32 PPP over a UART

Reviewed commit: 2474f11c2cf608e512f09f3fe349336c142fecee — esp32 network: Add PPP over a UART.

Recommendation

Request changes. The PPP teardown can free an interface while lwIP still owns an active
PPP session, and the default-route override defeats the documented failover behavior.

This review covers the commit against its first parent, including the C driver, Erlang
callbacks and configuration types, Kconfig, CI configuration, and network guide. Suggested
patches below are review proposals, not applied implementation changes.

An independent oracle review was requested and incorporated. In particular, the oracle
identified why an unconditional replacement with esp_netif_action_connected() would
introduce a delayed-event race; the routing suggestion below accounts for that concern.

Findings

1. [P1] Do not destroy PPP after an expired graceful-close deadline

Location: src/platforms/esp32/components/avm_builtins/network_driver.c:1785–1804

Trigger: Establish PPP, then stop the network while the peer is silent or has lost power
without completing LCP termination. Stopping while initial negotiation has stalled also
needs coverage.

esp_netif_action_stop() starts a graceful ppp_close(pcb, 0). The three-second wait is
not a guarantee that this close has finished. After the wait expires, stop_ppp() logs a
warning but still stops RX and destroys the netif and glue.

In the locally installed ESP-IDF v5.5.5:

  • components/esp_netif/lwip/esp_netif_lwip_ppp.c, esp_netif_stop_ppp(), calls
    ppp_close(..., 0).
  • components/lwip/lwip/src/include/netif/ppp/ppp_opts.h defines FSM_DEFTIMEOUT as
    6 seconds and FSM_DEFMAXTERMREQS as 2. terminate_layer() and fsm_timeout()
    in src/netif/ppp/fsm.c therefore allow roughly 12 seconds for an unacknowledged
    termination, longer than both PPP_STOP_MS and the Erlang stop deadline.
  • ppp_free() in src/netif/ppp/ppp.c returns ERR_CONN unless the session is in
    PPP_PHASE_DEAD. esp_netif_destroy_ppp() ignores that result and frees the enclosing
    context anyway; esp_netif_destroy_api() also frees the netif storage.

Consequently, a pending lwIP termination timer can still reference the session and its
freed netif/context. This is a memory-safety issue, not merely a delayed hangup or a harmless
warning.

Required fix: Keep the PPP session, netif, UART, and callback state alive until close is
confirmed. If network:stop/0 must remain bounded, implement an explicitly safe abort or
deferred teardown with ownership lasting through completion. Merely increasing
PPP_STOP_MS beyond the Erlang timeout, or returning from stop_ppp() while its caller
continues freeing ClientData, is not a complete fix. This lifecycle change is not safely
expressed as a one-line suggestion.

Regression check: Run an actual PPP peer, suppress its Terminate-Ack, stop the network,
and continue running past the lwIP retry deadline. Confirm no stale timer/callback access,
no leaked PPP PCB, and successful subsequent start/stop. Also test stopping during
negotiation and after an already-dead link.

2. [P1] Preserve automatic route selection instead of pinning PPP as default

Location: src/platforms/esp32/components/avm_builtins/network_driver.c:1008–1012

Trigger: Start STA or Ethernet together with PPP using the documented defaults, then
bring PPP up. Alternatively, bring STA/Ethernet up after PPP, or lose PPP while another
interface remains usable.

esp_netif_set_default_netif() is a persistent manual override, not a priority-aware
request to reconsider the default. In ESP-IDF v5.5.5,
esp_netif_update_default_netif_lwip() sets
s_is_last_default_esp_netif_overridden = true and ignores later automatic updates while
the selected netif remains registered. esp_netif_is_active() checks registration, not
whether the link is up.

Thus PPP with priority 20 displaces Ethernet with priority 128 or STA with priority 100.
Later IP/link events cannot restore the preferred interface, and a dead but undestroyed
PPP interface can remain the selected default. This contradicts the advertised fallback
behavior and can black-hole off-subnet traffic.

Suggested direction: Reevaluate routes without a permanent override. Simply deleting
the existing call is insufficient: the earlier address event can precede link-up, as the
commit itself notes. In the inspected SDK, the public connected action calls
esp_netif_up() and performs priority-aware ESP_NETIF_STARTED selection without DHCP
on PPP. However, it also forcibly marks the link up.

Oracle caveat: An old queued success event may run after lwIP has already disconnected
the session and marked its link down. Calling the connected action unconditionally would
resurrect that dead link's routing eligibility. A check made on the event-loop task alone
still races with teardown on the TCP/IP task.

The following untested candidate checks the live link and refreshes routing atomically
with respect to lwIP processing through the public esp_netif_tcpip_exec() API. It is a
small implementation direction, not a verified cross-version fix. Prefer a state-preserving
route refresh if a suitable supported API is available; the directly matching
esp_netif_update_default_netif() API is internal to ESP-IDF.

diff --git a/src/platforms/esp32/components/avm_builtins/network_driver.c b/src/platforms/esp32/components/avm_builtins/network_driver.c
--- a/src/platforms/esp32/components/avm_builtins/network_driver.c
+++ b/src/platforms/esp32/components/avm_builtins/network_driver.c
@@
+#ifdef HAVE_PPP
+static esp_err_t refresh_ppp_route(void *arg)
+{
+    esp_netif_t *netif = arg;
+    if (esp_netif_is_netif_up(netif)) {
+        esp_netif_action_connected(netif, NULL, 0, NULL);
+    }
+    return ESP_OK;
+}
+#endif
+
 static void event_handler(void *arg, esp_event_base_t event_base, int32_t event_id, void *event_data)
@@
         if (event_id == NETIF_PPP_ERRORNONE) {
             ESP_LOGI(TAG, "PPP connected.");
             if (data->ppp_glue != NULL) {
-                esp_netif_set_default_netif(data->ppp_glue->base.netif);
+                esp_netif_tcpip_exec(refresh_ppp_route, data->ppp_glue->base.netif);
             }

Validate API availability and behavior on the supported SDK versions, including IPv6 and
dual-stack configurations, and update the adjacent comment to describe priority-based
selection rather than unconditionally making PPP the default. This does not fix finding 1.

Regression check: Exercise PPP alone; STA/Ethernet before and after PPP; loss of the
higher-priority interface; loss of PPP; and a PPP priority above STA/Ethernet. Verify the
selected default and actual off-subnet packet delivery, not just got_ip callbacks. Delay
the event-loop task across a rapid PPP connect/disconnect to exercise stale success events.

Small documentation suggestion

Explain the UART ownership handoff after dialing

Location: doc/src/network-programming-guide.md:496–499

The guide tells applications to dial using the same UART before starting PPP, but does not
say to close the existing AtomVM UART port. start_ppp() always calls
uart_driver_install(), and ESP-IDF rejects an already-installed UART driver. The AtomVM
UART close path calls uart_driver_delete(), so an explicit handoff is necessary; leaving
the dialer's port open cannot work.

diff --git a/doc/src/network-programming-guide.md b/doc/src/network-programming-guide.md
--- a/doc/src/network-programming-guide.md
+++ b/doc/src/network-programming-guide.md
@@
 configuration. The other end must already be ready for PPP: typically a cellular modem that the
 application has powered on and dialed over the same UART (`ATD*99#` answered `CONNECT`) before
-calling `network:start/1`. The link negotiates its address and DNS servers. It can be used alone
+calling `network:start/1`. Close the dialing UART port with `uart:close/1` before starting PPP;
+the network driver installs and owns its own UART driver. Do not read from or write to that
+UART through another port while PPP is running.
+The link negotiates its address and DNS servers. It can be used alone
 or next to `sta`, `ap` and `eth`; with its default `route_priority` of 20 it carries the traffic

This documents the existing ownership requirement; it does not prove that a close/reopen
handoff preserves every modem's startup frames. That needs a real-modem check.

Validation and limits

  • Findings above are grounded in the changed code and direct inspection of the locally
    installed, supported ESP-IDF v5.5.5 sources at /Users/petermm/esp/esp-idf-v5.5.5.
  • No ESP32 firmware build, modem session, hardware teardown test, or runtime memory-safety
    reproduction was performed. The proposed diff has not been compiled or run.
  • The new CI job is compile coverage only; it does not exercise routing or close timing.
  • Only this Markdown report is being added; the reviewed implementation is unchanged.

network:start/1 takes {ppp, [{uart, N}, {tx, Pin}, {rx, Pin},
{speed, Baud}, {route_priority, P}]} and runs PPP on that UART with the
esp_netif PPP client, for a modem the application has dialed (it
answered CONNECT). Callbacks got_ip and disconnected.

lwIP sets the address before the link is up, so the address event finds
it down and the default netif is not updated: once the link is up the
driver runs esp_netif_action_connected on the tcpip thread, and the
interface with the highest route_priority that is up becomes the
default. Stopping closes the link (LCP terminate, so the modem hangs
up). If the peer does not answer within 3 s, network:stop/0 returns and
a task frees the link when lwIP reports it dead. Bytes in and out are
logged on close.

CONFIG_AVM_ENABLE_PPP, off by default, needs LWIP_PPP_SUPPORT and
selects LWIP_PPP_NOTIFY_PHASE_SUPPORT. The ESP32 build matrix gets a
PPP build, and the network guide a section.

Signed-off-by: Gabriel Mancini <gabriel.mancini@gmail.com>
@gabrielmancini

Copy link
Copy Markdown
Author

Thanks for the review, both points were real. Fixed in the last push:

Teardown. network:stop/0 still waits up to 3 s for the session to end. If it hasn't, a small task takes over the glue and frees it only after lwIP reports PPP_PHASE_DEAD. The dead handler is now registered per glue, not per driver, so it outlives the driver. In the worst case (no dead within 30 s) the glue is leaked, never freed early.

Default route. I dropped esp_netif_set_default_netif. On NETIF_PPP_ERRORNONE the driver now runs esp_netif_action_connected on the tcpip thread, once the link is up. That gives the IDF's priority selection the update it skipped: GOT_IP arrives before sifup, so the link is still down when it's posted. route_priority now decides.

The guide now says the application closes its UART before starting PPP. It also describes the default route rule and the background close. In CI, the size failure came from the PPP include entry merging into the libsodium + Ethernet job. PPP now builds in the esp32 v5.5.5 job.

Tested on a T-Call (SIM800L, GPRS), with a test app:

  • STA + PPP at route_priority 20: traffic leaves by the STA.
  • STA + PPP at route_priority 200: traffic leaves by PPP.
  • PPP up, then modem power cut, then network:stop/0: returns in 3.0 s, "finishing in the background", dead 9 s later, glue freed.
  • Dial and network:start/1 again: works, stop in 0.2 s.
  • Heap stable across the runs.

@github-advanced-security

Copy link
Copy Markdown

You are seeing this message because GitHub Code Scanning has recently been set up for this repository, or this pull request contains the workflow file for the Code Scanning tool.

What Enabling Code Scanning Means:

  • The 'Security' tab will display more code scanning analysis results (e.g., for the default branch).
  • Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results.
  • You will be able to see the analysis results for the pull request's branch on this overview once the scans have completed and the checks have passed.

For more information about GitHub Code Scanning, check out the documentation.

@petermm

petermm commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

Follow-up review — PPP over a UART

Reviewed head: d7b3038, compared with the previously reviewed 2474f11.

Thanks for the fixes and hardware testing. Both previous P1 findings are addressed: the netif is no longer unconditionally destroyed after the three-second deadline, and routing no longer pins PPP as the default. The UART handoff documentation is also fixed, and the PPP-enabled ESP32/v5.5.5 CI job passed.

An additional oracle review challenged the remaining lifetime concern and confirmed one conditional P2. This is not a regression introduced by the follow-up push; it also existed in the original PPP implementation.

[P2] Drain queued PPP input before destroying its netif

Location: src/platforms/esp32/components/avm_builtins/network_driver.c:1804–1815, particularly esp_netif_destroy(netif) at line 1814.

Trigger: CONFIG_LWIP_TCPIP_CORE_LOCKING=y, CONFIG_LWIP_TCPIP_CORE_LOCKING_INPUT=n, and UART input still queued when teardown begins. This is a supported SDK combination, but core locking is off by default.

Joining the RX task stops the producer; it does not drain input already submitted to lwIP. In ESP-IDF v5.5.5:

  1. pppos_input_tcpip_as_ram_pbuf() calls tcpip_inpkt(), which stores the raw lwIP netif pointer in a mailbox message when input core locking is disabled.
  2. With core locking enabled, esp_netif_destroy() executes its destruction function inline under the core mutex rather than posting behind pending input.
  3. Cleanup can therefore free the netif and PPP PCB before an earlier input message runs. pppos_input_sys() then immediately dereferences the freed inp->state.

The possible ordering is RX queues input → RX signals stopped → teardown acquires the core mutex and frees the netif → TCP/IP processes queued input and dereferences the freed netif. Reaching PPP's DEAD phase does not drain that input queue.

With core locking disabled, destruction is submitted behind previously queued input, so this particular interleaving does not apply.

SDK evidence: components/esp_netif/lwip/netif/ppp.c:15–27; components/lwip/lwip/src/api/tcpip.c:245–272,442–449,610–616; components/lwip/lwip/src/netif/ppp/pppos.c:444–450; and components/esp_netif/lwip/esp_netif_lwip.c:212–226,1012–1042.

Small suggested fix

After joining RX, enqueue a real TCP/IP mailbox barrier before destroying the netif. The existing stopped semaphore can be reused: the first take consumes RX completion, and the second waits for the callback. tcpip_callback() always posts a message on success, including with core locking enabled. Earlier RX submissions precede that callback in the FIFO mailbox.

Do not substitute esp_netif_tcpip_exec() or tcpip_callback_wait() here: both can execute inline under core locking.

The following is a review-only candidate against this PR head, not an applied, compiled, or runtime-tested fix:

diff --git a/src/platforms/esp32/components/avm_builtins/network_driver.c b/src/platforms/esp32/components/avm_builtins/network_driver.c
--- a/src/platforms/esp32/components/avm_builtins/network_driver.c
+++ b/src/platforms/esp32/components/avm_builtins/network_driver.c
@@ -67,6 +67,7 @@
 #include <freertos/FreeRTOS.h>
 #include <freertos/semphr.h>
 #include <freertos/task.h>
+#include <lwip/tcpip.h>
 #define HAVE_PPP 1
 #endif
 #pragma GCC diagnostic pop
@@ -1791,6 +1792,11 @@ error:
     return false;
 }
 
+static void ppp_rx_drained(void *arg)
+{
+    xSemaphoreGive((SemaphoreHandle_t) arg);
+}
+
 // Frees what start_ppp set up, once the session is dead (or never started): lwIP keeps timers
 // on a session until its dead phase, and the netif and its PPP control block must outlive them.
 static void free_ppp(struct PPPGlue *glue)
@@ -1801,8 +1807,15 @@ static void free_ppp(struct PPPGlue *glue)
     }
     if (glue->running) {
         ESP_LOGI(TAG, "PPP closed: %u bytes in, %u bytes out", (unsigned) glue->rx_bytes, (unsigned) glue->tx_bytes);
         glue->stop = true;
         xSemaphoreTake(glue->stopped, portMAX_DELAY);
+        // RX has stopped submitting input, but queued input still holds the netif.
+        // Use the mailbox even when esp_netif calls execute under the core lock.
+        if (tcpip_callback(ppp_rx_drained, glue->stopped) != ERR_OK) {
+            ESP_LOGE(TAG, "Failed to drain PPP input, left allocated");
+            return;
+        }
+        xSemaphoreTake(glue->stopped, portMAX_DELAY);
     }
     if (glue->stopped != NULL) {
         vSemaphoreDelete(glue->stopped);
@@ -1827,9 +1840,9 @@ static void ppp_teardown_task(void *args)
 {
     struct PPPGlue *glue = args;
     if (xSemaphoreTake(glue->died, pdMS_TO_TICKS(PPP_TEARDOWN_MS)) == pdTRUE) {
         free_ppp(glue);
-        ESP_LOGI(TAG, "PPP closed late, freed");
+        ESP_LOGI(TAG, "PPP late close teardown finished");
     } else {
         ESP_LOGE(TAG, "PPP did not close in %d ms, left allocated", PPP_TEARDOWN_MS);
     }

The failure return deliberately retains resources instead of freeing pointers potentially referenced by pending input. The final log adjustment avoids claiming the glue was freed if the barrier could not be allocated.

The unconditional barrier is redundant but correct when core locking is disabled or input is synchronous; avoiding configuration guards keeps the lifetime invariant explicit. It adds one message allocation per started-session teardown. Posting can block on mailbox space, and waiting assumes a functioning TCP/IP task. This must run from the current teardown tasks, not from the TCP/IP task or while holding its core mutex.

What otherwise looks correct

  • The per-glue DEAD handler outlives ClientData, and the binary semaphore retains a notification arriving between the foreground timeout and background-task startup.
  • Event dispatch and handler unregistration synchronize through the SDK event-loop mutex, so cleanup waits out active DEAD callbacks before deleting their semaphore or glue. No additional race was found in the new ownership transfer.
  • The route refresh checks live link state and runs the connected action within the TCP/IP execution context, preventing a delayed success event from resurrecting a link already taken down by lwIP.
  • Documentation and CI matrix changes address the earlier feedback.

Validation and limits

The oracle checked the relevant callback, queued-input, inline core-locking dispatch, PPP receive, and netif-dereference implementations in the SDK/lwIP revisions used by ESP-IDF v5.2.8, v5.3.6, v5.4.4, and v5.5.5. This is source-level evidence, not a hardware-reproduced crash. No new local firmware build or runtime test of the suggested patch was performed. The existing PPP-enabled CI job passed for the reviewed head; that does not exercise this teardown interleaving.

Suggested regression check: enable core locking with input locking disabled, keep UART traffic arriving around the DEAD notification, and stop/restart repeatedly with heap poisoning enabled. Arrange pending RX messages before cleanup and verify they finish before destruction. Repeat with core locking disabled as a control, and cover barrier-allocation failure without early free.

Recommendation: one targeted lifetime fix before approval; the two previous P1 findings can be considered addressed.

@bettio

bettio commented Nov 5, 2026

Copy link
Copy Markdown
Collaborator

This PR has been approved and merged locally, do not change.

@bettio
bettio merged commit a92ae69 into atomvm:release-0.7 Nov 6, 2026
232 of 234 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants