Skip to content

Update fetchmail.pl to current PostfixAdmin - #104

Open
AndrewSav wants to merge 1 commit into
mailserver2:masterfrom
AndrewSav:fetchmail-upstream
Open

AndrewSav wants to merge 1 commit into
mailserver2:masterfrom
AndrewSav:fetchmail-upstream

Conversation

@AndrewSav

Copy link
Copy Markdown
Collaborator

Description

What fetchmail is and what this image uses it for

fetchmail is a mail retrieval client. It logs in to a mailbox on another server over POP3 or IMAP, downloads the messages, and hands them to the local mail system for delivery. This image runs it to pull mail from mailboxes elsewhere into mailboxes on this server at regular intervals (README feature list: "fetch emails from external IMAP/POP3 server into local mailbox"). For example, a user who has moved here can keep receiving the mail that still arrives at their old provider's mailbox.

What to fetch is not configured in this image. The jobs are defined in PostfixAdmin, the web administration interface, which runs as a separate container (the mailserver2/postfixadmin image) and writes to the same database this image reads its domains and mailboxes from. PostfixAdmin only stores the jobs. This image does the fetching. The feature is off by default, and works like this:

  1. On PostfixAdmin's "Fetch Email" page, an administrator adds a job: the remote server, protocol, port, user name and password, folder, options such as keep and fetchall, how often to poll, and the local mailbox to deliver into. Each job is a row in the fetchmail table of the shared database.
  2. With ENABLE_FETCHMAIL=true, the image installs a cron entry that runs /usr/local/bin/fetchmail.pl as root every FETCHMAIL_INTERVAL minutes (default 10). PostfixAdmin publishes this script in its source tree, as ADDITIONS/fetchmail.pl, for whichever host runs fetchmail. This image keeps a copy with two local changes: it reads its database settings from /etc/postfixadmin/fetchmail.conf, which setup.sh fills in from DBDRIVER, DBHOST, DBPORT, DBNAME, DBUSER and DBPASS, and it adds DBPORT to the connection string.
  3. On each run, the script selects the active jobs whose poll interval has elapsed, writes a one-job fetchmail configuration for each, runs fetchmail on it, and stores the time of the run back in the job's row.
  4. fetchmail delivers what it downloads by SMTP to localhost, or through a command named in the job's mda field. On port 25, Postfix's reject_sender_login_mismatch refuses fetched mail whose sender is an address on this server, because the client hasn't logged in (553 ... not logged in). So setup.sh also adds an SMTP listener on 127.0.0.1:10025 with fewer restrictions when fetchmail is enabled (Fetchmail fails to deliver mails from a local sender hardware/mailserver#276). A job uses it with smtp localhost/10025 in its extra options. The listener is bound to the container's loopback interface and accepts mail only from loopback clients (mynetworks=127.0.0.0/8,[::1]/128, then reject). Docker forwards a published port to the container's network interface, not to its loopback, so publishing 10025 would not make it reachable from outside.
  • What is the current behavior (you can also link to an open issue here) ?

    The image's fetchmail.pl is upstream 483bb83 from August 2016 plus the two local changes. It misses every upstream fix since. For example, it puts the source password into fetchmail's configuration without escaping, and a test password with " and \ in it, pa"ss\word, fails to log in. It also ignores a job's port, and it has no dovecot keyword for the mda field.

  • What is the new behavior (if this is a feature change) ?

    fetchmail.pl is upstream's current script, a2ee929 (2026-11-12) on PostfixAdmin's master, plus the same two local changes. Upstream's changes to the file since 2016:

    This copy comes from a2ee929, which is on PostfixAdmin's master but is newer than their latest release at the time of writing, v4.0.6 (2026-11-06). v4.0.5 and v4.0.6 ship the same fetchmail.pl, which differs from this one only by the SQLite lines.

    Changes in the image to go with it:

    • setup.sh creates /var/lock/fetchmail instead of /var/run/fetchmail. Without it, the script logs Please create: /var/lock/fetchmail and exits.
    • The test schemas for MariaDB and PostgreSQL gain src_port, as PostfixAdmin's database upgrade 1843 adds it.

    Compatibility: the script now selects src_port, which PostfixAdmin first shipped in 3.3.0 (January 2021). On an older schema the query fails. I checked what happens then: the error goes to stderr, which the cron entry discards, the script exits 0, and syslog shows nothing. Fetch jobs stop without any message. The mailserver2/postfixadmin image is 4.0.5, so it has the column.

    Note: rspamd and Zeyple are attached to Postfix, rspamd as a milter and Zeyple as a content_filter. Mail that fetchmail delivers through port 10025 doesn't pass through either, because that listener sets no_milters and clears content_filter. Mail delivered with mda dovecot doesn't either, because it goes straight to Dovecot's LDA. This PR doesn't change that: the old script worked the same way with the LDA command written out in mda. (This is from the configuration; I haven't tested it.)

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • Test (adding missing tests or correcting existing ones)

Status

  • Ready

Todo List

  • Implementation
  • Tests

How has this been tested ?

Only the reverse suite enables fetchmail, so the new tests are there, on PostgreSQL. They run the script the way cron does, against a job that fetches john.doe@domain.tld's INBOX from this server over IMAP into sarah.connor@domain.tld.

  • "a job delivers all its mail through port 10025" fetches all 4 messages through smtp localhost/10025 with src_port 143, and checks the job's poll time was updated.
  • "mda dovecot delivers through dovecot's LDA" fetches the 4 messages with mda dovecot and checks that Dovecot's LDA logged saving each one.
  • "a source password with a double quote and a backslash in it works" adds a mailbox with the password pa"ss\word, fetches from it, and checks that the login succeeded and no Authorization failure was logged.
  • Against the old script, the second and third tests fail. The first passes, because 143 is fetchmail's default IMAP port anyway. Without /var/lock/fetchmail, all three fail.
  • MySQL, by hand in the default container: a job runs only once its poll interval has elapsed, src_port and DBPORT are used, and delivery through the LDA works. This was done with v4.0.5's script. On MySQL, this branch's script sends the same SQL.
  • All nine suites pass on this branch (on master 1d30a54), 361/361: default 139, reverse 88, ldap 66, ldap2 30, sieve 2, ecdsa 4, traefik_acmev1 11, traefik_acmev2 12, traefik_v3 9.

The image's copy was PostfixAdmin's ADDITIONS/fetchmail.pl from 2016,
plus two local changes: the config file path and the database port.
This takes upstream's script at a2ee929, the latest commit to touch it,
and keeps those two changes. a2ee929 is newer than PostfixAdmin 4.0.6;
4.0.5 and 4.0.6 ship the same script without its SQLite lines.
Upstream has since added escaping for " and \ in source passwords,
"dovecot" as an mda keyword, the src_port column, --pidfile in place of
-i (fetchmail's id file), a configurable lock directory, and SQLite
support. The SQLite path is unreachable here: fetchmail.conf only sets
mysql or Pg, and on those the SQL is unchanged.

- setup.sh creates /var/lock/fetchmail, upstream's lock directory. The
  script exits if it is missing.
- The test schemas gain src_port, as PostfixAdmin's upgrade 1843 adds
  it (PostfixAdmin 3.3.0 and later). The script now selects it.
- reverse.bats gains three tests that run the script as cron does:
  delivery through port 10025, mda dovecot, and a password with " and \.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant