Skip to content

Mailchimp Plugin for Kestra - #151

Closed
23abdul23 wants to merge 14 commits into
kestra-io:mainfrom
23abdul23:feat/mailchimp
Closed

23abdul23 wants to merge 14 commits into
kestra-io:mainfrom
23abdul23:feat/mailchimp

Conversation

@23abdul23

Copy link
Copy Markdown

What changes are being made and why?

Adds a Mailchimp plugin (io.kestra.plugin.mailchimp) so audience and campaign steps can live in Kestra flows instead of shell or Python tasks calling the Marketing API.

Tasks

  • account.Ping
  • audiences.ListAudiences, ListMembers, UpsertMember, BatchSubscribe, UpdateMemberTags
  • campaigns.ListCampaigns, SendCampaign
  • reports.GetCampaignReport, ListEmailActivity

Triggers

  • audiences.NewSubscriberTrigger, audiences.MemberStatusChangeTrigger, campaigns.CampaignSentTrigger

Authentication is an API key (data center derived from the key suffix unless server is set) or an OAuth access token.

Deviations from the usual plugin shape

  • Uses the JDK java.net.http.HttpClient instead of Kestra's HttpClient: Kestra's client replays POST on 429/503 and that cannot be turned off, which is unsafe for SendCampaign and BatchSubscribe. Retry (GET and idempotent calls only, honoring Retry-After) lives in MailchimpClient.
  • Ping sits in an account subpackage so every subgroup has its own icon and category.
  • Polling triggers read one page (1000 rows) per poll and keep a cursor (newest timestamp plus ids seen at it) in the namespace KV Store; a larger backlog drains over several polls.

How the changes have been QAed?

Unit tests (./gradlew build, JDK 23, no Mailchimp credentials): 176 tests, 163 passed, 0 failed. A local fake Mailchimp server covers pagination, retries, error mapping and trigger cursors.

Live tests against a throwaway account (./gradlew test --tests '*integration*'): 13 integration tests passed, 0 failed, 0 skipped. They create kestra-it-<uuid> members as unsubscribed in one audience only, archive them afterwards, and never send a campaign (a guard test fails if the integration sources reference the send action). Covered: ping, server derived from key, list audiences and campaigns, upsert + tag + list round trip, 3-member batch, campaign report and email activity (read only), real 401/404/400 mapping, NewSubscriberTrigger first poll, and MemberStatusChangeTrigger (first poll fires nothing, a new unsubscribed member fires exactly once, third poll fires nothing).

Run in a local Kestra 2.x container with the plugin jar mounted: the plugin registers 13 plugin classes with no load errors, and this flow succeeded (kestra flow test, secret read from SECRET_MAILCHIMP_API_KEY):

id: mailchimp_hardening
namespace: company.team

tasks:
  - id: ping
    type: io.kestra.plugin.mailchimp.account.Ping
    apiKey: "{{ secret('MAILCHIMP_API_KEY') }}"

  - id: audiences
    type: io.kestra.plugin.mailchimp.audiences.ListAudiences
    apiKey: "{{ secret('MAILCHIMP_API_KEY') }}"
    maxItems: 5

  - id: log
    type: io.kestra.plugin.core.log.Log
    message: "{{ outputs.ping.healthStatus }} / audiences: {{ outputs.audiences.size }}"

Not exercised: a flow run through the UI/API (the local instance requires a Setup-page account), JDK 21 (only JDK 23 available locally), SendCampaign against a real campaign (deliberately never run), 403 and Retry-After on 429.


Setup Instructions

  • API key: Account > Extras > API keys; audience id: Audience > Settings > Audience name and defaults. Copy .env.example to .env (git-ignored) and load it for integration tests, see the README "Integration tests" section.
  • Known limitations: Mailchimp rejects example.com for test members, so set MAILCHIMP_TEST_EMAIL_DOMAIN to a domain with MX records; since_* filters are exclusive with whole-second precision (the triggers re-read one second and de-duplicate by id); DELETE only archives members; filter_bots is accepted but its effect was not observable; one page per trigger poll.

Contributor Checklist ✅

@kestrabot kestrabot Bot added this to Pull Requests Oct 8, 2026
NewSubscriberTrigger, MemberStatusChangeTrigger and CampaignSentTrigger
poll Mailchimp sorted ascending on timestamp_opt / last_changed /
send_time with 1 s since_* slack. TriggerCursor (newest timestamp + ids
seen at it) in the namespace KV store drops boundary rows client-side,
so inclusive or exclusive server filters both fire each row once. One
execution per poll carries all new rows as ION. Shared logic lives in
models/CursorPoll and audiences/AbstractMemberTrigger; interval
(default PT5M, minimum PT30S) in AbstractMailchimpTrigger.
@23abdul23

Copy link
Copy Markdown
Author

@fdelbrayelle Could you please review this?

@Coding-with-Adam
Coding-with-Adam requested review from a team and fdelbrayelle October 9, 2026 09:47
@Coding-with-Adam Coding-with-Adam added area/plugin Plugin-related issue or feature request kind/external Pull requests raised by community contributors labels Oct 9, 2026

@fdelbrayelle fdelbrayelle left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by the kestra-plugins:code-reviewer agent, run on behalf of @fdelbrayelle. Static read of the diff, the build and tests were not run.

VERDICT: APPROVE

Blocking: none found. I read the full diff (100 files, including tests, flows and docs). CI is green (check / main). There is no prior review on the PR.

Non-blocking notes:

  1. MailchimpClient.java and MailchimpConnectionInterface.java use the JDK HttpClient instead of io.kestra.core.http.client. The guidelines require Kestra's client. The PR body explains the choice: Kestra's client replays POST on 429/503, which is unsafe for SendCampaign and BatchSubscribe. A maintainer should confirm this deviation. Response bodies are also read whole with BodyHandlers.ofString(), which is bounded in practice by the 1000-row pages.
  2. AbstractMailchimpTask.java:24 uses @EqualsAndHashCode without callSuper = true (the trigger base class sets it). maxRetries, timeout and interval have no @Min bound in the schema, although the code checks the bounds at runtime. This breaks the guidelines' range rule.
  3. AbstractMailchimpTask.java:59 and AbstractMailchimpTrigger.java:59 both define the baseUrl override. Any https host receives the bearer token, so a flow author can send credentials anywhere. The schema marks it as for testing and proxies, so this is acceptable. Consider documenting it in src/main/resources/doc/io.kestra.plugin.mailchimp.md.

@fdelbrayelle fdelbrayelle left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. See my earlier review comment for non-blocking suggestions.

@23abdul23

Copy link
Copy Markdown
Author

@fdelbrayelle Thanks for the review , for the use of HttpClient instead of Kestra HttpClient is mainly due to prevention of double sending out the campains or batch.

For the other two notes I have pushed a small commit , to follow the guideline's range rule and the required documentation.

@fdelbrayelle

Copy link
Copy Markdown
Member

@23abdul23 Closing it since it should be a PR against plugin-mailchimp. Plus, what is the initial issue it addresses? Thanks!

@github-project-automation github-project-automation Bot moved this from To review to Done in Pull Requests Oct 12, 2026
@23abdul23

Copy link
Copy Markdown
Author

@fdelbrayelle Well this is a new plugin that I want to submit, currently this plugin doesn't exist and I wasn't assigned this issue (there was no issue for mailchimp in first place)

The usecase is like , for eg: we have an event and want to send campaign/poster invites to various past participants and insititutes / orgs , so mailchimp helps in analytics realted to campaings, clicks etc

@fdelbrayelle

Copy link
Copy Markdown
Member

@23abdul23 Could you first open an issue properly in https://github.com/kestra-io/kestra with why we should add this plugin? Then we'll consider to create the plugin-mailchamp and you could open a PR on it accordingly. Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/plugin Plugin-related issue or feature request kind/external Pull requests raised by community contributors

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants