Skip to content

feat(chartctl): Chart Deployments — remote EA deployment over HTTP - #10

Open
Marinski wants to merge 2 commits into
psyb0t:masterfrom
Marinski:feature/chart-deployments
Open

feat(chartctl): Chart Deployments — remote EA deployment over HTTP#10
Marinski wants to merge 2 commits into
psyb0t:masterfrom
Marinski:feature/chart-deployments

Conversation

@Marinski

@Marinski Marinski commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Chart Deployments (chartctl)

Chart Deployments (chartctl) lets you attach Expert Advisors to charts with .set files over the HTTP API — no RDP, no terminal restart, and no manual chart work.

The API stores the desired state, while a resident Loader EA inside the terminal reconciles charts to match that state and reports the observed state back. A deployment only reaches the running state after the Loader confirms the Expert Advisor is actually live on a chart.

New Endpoints

Method Endpoint Description
POST / GET / DELETE /experts, /experts/<hash> Stage, list, or remove EA .ex5 files
POST / GET /sets, /sets/<name> Stage, list, or inspect .set files (returns parsed inputs)
POST / GET /deployments Create or list deployments
GET / PATCH / DELETE /deployments/<id> Inspect, pause/resume, change set, or delete a deployment
POST /deployments/reconcile Force an immediate reconcile
GET /charts Live chart / EA inventory
GET /loader Loader EA status and version
POST /charts/<id>/screenshot Capture a chart as PNG
POST /charts/<id>/close Close any chart by ID
GET / PUT /webrequest Read or update the WebRequest allowlist
POST /webrequest/apply Re-apply the allowlist immediately

What's Included

  • mt5api/chartctl/ — paths, setparse, tpl_builder, registry, command
  • mt5api/handlers/chartctl.py — lock-free REST surface, gated by CHARTCTL_ENABLED (config-driven; routes are omitted when disabled)
  • mt5api/handlers/webrequest.py — WebRequest allowlist provisioning
  • mt5api/chartctl/autoit_webrequest.py — AutoIt GUI automation
  • assets/experts/MT5ChartLoader.mq5 — reference Loader EA (~60 lines)
  • assets/experts/include/ChartControl.mqh — portable protocol include
  • assets/autoit/ — AutoIt interpreter and GUI automation scripts
  • scripts/compile-chartctl-loader.bat — zero-touch bootstrap compiler
  • scripts/config_helper.pywrite_ini emits [StartUp] Expert= and provides the chartctl_enabled query command
  • docs/chart-control-protocol.md — complete protocol specification
  • tests/test_chartctl_units.py + tests/test_chartctl_endpoints.py — 43 tests plus a Python fake_loader for Linux CI (no MT5 dependency)
  • tests/test_webrequest.py — 23 tests covering the allowlist codec and endpoints
  • examples/chartctl/deploy.sh — runnable deployment example

Zero-Touch Bootstrap

No RDP step is required.

On VM boot:

  1. compile-chartctl-loader.bat compiles the Loader in every broker base using MetaEditor64.
  2. The resulting .ex5 is propagated to every terminal instance.
  3. config_helper.py write_ini adds a [StartUp] section to mt5start.ini.

The Loader automatically attaches when the terminal starts.

To disable this behavior for a specific terminal, set:

chartctl: false

How It Works

The Loader polls desired.json, reconciles the desired state with actual charts, writes observed.json so the API knows what is actually running. All handler endpoints are lock-free and perform only file I/O.

WebRequest Allowlist

Two application methods selected automatically at runtime: AutoIt GUI automation (Windows VM) or common.ini encoding (Bare Metal).

Changelog

See the CHANGELOG.md Unreleased section for the complete list of changes.

Testing

pytest tests/ --ignore=tests/integration --ignore=tests/test_requirements_sync.py
→ 284 passed
  • 43 chartctl tests
  • 23 webrequest tests
  • 218 existing tests

@Marinski
Marinski force-pushed the feature/chart-deployments branch 2 times, most recently from 90905af to a960a7d Compare August 5, 2026 10:49
@Marinski
Marinski force-pushed the feature/chart-deployments branch 2 times, most recently from 4110755 to bdadec7 Compare August 10, 2026 19:26
@Marinski
Marinski force-pushed the feature/chart-deployments branch 2 times, most recently from 295c983 to 60932f3 Compare August 20, 2026 15:08
@Marinski
Marinski marked this pull request as ready for review August 20, 2026 15:40
@Marinski

Copy link
Copy Markdown
Contributor Author

Merge order: this one goes last

These three were built in a sequence and should land in it:

  1. feat(ops): recreate VM containers that are persistently unhealthy #15 — vm-watchdog
  2. feat(compile): POST /compile — MQL5 source in, .ex5 out #16POST /compile
  3. feat(chartctl): Chart Deployments — remote EA deployment over HTTP #10 — this PR

All three are now rebased on current master independently, so each shows MERGEABLE on its own. That is a little misleading, and worth stating rather than letting it be discovered at merge time.

Merging this one before the other two would not break anything — but it makes the other two harder, because the overlap lands on them instead. This PR is the widest of the three (it touches mt5api/config.py, main.py, server.py, scripts/config_helper.py, Dockerfile.test, run.sh, README.md, CHANGELOG.md), so it is the cheapest one to rebase and the most expensive one to rebase around.

What actually happens, rehearsed rather than assumed

I merged all three locally in the order above:

Step Result
master + #15 clean
+ #16 clean
+ #10 (this PR) conflicts in 4 files

So even in the right order this one needs a rebase once the first two land. The conflicts are all additive collisions, not semantic ones:

Every one resolves as a union. Nothing here disagrees about behaviour; the two features just grew into the same lines.

I will rebase and force-push this within a few minutes of #16 landing, so it does not sit in a conflicted state. Ping me if you would rather I do it in a different order, or if you plan to merge this one first — the rehearsal is cheap to redo the other way round.

@psyb0t

psyb0t commented Aug 25, 2026

Copy link
Copy Markdown
Owner

I verified assets/autoit/AutoIt3_x64.exe against AutoIt's official v3.3.18.0 portable archive. The SHA-256 is 5d69a932a077fee044b193c28e84564143f5c7e51079ab48e88fef74ab0b77b7 on both sides, and the files are byte-for-byte identical. That resolves the binary provenance concern.

One required packaging fix remains before merge. AutoIt's EULA permits redistribution, including inside another product, but requires every copy to include its copyright and trademark notices and a copy of the EULA. Please add the unmodified official AutoIt EULA and required notices under assets/autoit/, then this part is good to go.

Attach Expert Advisors to charts with set files over the HTTP API. No RDP,
no terminal restart, and nobody clicking through the Navigator.

Stage `.ex5`/`.set` artifacts, declare deployments (expert + set + symbol
+ timeframe) as desired state, and a resident loader EA reconciles the
terminal's charts to it. New endpoints: `POST/GET/DELETE /experts`,
`POST/GET /sets` + `GET /sets/<name>`, `POST/GET /deployments` +
`GET/PATCH/DELETE /deployments/<id>`, `POST /deployments/reconcile`,
`GET /charts`, `GET /loader`, `POST /charts/<id>/screenshot`,
`POST /charts/<id>/close`.

Chart Control Protocol v1 is a file-based contract in `MQL5\Files\chartctl\`
(`desired.json` / `observed.json` / command channel), documented in
docs/chart-control-protocol.md. A deployment only reports `running` once
the loader confirms the expert is live on a chart, so the status reflects
the terminal rather than the request. Drift and failures surface in
`observed.json`.

Ships a reference loader (`assets/experts/MT5ChartLoader.mq5`) plus a
portable include so an existing resident EA can adopt the protocol with
three calls instead of running a second EA. A terminal GlobalVariable
mutex makes co-existence and duplicate launches safe.

OPT-IN. `chartctl.enabled` defaults to false. Enabling it writes a
`[StartUp]` section into every live terminal's INI so the loader attaches
at launch - fleet-wide behaviour that has to be asked for, never
inherited by upgrading. With the block absent nothing changes and the
endpoints return 404. Any single terminal opts out with `chartctl: false`.

WebRequest allowlist provisioning (`GET`/`PUT /webrequest`,
`POST /webrequest/apply`) sets the terminal's allowed-URL list over the
API instead of the Options dialog. Two apply paths, chosen at runtime:
on this Windows build the list lives in machine-bound `MQL5\experts.dat`
and is dropped on every restart, so a bundled portable AutoIt interpreter
drives the dialog and main.py re-applies the persisted list ~25s after
each terminal start; on bare metal, where `common.ini` is the store, it
encodes the list directly (format reverse-engineered from terminal64.exe
and verified byte-identical against 18 real broker blobs). GUI applies
serialize host-wide behind a named kernel mutex, since desktop focus is
shared across terminals.

All chartctl handlers are lock-free - pure file I/O against the terminal
data dir - so they never queue behind the process-wide MT5 SDK lock.

Tests: chartctl units, endpoints, and webrequest, plus a Python fake
loader that plays the EA side of the protocol so the endpoint suite runs
on Linux with no MT5. 484 passing.
…binary

AutoIt's EULA permits redistribution, including inside another product,
on the condition that every copy 'should include all copyright and
trademark notices, and shall be accompanied by a copy of this EULA'.
The bundled AutoIt3_x64.exe met the provenance bar (byte-identical to
the official portable archive, verified independently on both sides of
review) but shipped bare of both.

assets/autoit/ now carries:

- EULA.htm - the official EULA, verbatim as published by the AutoIt
  Team in the AutoIt v3 documentation, matching the release the binary
  ships in. Not retyped or reformatted.
- NOTICE.txt - the copyright notice taken from the binary's own version
  resource ('(c) 1999-2025 Jonathan Bennett & AutoIt Team'), the
  trademark notice from autoitscript.com, provenance (version, sha256,
  source archive), and a line making clear the .au3 scripts beside it
  are this repository's own, not AutoIt's.

binaries.lock.json's provenance note is updated to record that the
'no vendor-published hash' caveat no longer stands alone: the file has
been verified byte-identical to the official archive, and the EULA and
notices are named as the redistribution terms. docs/chart-deployments.md
points at NOTICE.txt where the bundled interpreter is introduced.
@Marinski

Copy link
Copy Markdown
Contributor Author

Done in 4eae86bassets/autoit/ now satisfies the EULA's own redistribution condition ("each copy should include all copyright and trademark notices, and shall be accompanied by a copy of this EULA"):

  • EULA.htm — the official EULA, verbatim as the AutoIt Team publishes it in the AutoIt v3 documentation, matching the release the bundled binary ships in. Not retyped, not reformatted.
  • NOTICE.txt — the copyright notice taken from the binary's own version resource (© 1999-2025 Jonathan Bennett & AutoIt Team), the trademark notice from autoitscript.com, full provenance (version 3.3.18.0, the sha256 you verified, source archive), and an explicit line that the .au3 scripts beside it are this repository's own automation, not AutoIt's.

While there I re-downloaded the official portable archive and confirmed the same byte-identity you did (5d69a932…), so binaries.lock.json's provenance note no longer leans on the signature alone — it now records the archive verification from both sides of this review and points at the EULA/notice files as the redistribution terms. docs/chart-deployments.md references NOTICE.txt where the bundled interpreter is introduced. make verify-binaries still passes.

Merge order

#15#16#18#10 — this one last, as before. It is the widest of the four and shares mt5api/config.py, scripts/config_helper.py, config.yaml.example, and the [Unreleased] changelog section with #16 and #18, plus run.sh/Dockerfile.test with #15. It is already rebased onto current master (post-#17; the Dockerfile.test COPY conflict resolved as the union), and the standing commitment from the earlier rehearsal holds: I will re-rebase promptly after each predecessor merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants