feat(media): moderate every uploaded image and video

[aquie00t]

Summary

Uploaded images and videos reach the platform with no content check at all. This adds automated moderation to every upload endpoint — post and comment media, article covers, avatars and banners — and to the paths that publish what was uploaded.

Images are scanned inside the upload request, so a refused file never reaches storage and never gets a URL. Videos cannot be judged that way — the provider has to fetch and sample them — so they are stored withheld from the read path and resolved by a cron worker. Verdicts are tiered: explicit sexual content, gore, self-harm and hate imagery are refused outright, while suggestive content, weapons and depicted violence only mark the media sensitive so the client blurs it. That split is deliberate — this platform is full of game screenshots, and a filter that deletes those is one people route around.

Content whose media has no verdict yet is still served; only its mediaUrls are withheld, and mediaPending says why. The text was never in question, and a video usually clears within a minute.

Sightengine is the provider: one API covers both stills and video and accepts video by URL, which the alternatives do not — they read video only from their own cloud's object storage, and this platform stores media in R2. MODERATION_ENABLED=false swaps in a stand-in that approves everything, which is what the test environment and a laptop without credentials use. It is never a fallback: an upload that could not be checked is refused, because otherwise every provider outage becomes the way past the filter.

Note that this branch also carries a27aa64, the earlier feed fix, which was already on the base branch.

Root cause

There was no moderation anywhere, but the more specific problem is that adding it at the upload endpoint would not have been enough.

POST /media trusted the client's Content-Type and filename — both attacker-controlled, so a request could claim image/png while carrying an SVG, and clip.png.html read as an image to an extension check. UpdateAvatarUseCase and UpdateBannerUseCase had the same weakness. Only the article cover endpoint read the bytes, and that check is now the shared one.

The larger hole was in publishing rather than uploading. createPostBodySchema.mediaUrls accepted any format: "uri" string and CreatePostUseCase stored it unexamined, so a client could skip the upload endpoint entirely and put whatever URLs it liked into a post body. Scanning at upload time governs only what the upload endpoint writes; without a check at publish time the whole pipeline would have been decorative. Every stored file therefore gets a media_assets row, and post and comment creation resolve each submitted URL back to one — refusing it unless this author uploaded it, through the matching channel, and nothing else has already claimed it. The attach itself is the atomic claim, guarded on owner_id IS NULL, so two requests carrying the same key cannot both win.

The worker is written around the ways a video could otherwise be hidden forever. It re-reads an asset after scanning rather than trusting the snapshot it claimed, because an upload is routinely claimed before the post using it is submitted. Claims carry a lease, so a process killed mid-scan releases the asset instead of stranding it in SCANNING, which nothing else selects. A failure handler that fails cannot abandon the rest of its batch. And an owner deleted mid-scan is a no-op rather than an error, so a verdict already paid for is not discarded and re-derived into a false rejection.

Tests

• Unit — 1118 passing. New coverage for the shared upload path (a refused image never reaches storage; a provider error fails closed; a video is stored pending), byte-level format detection including MP4-family brand disambiguation and SVG rejection, the score-to-verdict thresholds as a pure function, the ownership resolver's five refusal cases, and the worker — stale-owner recovery, batch isolation, retry budget, and the reject-and-notify path.
• Integration — 180 passing. PrismaMediaAssetRepository against Postgres, including the concurrent claim (two workers never receive the same asset), lease reclaim of a stranded row, the attach race guard, and detach.
• E2E. Covers the guards that need neither storage nor a provider: format rejection before anything is written, and refusal of media URLs the author did not upload, on both posts and comments. The happy path needs a live R2 connection and stays out of scope, as it already does for the other upload endpoints.

The last full local E2E run was 380/382, taken before the review fixes in this branch; the two failures were timeouts in the feed and quote-notification suites, unrelated to this change — both use text-only posts, and this pipeline performs no queries for those. Local E2E could not be re-run afterwards: prisma migrate reset repeatedly failed with P1002 against the shared Neon test database. CI runs all three suites against a local Postgres.

Deployment

MODERATION_ENABLED=true requires SIGHTENGINE_API_USER and SIGHTENGINE_API_SECRET; the service refuses to boot without them rather than answering 503 to every upload while nobody notices. Existing rows are unaffected — they predate the pipeline, carry no assets, and default to approved and not sensitive.

---

AI asistan: Claude Opus 5

[github-actions]

🎉 This PR is included in version 1.15.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀