PolygonRayIntersection failing on non-Euclidean/concave/hollow polygons

[adario]

What happened?

When using a PolygonHitbox for non-Euclidean/concave/hollow polygons, the PolygonRayIntersection mixin fails when a ray traverses multiple hollow parts of a concave polygon.

What do you expect?

As long as a PolygonHitbox defines a closed polygon correctly, the actual topology should not confuse PolygonRayIntersection.

How can we reproduce this?

While I no longer have steps to reproduce the issue, there is a testbed to verify the refactoring result in PolygonRayIntersection, which uses my Flame fork mentioned below.

What steps should take to fix this?

These are the relevant changes in my Flame fork; eventually I plan to submit a PR.

Do have an example of where the bug occurs?

Without my changes, the bug used to occur with all the new Path shapes of the RaysInShapeExample in the testbed; the pre-existing shapes (circle, rectangle, quadrilateral) used to work, but changing the quad into a sort of 'U' shape also failed.

Relevant log output

Execute in a terminal and put output into the code block below

Output of: flutter doctor -v

[✓] Flutter (Channel stable, 3.47.2, on macOS 26.6.2 25G83 darwin-arm64, locale en-IT) [881ms]
• Flutter version 3.47.2 on channel stable at /Users/d.accornero/Applications/flutter
• Upstream repository https://github.com/flutter/flutter.git
• Framework revision d3b14c8769 (12 days ago), 2026-08-26 16:07:51 -0700
• Engine revision a804b26164
• Dart version 3.13.2
• DevTools version 2.60.0
• Feature flags: enable-web, no-enable-linux-desktop, enable-macos-desktop, no-enable-windows-desktop, enable-android, enable-ios, cli-animations, enable-native-assets, enable-record-use,
enable-swift-package-manager, omit-legacy-version-file, enable-lldb-debugging, enable-uiscene-migration

[!] Android toolchain - develop for Android devices (Android SDK version 36.1.0) [1,658ms]
• Android SDK at /Users/d.accornero/Library/Android/sdk
• Emulator version 37.1.11.0 (build_id 15917651) (CL:N/A)
• Platform android-36.1, build-tools 36.1.0
• ANDROID_HOME = /Users/d.accornero/Library/Android/sdk
• Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
This is the JDK bundled with the latest Android Studio installation on this machine.
To manually set the JDK path, use: flutter config --jdk-dir="path/to/jdk".
• Java version OpenJDK Runtime Environment (build 25.0.3+-15898627-b508.16)
✗ Android license status unknown.
Run flutter doctor --android-licenses to accept the SDK licenses.
See https://flutter.dev/to/macos-android-setup for more details.

[✓] Xcode - develop for iOS and macOS (Xcode 26.6) [1,716ms]
• Xcode at /Applications/Xcode.app/Contents/Developer
• Build 17F113
• CocoaPods version 1.17.0

[✓] Chrome - develop for the web [6ms]
• Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Connected device (4 available) [6.6s]
• Undici (mobile) • 00008030-001065060C42802E • ios • iOS 26.6.1 23G83
• iPhone 17e (mobile) • FD5942B7-8265-4568-B74E-698F1FB20941 • ios • com.apple.CoreSimulator.SimRuntime.iOS-26-5 (simulator)
• macOS (desktop) • macos • darwin-arm64 • macOS 26.6.2 25G83 darwin-arm64
• Chrome (web) • chrome • web-javascript • Google Chrome 152.0.7977.83
! Error: Browsing on the local area network for iPad di Sviluppo (2). Ensure the device is unlocked and attached with a cable or associated with the same local area network as this Mac.
The device must be opted into Developer Mode to connect wirelessly. (code -27)
! Error: Browsing on the local area network for Eleven. Ensure the device is unlocked and attached with a cable or associated with the same local area network as this Mac.
The device must be opted into Developer Mode to connect wirelessly. (code -27)
! Error: Browsing on the local area network for iPad di Julie. Ensure the device is unlocked and attached with a cable or associated with the same local area network as this Mac.
The device must be opted into Developer Mode to connect wirelessly. (code -27)

[✓] Network resources [285ms]
• All expected network resources are available.

! Doctor found issues in 1 category.

Affected platforms

iOS, Android, Web

Other information

No response

Are you interested in working on a PR for this?

• I want to work on this

[spydon]

This is mentioned in the PolygonHitbox docs and it's due to that when we tried to support all polygons we got a lot worse performance. If you can make it on par with the performance that we get today (or only worse for concave polygons) we'd be happy to review a PR!
Several other game engines have the same restriction.

[adario]

This is mentioned in the PolygonHitbox docs and it's due to that when we tried to support all polygons we got a lot worse performance. If you can make it on par with the performance that we get today (or only worse for concave polygons) we'd be happy to review a PR! Several other game engines have the same restriction.

My bad, I apologise: looks like the bug label should be removed then, but obviously I can't do that myself...
Of course I understand about the performance issues: I'm not claiming a breakthrough in segment intersection calculations... ;-)
Point containment was already working fine, so I haven't touched it (apart from exposing it internally). I understand the PolygonRayIntersection changes can't be as cheap as the previous, classic edge-crossing approach; I'll report back if/once I manage to benchmark properly.

[adario]

This is mentioned in the PolygonHitbox docs and it's due to that when we tried to support all polygons we got a lot worse performance. If you can make it on par with the performance that we get today (or only worse for concave polygons) we'd be happy to review a PR! Several other game engines have the same restriction.

I've got a few questions about benchmarks: I've tried to create one using collision_detection_benchmark as a template, but I'm sure I'm doing something wrong (calling mountGame fails)... please let me know whether/where I could bother you with my questions.

Anyways, I've added support in my fork for the point-containment behaviour, via a useContainment parameter to raycast() and rayIntersection(), which defaults to false to maintain compatibility.
The PR is just an internal draft; I've updated the RaysInShapeExample in the testbed with the new parameter.