Allow system pybind

[dellaert]

so https://github.com/borglab/gtsam/blob/ffd471d5ca40a10a3a06e9ce83478817f1e507b0/.github/workflows/build-special.yml#L109-L110 can use system