Hello,
Did you define your coordinate systems using well reconstructed 3D tracks (that means with good reconstruction score, or residual, so they're shown in green).
The best is to used rock-solid tracks, sparse enough in the 3D space to have a better accuracy. Use as first axis the one you're the most confident in.
You can send your project file if you want us to check that.
Thanks