Hello,
In fact, your shot is what we call a "nodal pan". That means the camera rotates almost around its optical center. This kind of camera move does not give any perspective information (it's not a matchmover issue, it's a physical limitation). So you'll only be able to compute a camera rotation with this data. The 3D tracks will be useless.
In this case, you can add the nodal pan constraint to the camera to force it to only rotate to have a more accurate result.
Therefore, you won't be able to define a coordinate system based on the tracks. You should export this camera move directly in your 3D package and work with it.
What you can do to add perspective information is either:
- add some images from another point of view (one can be enough)
- input the 3d coordinates of some of the point
Hope this helps.