Investigating Google ARCore for potential robotics usage, it was useful to review their Fundamental Concepts and Design Guidelines because it tells us the motivation behind various details and the priorities of the project. That gives us context around what we see in the nuts and bolts of the actual APIs.

But the APIs are where "the rubber meets the road" and where we leave all the ambitions and desires behind: the actual APIs implemented in shipping phones define the limitations of reality.

We get a dose of reality pretty immediately: estimation of phone pose in the world comes with basically no guarantees on global consistency.

World Coordinate Space
As ARCore's understanding of the environment changes, it adjusts its model of the world to keep things consistent. When this happens, the numerical location (coordinates) of the camera and Anchors can change significantly to maintain appropriate relative positions of the physical locations they represent.

These changes mean that every frame should be considered to be in a completely unique world coordinate space. The numerical coordinates of anchors and the camera should never be used outside the rendering frame during which they were retrieved. If a position needs to be considered beyond the scope of a single rendering frame, either an anchor should be created or a position relative to a nearby existing anchor should be used.

Since it is on a per-frame basis, we could get Pose and PointCloud from a Frame. And based on that text, these would then need to be translated through anchors somehow? The first line of Anchor page makes it sound that way:

Describes a fixed location and orientation in the real world. To stay at a fixed location in physical space, the numerical description of this position will update as ARCore's understanding of the space improves.

However, I saw no way to retrieve any kind of identifying context for these points. Ideally I would want "Put an anchor on that distinctive corner of the table" or some such. But still, "Working with anchors" has basic information on how it is useful. But as covered in many points throughout ARCore documentation, use of anchors must be kept at a minimum due to computational expense. Each Anchor is placed relative to a Trackable, and there are many ways to get one. The biggest hammer seems to be getAllTrackables from Sesson, which has a shortcut of createAnchor. There are more narrowly scoped ways to query for Trackable points depending on scenario.

Given what I see of ARCore APIs right now, I'm still a huge fan of future potential. Unfortunately its current state is not a slam dunk for robotics application, and that is not likely to change in the near future due to explicit priorities set by the product team.

But while I had my head buried in studying ARCore documentation, another approach popped up on the radar: the OpenCV AI Kit.