S.L.A.M. and Optical Tracking for XR
S.L.A.M. and Optical Tracking for XR
Simultaneous Localization and Mapping is the computational problem of constructing or updating a map of an unknown environment while simultaneously keeping track of an model’s location within it. First I’ll start with the basics.
Tracking refers to the alignment of spatial properties. Registration is used to describe sensing and measurement of AR systems. For us to be a part of AR we must know where our position and orientation (pose) is in ‘real time’ and have it continuously updated. These are measured in 3-D position and 6-dimensional pose. This is also referred to as 3DoF (degrees of freedom)and 6DoF.
Registration in AR refers to the alignment to the coordinating systems between the real world and the virtual world. This requires the user’s head or camera within their smartphone, glasses, or headwear. There are 2 types of registration; static and dynamic registration. Static registration is when the user or camera is not moving requiring a calibration of the tracking system between the real world and AR. Dynamic registration is when the user or camera is moving and requires tracking.
TRACKING STYSTEMS
Stationary tracking systems were the first to become popular for virtual reality applications. As AR and VR progressed, Stationary Tracking is not as common in today’s systems, instead we use Mobile Tracking. Modern devices, smartphones in particular, have multiple sensors and access to wireless networks. While these devices already come equipped with all these sensors, they still find themselves in need of external sensors to connect with to produce an AR experience.
OPTICAL TRACKING
Optical methods of Tracking represent a set of computer vision algorithms and tracking devices such as a camera of visible or infrared range, a stereo camera, and a depth camera. Using images from these cameras requires combining them with a reference model. If a model is obtained before the tracking system, it is model-based tracking. The alternative is model-free tracking witch refers to when the model and tracking are being obtained at the same time. This is referred to as Simultaneous localization and mapping (or S.L.A.M.).
S.L.A.M. can combine 3D modeling with 3D tracking. This means that model-free tracking begins when the experience begins. With this type of tracking AR objects must be placed spontaneously and cannot be connected to the real world.
Markers and Natural features
When we talk about AR objects, we separate them into two categories; a marker or a natural feature. A marker or fiducial, help provide tracking for objects that lack variety in color and/or textures or repeating patterns that make it difficult for sensors differentiate between multiple features. Markers can come in different shapes and sizes but mostly are either round or square and provide high contrast typically in black and white.
Markers or fiducial
Markers or fiducial
Sphere shaped markers tend to have a greater advantage because they are the same shape as a lens or disc. The sphere is also ideal for tracking a user or other objects that are agile and change positions and orientation to the camera. They also have one main focal point as opposed to a quadrangular shape that requires a minimum of 3 focal points. The square design is common for barcodes and other stand alone objects of identification.
Alternatively, retro-reflective foil can be constructed as a markers. This material is commonly used in safety belts and on other safety clothing items and produces high contrast. This material is placed on spherical markers and placed around the camera to act as a high contrast markers.
Sometimes developers do not want to use markers or simply don’t have the means to make them so they then rely on natural features to communicate with sensors. Natural feature tracking requires high image quality and more computational resources. The most common types of these can be edge features or key frames assuming the outline features are easily detectable. Key frames are used but must be from a specific vantage point.
This is a natural feature in which the sensors from your phone detect the natural edge of the milk carton lid to provide an AR experience.
This is a natural feature in which the sensors from your phone detect the natural edge of the milk carton lid to provide an AR experience.
