When first trying to build an AR project with Vuforia plugin in Unity, the writer was amazed by the robustness of the image recognition algorithm within the Vuforia package. Tracking targets varies from images, cylindrical targets and even complex custom real-world objects. On a previous image processing course, the writer wrote a simple hand detection algorithm by chromatic aberration, which seems to be a kid thing compared with Vuforia.
However, with time goes by, the writer find some constrains when developing AR implementations both from Vuforia and Unity, such as losing tracking during user interaction, incompatibility of deploying projects to mobile platforms and etc.
Still very excited about AR projects, especially look forward to new possibilities of the ARKit when iOS 11 released, which is an real-world scene recognition technique with the 2D builtin cameras of iphone X.
Augmented Reality
1. ARmobile, August 2017 - present
Preface
ARchess and ARpiano are the first and second augmented reality applications the writer built. The aim of building such kinds of applications is to reduce the cost of buying as well as carrying real objects, which makes more sense to the piano. For the ARchess, it is hard for traditional technologies to achieve remote interaction when playing real world chess game while this augmented reality technique presents some possibilities.
By a customized image recognition package, in this case, Vuforia, the most popular AR development kit for Unity in the world, some real-world images are used as "nodes" between the virtual world and the real world. The writer first used poker cards to be these images for tracking and they were further proved to be especially good for tracking because of shapes and colors with many characteristic points.
Unity and Vuforia bring plenty of possibilities for potential AR apps. As far as the writer can think about, restaurant menu AR app, which recognizes the words on the menu and then shows a corresponding image or 3d model of the food, could be an interesting case.
1.1 Real-world objects supplement/replacement
Besides, virtual keyboard and virtual mouse are also useful implementation examples of augmented reality as presented in Microsoft Hololens. Traditional keyboard and mouse seem to be cumbersome when compared with future technologies in some fiction movies that people can touch any surfaces (e.g. tables and walls) to type commands and watch feedbacks.
Additionally, painting in three-dimensional space will be super exciting if the painting work can also be saved and easily shared on social media. With augmented reality, these seems is not a big issue.
1.2 Mobile platform AR
Although there exists several devices such as Microsoft Hololens and Google glasses that were designed intendedly for augmented reality usage, none of them actually changed life of the most ordinary individuals. Because the expensive price of some these devices are not affordable for most people, on the other hand, not reasonable for relatively low display area and functionalities compared with price. In the writer's opinion, in the visible future, mobile AR apps will be the favorable form of augmented reality implementations for most people due to current high market occupancy of smart phones. Anyone with a smart phone can enjoy amazing and unforgettable experience from mobile AR apps.
This is a simple iOS AR app the writer developed recently (see the video above). There are only four models to change inside the app. User can swipe the screen leftwards or rightwards to switch among those models. Unlike Pokemon Go, which is always debated as not a real AR app, the app understands the relative positions between the phone camera and the real world table so it stays in the 3d space instead of keeping in the center of the screen.
In the above video, a cylindrical target (i.e. a water bottle) is used for tracking. As the "node" between the real world and the virtual world, the water bottle can display a video and several 3d letters in front of it. Right now, some constrains still exist when deploying into mobile platform because Unity does not support MovieTexture class on mobile devices. The videos that can be displayed in mobile devices are full-screen rather than a 3d object in arbitrary position in 3d space. However, although some problems still exist when exporting Unity project to mobile platforms due to the reason above, the essential idea of this video brings the project to the next level - customized service.
1.3 Customized service
When users gets to a specific tracking target at a certain location, pre-designed texts, audios, videos and models can be displayed in magical ways to help understand the information of a target (here is the Duke chapel).
1.3 An AR Online Platform - real-time interaction with the real world
(In progressing ...)