Meta lays out eight guidelines for designing AI systems for 24/7 AR glasses

Editor: Esther

Source: Qingting.com

As we all know, Meta is not only limited to VR headsets such as Quest, but is also creating lighter AR glasses, with the goal of making the products better integrated into people's daily lives. In addition to lightweight hardware, it is also crucial in terms of functions and interactive experience, such as natural interaction methods, such as gesture input, and AI visual assistants, etc., which are inseparable from computer vision and AI technology.

AI technology will be an important part of AR glasses. By providing users with answers and suggestions in real time (such as recommended navigation routes, even schedules, and recommended dishes according to preferences, etc.) to improve the practicability of AR glasses, it will become Meta AR. One of the selling points of glasses. Especially, considering Meta's investment in AI technology in recent years, we have every reason to believe this.

Reality Labs released a study: XAIR, from which we can learn the design principles of AI systems in AR glasses. The framework is based on researches such as the Explainable Artificial Intelligence Framework (XAI) and Human-Computer Interaction (HCI), which contains 8 major design principles, which can provide valuable reference for the AI design of AR glasses.

**What is XAI? **

According to Qingting.com, XAI (Explainable AI), also known as Transparent AI (Transparent AI), is characterized by behaviors that are easy to understand. Most machine learning-based AI operates in so-called black boxes, and since it cannot provide the reasons and insights behind decisions, such AI is risky because it is uncertain whether it is trustworthy, reliable, or biased.

The concept of XAI can be traced back more than forty years ago. Later, with the success of the black-box AI/ML model, XAI technology began to attract the attention of academics, regulatory agencies and other industries. Research shows that XAI will hopefully provide users with clear decisions and build trust. Therefore, in the industrial field, XAI has begun to be applied to daily scenarios to improve user experience.

XAI can serve different target audiences and have various uses. Early XAI research only focused on algorithm developers, data scientists, and experts in the fields of clinical medicine. In recent years, more and more XAI has begun to target ordinary users and integrate with consumer products, such as displaying and recommending a certain product on a shopping website reasons and so on. However, it's still early days.

Importance of XAI

Making AI widely understood by humans will involve multidisciplinary research efforts. For example, ML researchers have developed algorithms that generate transparent models (e.g., decision trees, Bayesian models), or use post-hoc interpretation techniques (e.g., feature importance, visual explanations) to generate explanations. HCI researchers, on the other hand, focus on improving user trust and understanding of machine-generated explanations. Psychology researchers, on the other hand, study XAI from a more fundamental perspective, looking at how people generate, communicate, and understand.

Open and transparent AI is also very important, and it is in line with its future development strategy in the field of AR/VR. In XAIR research, the purpose of XAI is to help users clearly and easily understand AI decisions and functions by generating details or reasons. Meta pointed out that XAI is an important part of the AI-driven interactive system, and it will also play an important role in daily AR applications in the future, assisting users to interact with visualized smart services. XAI can better understand the behavior of AR intelligent systems, avoid unexpected AI decisions, cultivate privacy awareness, and gain user trust.

One challenge Meta currently faces, though, is creating effective XAI experiences for everyday AR applications. Most existing XAI research focuses on categorizing interpretation types and generation techniques, without considering the characteristics of everyday AR scenarios, such as perceptual information generated by users and context, running around the clock, and good adaptability. These factors can not only form more human-friendly explanations, but also influence the design of the interpretation interface.

Therefore, Meta proposed the XAIR design framework, which describes when and how to explain the decisions of AI in AR. In order to build the XAIR framework, an experiment with 500 people was also conducted to collect their preferences for AR experience design. In addition, the insights of 12 experts on AR interaction are also referred to.

The focus of this research is to identify three questions:

• When should the AI explain;
• what can be explained;
• How to explain.

Previous studies have explored the first two issues, and although not specific to AR, provided some useful information for the design of XAIR.

XAI Design Guidelines

Meta believes that if the AR glasses have intelligent services, then AI will play an important role, such as providing users with context-based suggestions based on the information captured by the sensors of the AR glasses. In addition, the interaction between AI and users needs to be based on effective XAI design to ensure that AI decisions are reliable and trustworthy, thereby improving user experience.

Different from the existing XAI framework for computers and mobile phones, AR's XAI design needs to incorporate deeper and richer contextual information (even considering the user's state), so it needs to be redesigned specifically for AR. Moreover, AR's XAI also needs to have 3D perception capabilities and be online in real time before it can be applied to daily AR scenarios and integrate interpretation content with physical space. For example, when recommending recipes, highlight the ingredients in the user's refrigerator at the same time, that is, explain the decision based on the context of the scene. In contrast, the existing XAI frameworks on the market cannot meet these needs.

Therefore, Meta summarized 8 major design guidelines through user surveys:

1. Always generate AI results to ensure that users can easily access them when they need them;
2. Do not automatically trigger the explanation, unless two conditions are met—identify the user's high cognitive load, sense of urgency, etc., or identify the user's surprise, confusion, unfamiliarity, uncertainty, etc.;
3. Three factors need to be considered for personalized explanation content: system goals, user goals and user portraits;
4. In the default state, give priority to the why explanation, and choose a concise explanation;
5. Always provide more detailed explanations, allowing users to expand according to their needs through small prompt windows, etc.;
6. By default, the same interpretation method as AI output is used (except tactile, audio), and when one mode load is high, another one is selected;
7. The content is mainly text. If it is a picture, it should be simplified as much as possible to make it easy for users to understand;
8. Embed explanatory content into the scene as covertly as possible, or overtly if not appropriate.

Meta combined with the design guidelines, developed some application cases and verified them among 10 designers. As a result, the designers believe that XAIR can provide comprehensive reference or help for the XAI framework design of AR, which helps to stimulate the thinking and imagination of designers. force. The 12 end users who participated in the experiment also reported that XAIR has excellent usability.

Applications

In this research, Meta has designed two demonstration cases, if you are interested, you can take a look:

1. When the user is jogging on the path, the AR glasses will show the user a nearby map considering the current season and scenery, suggesting a detour to the nearby road to enjoy the cherry blossoms. The explanations that AI can provide include: better scenery, the right length of the route, and the user's schedule. Explanation forms include text, pictures of cherry blossoms, and more.

2) When the AR user returns home after discussing gardening with neighbors, the AR glasses will display a "maintenance" prompt on the surrounding plants and provide the user with instructions on fertilizing the plants. This suggestion needs to be manually triggered by the user to avoid thinking that AI invades privacy, and manual triggering can better build trust. In addition, AI can also prompt: After the system scans, there are abnormal spots on the leaves of plants, indicating that they may suffer from fungal or bacterial infections. In addition to the text, the explanation form can also use AR to mark abnormal points on the leaves (the text is an obvious hint, while the AR hint is a hidden type, which is integrated with the scene).

reference: