Special Articles on the Potential of LLM Technologies to Support Business
AI Customer Service Enabled by Communication AI

Automatic Response System Speech-rate Analysis Customer Behavior Analysis

Aogu Yamada, Satoshi Konishi, Takuya Kobayashi, Ayaka Ueda, Yuka Yokogawa and Yuriko Takahashi
Service Innovation Department

Abstract
Existing chatbots and AI for customer service have been problematic because they cannot respond in a way tailored to individuals and their situations. Solving this problem necessitates understanding the pace of conversation and needs of the customer and interpreting the unspoken intentions contained in the customer's inquiry. Considering these necessities, NTT DOCOMO has developed a “Communication AI” that combines multiple AI technologies, namely, speech-rate measurement, customer behavior analysis, and questioning-in-return for collecting information using a LLM, to respond to customer inquiries, and we aim to use Communication AI to provide appropriate support to customers.

01. Introduction

  • Existing automatic response systems*1 for handling customer ...

    Open

    Existing automatic response systems*1 for handling customer inquiries received at customer-service online help desks and call centers give uniform responses regardless of a variety of customer situations, and they do not always provide responses that satisfy the customer. To address such issues, it is necessary for the system side to understand the situation and attributes of each customer and to respond on the basis of that information. Another major issue with automatic response systems is that they are unable to provide an appropriate response if the information entered by the customer in their inquiry via voice or chat text is insufficient.

    At NTT DOCOMO, we have developed “intent-interpretation technology” [1] for automatic response systems that can identify the intended purpose of a customer inquiry from the voice of the customer and the text that they input. However, if the information in the inquiry is insufficient, the accuracy of intent identification drops significantly. It has therefore been a challenge to resolve such information shortages.

    In light of that challenge, NTT DOCOMO has developed an automatic response system that combines multiple AI technologies developed by NTT DOCOMO to date, called “Communication AI,” and we apply this system to perform two tasks: (i) generate a response that accounts for the situation and attributes of each individual customer and (ii) resolve the issue of insufficient information to identify the purpose of the customer's inquiry.

    In this article, Communication AI is described in detail.

    1. Automatic response system: A system that—in place of an actual person—automatically responds to customer calls or online-chat inquiries at customer-service desks (e.g., call centers) and provides guidance to the customer.

    • 02. Overview of Communication AI

  • Communication AI is a system that accepts verbal inquiries from ...

    Open

    Communication AI is a system that accepts verbal inquiries from a customer and outputs a verbal response to the customer that is based on reference information registered in advance in response manuals and other sources.

    The configuration of this system using NTT DOCOMO's assets is shown schematically in Figure 1. Communication AI has three major functions.

    The first function is to answer the customer's inquiries in a more personalized manner than simply responding to them in the conventional way. To determine the customer's level of urgency and the conversation rate appropriate for each customer, the system works in conjunction with NTT DOCOMO's “Speech Digital Transformation (DX)*2 Platform” [2], which consolidates speech analysis technologies developed by NTT DOCOMO (such as voice recognition and speech-rate measurement) on one platform (Fig. 1 (1)). By utilizing these technologies, Communication AI responds to the customer according to their non-verbal information*3.

    The second function is to use customer behavior analysis technology [3] and “docomo Sense” [4], which use NTT DOCOMO's membership base to determine the customer's needs according to their customer behavioral data*4 and attribute data*5, to autonomously make proposals that satisfy the potential needs of the customer in question (Fig. 1 (2)).

    The third function is to ask a question-in-return (to the customer) to clarify the purpose of the inquiry in a manner that avoids responding while the content of the customer's inquiry is still unclear. Communication AI uses intent-interpretation technology to identify the purpose of the inquiry and extract the reference information needed to respond to the customer; however, as a preparatory step, it asks questions in return to gather information and confirm understanding (Fig. 1 (3)). The latter step significantly improves the accuracy of purpose identification compared to not asking a question-in-return.

    The mechanism by which this system, with the three above-described functions, operates is described hereinafter.

    Figure 1 NTT DOCOMO's AI assets that make up Communication AI
    1. DX: Using IT to transform services and business models to facilitate business and enhance all aspects of people's lives.
    2. Non-verbal information: Information (other than that conveyed via verbal communication) conveyed to the other party during communication. It includes tone of voice and facial expressions.
    3. Behavioral data: Data that captures behavioral patterns of customers such as their Web-browsing history.
    4. Attribute data: Data that captures customer profiles such as age and gender.

    • 03. Personalized Response to Customers

  • The key to more effective communication when responding ...

    Open

    The key to more effective communication when responding to customers is to consider not only verbal information but also non-verbal information. By utilizing NTT DOCOMO's Speech DX Platform, it is possible to grasp the rate of the customer's speech in real time and reflect it in the customer service.

    An example of a personalized response to customers by Communication AI is shown in Figure 2. If the customer asks a question with high speech rate, the system judges that the customer is in a hurry and increases the Communication-AI speech rate to match that of the customer. Moreover, by making the explanation more concise than usual, the system provides a quicker response to customers who are feeling impatient. On the contrary, if the customer is calm and ask a question with low speech rate, the system slows its speech to a lower rate than usual. Moreover, by making the response explanations more detailed than usual, the system gives the customer a sense of security. We expect that by achieving personalized customer communication in this way, the system will significantly improve the quality of the customer experience.

    Figure 2 Examples of personalized response to customers enabled by Communication AI

    • 04. Response Based on Customer's Needs

  • If companies are to provide personalized customer ...

    Open

    If companies are to provide personalized customer support, they must identify customer needs and issues on the basis of customer information, such as attributes and behavior, and support customers in the most appropriate way. Communication AI meets this need using docomo Sense and customer behavior analysis technology that identify customer needs from behavioral data by utilizing NTT DOCOMO's membership base.

    For example, even if it is predicted that a certain customer is highly likely to purchase a smartphone, if that customer has questions about the purchasing procedure, it is necessary to refer them to the information necessary to resolve those questions at the same time as recommending the new smartphone. Customer behavior analysis technology predicts a customer's future actions and calculates the probability of them occurring, and from the calculated probability, the appropriate method of support can be determined. By making suggestions based on the prediction results, it is possible to help the customer complete the procedures smoothly (Figure 3).

    As for another example, when announcing a sales campaign to a customer, it is normal to generally announce the campaign to each customer in the same way. However, it is more effective to mention that the campaign matches certain customers' hobbies and preferences. In that way, the benefits and appeal of the campaign can be emphasized. When introducing the same campaign to different customers, it is more effective to announce the campaign to sports lovers by using a sports-video service as an example and to movie lovers by using a video service with a wide selection of movies as an example. Since docomo Sense can identify the interests of customers from NTT DOCOMO's membership base, it is possible to use this information to respond to individual customers according to their needs.

    Figure 3 Guidance that meets the customer's needs enabled by customer behavior analysis

    • 05. Question-in-return Technology for Identifying Purpose of Customer's Inquiry

  • When a customer's inquiry is ambiguous, it is ...

    Open

    When a customer's inquiry is ambiguous, it is difficult to identify the customer's purpose from the information in the inquiry alone. As shown in Figure 4, we therefore aim to identify the purpose accurately by using the question-in-return technology, namely, asking the customer for the missing information and using the information obtained from the response to identify their purpose.

    The flow of purpose identification using the question-in-return technology is shown in Figure 5. First, to obtain candidates for the purpose of the inquiry, an intent-interpretation model*6 is applied to the text comprising the customer's inquiry. The search system specifies the manual that corresponds to the target candidate and, as shown in Figure 6, it uses the manual as reference information to generate the content for a Large Language Model (LLM)*7 to ask further questions. Note that “manual” stated here is the information that operators refer to when they are providing support to customers. When a question-in-return is prepared, a summary of the customer's inquiry is created first. This summary is then used to retrieve manual reference information, and candidate purposes are generated from that information by the LLM. Next, the relevance of each candidate purpose to the inquiry text is calculated, and the customer is asked whether the candidate purpose with the highest relevance matches the purpose of their inquiry. Finally, on the basis of the customer's response, the intent-interpretation model is again used to identify the purpose.

    We tested the question-in-return technology on actual inquiries from customers. Using the experimental set up shown in Figure 7, we evaluated the technology with 247 inquiries sent to NTT DOCOMO's chat support desk. The results of the evaluation confirmed that the introduction of this technology improved the accuracy of purpose identification from 66.80% to 82.18% (Table 1).

    An example of a case in which the introduction of the question-in-return technology resolved a lack of information is shown in Figure 8. What the initial query was asking about was not clear, and the inquires' purpose could not be correctly identified at the initial stage. This example confirms that using the question-in-return technology to supplement the missing information in ambiguous inquiries makes it possible to generate question text that clearly conveys the intent of the customer's inquiry. In this way, the final purpose of the customer's inquiry can be identified in a manner that enables smooth transitions to subsequent responses.

    In contrast, an example of a question-in-return generated when the question-in-return technology did not work properly is shown in Figure 9. In this example, the question-in-return was unrelated to the text of the original inquiry, and the purpose was not identified correctly. Possible causes of this failure are thought to be twofold: (i) the search system could not properly retrieve the manuals related to the query and (ii) the LLM mistakenly gave high relevance scores to the manuals that had low relevance to the query. A future challenge will be to address these causes of failure by improving the prompts*8 for the LLM.

    Figure 4 Example of identifying purpose by introducing question-in-return technology, Figure 5 Processing flow for identifying purpose by using question-in-return technology
    Figure 6 Example of processes and output for generating question-in-return by generative AI, Figure 7 Experimental set up
    Table 1 Results of evaluation, Figure 8 Example response using question-in-return technology (successful example)
    Figure 9 Example response using question-in-return technology (unsuccessful example)
    1. Intent-interpretation model: A machine-learning model that interprets intent. After receiving the inquiry from the customer as text input, it outputs the estimated purpose of the inquiry.
    2. LLM: A large-scale Artificial-Intelligence (AI) model trained by machine learning with a large amount of text data. In this article, it refers to models that have the ability to understand context and generate human-like natural sentences.
    3. Prompt: Instructions or text that the user inputs when using an LLM.

    • 06. Conclusion

  • The features of Communication AI—incorporating DOCOMO's AI ...

    Open

    The features of Communication AI—incorporating DOCOMO's AI technology—and examples of how it can be used were described. By utilizing multiple AI technologies, namely, Speech DX Platform, customer behavior analysis technology, docomo Sense, and question-in-return technology, Communication AI can respond to customers according to their situations and needs and interact with them in such a way as to collect—autonomously—information necessary to resolve their inquiries. It thus enables flexible responses to customer inquiries that are much closer to human responses than possible with conventional automatic response systems. From now onwards, we will continue to verify the effectiveness of Communication AI through proof-of-concept experiments both inside and outside the company and contribute to promoting DX in customer service operations.

  • REFERENCES

    Open
    1. [1] T. Hashimoto et al.: “Improving Customer Satisfaction and Operator Efficiency in Call Centers Using AI—Speech Recognition IVR—,” NTT DOCOMO Technical Journal, Vol. 19, No. 4, pp. 4–10, Apr. 2018.
      PDFhttps://www.docomo.ne.jp/english/binary/pdf/corporate/technology/rd/technical_journal/bn/vol19_4/vol19_4_002en.pdf (PDF format:1,433KB)
    2. [2] M. Chiba et al.: “Speech DX Platform Pioneers Speech Data Usage,” NTT DOCOMO Technical Journal, Vol. 26, No. 1, Apr. 2024.
      https://www.docomo.ne.jp/english/corporate/technology/rd/technical_journal/bn/vol26_1/003.html
    3. [3] A. Chiba et al.: “Customer Behavior Analysis for Support of Customer Procedures,” NTT DOCOMO Technical Journal, Vol. 25, No. 4, Apr. 2024.
      https://www.docomo.ne.jp/english/corporate/technology/rd/technical_journal/bn/vol25_4/002.html
    4. [4] NTT DOCOMO: “Customer-understanding engine: docomo Sense” (in Japanese).
      New windowhttps://ssw.web.docomo.ne.jp/marketing/strengths/sense/

VOL.26 NO.4

Go to top of the page