Shogo Ando

Hello everyone.

There are many products that incorporate AmiVoice, some of which incorporate multiple AmiVoice engines, allowing you to choose the appropriate one depending on your needs.

For example, AmiVoice Cloud Platform (ACP), a service for developers, has five conversation engines and six voice input engines for Japanese, as shown in the image below, which can be used depending on the purpose. (Information as of September 2021)

Incidentally, ACP calls these engines "domain-specific engines."*1. 

 Further information and updates can be found below.

AmiVoice API Engine Types and Prices

AmiVoice API is an API that allows you to implement AI speech recognition technology, which has 20 years of experience and is the No. 1 market share in Japan, into your services and products. You can use high-quality speech recognition at a low price.

acp.amivoice.com

I think many people think of speech recognition as something similar to "kana-kanji conversion" like Microsoft IME, so it may be hard to imagine it as being like this domain-specific engine, where you can choose from multiple engines to find the one that's right for your purpose.

This time, I will explain why AmiVoice offers multiple engines, such as this domain-specific engine.

Conclusion: AmiVoice achieves outstanding accuracy by specializing in the domain

Let's start with the conclusion. Because speech recognition deals with the ambiguous concept of "speech," there is an inevitable problem of misrecognition. In order to reduce these misrecognitions and achieve highly accurate speech recognition, AmiVoice has devised a system that provides speech recognition engines specialized for each application.

Difficulties of speech recognition and the difference with IME kana-kanji conversion

I would like to explain why domain specialization improves speech recognition accuracy, but before that, I will take a slight detour and talk about why speech recognition is difficult.

I'm sure many of you have had the experience of using voice recognition and seeing something completely different from what you intended to say. The reason this happens is that voice input is not as clear as keyboard input and is very ambiguous. With keyboard input, you can confirm each character you input, but this is very difficult with voice input.

Let me explain with a little example. For example, to type "hello" using a keyboard (in romaji), you would press the keys "konnnichiha" in order 11 times. There are differences, such as "chi" can be "chi" or "ti," but since anyone will press roughly the same keys in roughly the same order, mistakes are unlikely to occur.

So what happens with voice? Take a look below. This is an audio waveform recorded with a microphone of me saying "Hello." The horizontal axis is time, and the vertical axis shows the air vibrations (changes in air pressure) caused by my voice.

This waveform changes slightly every time you speak, and if your voice is not in good shape or you speak quickly, the waveform will be even more different. It will also change when a different person speaks, and it will be even more different if there is noise. In short, unlike a keyboard, with voice recognition, different data is input each time, making it very difficult to identify characters (kana).*2.

Therefore, rather than identifying each kana character one by one, we analyze "which kana does this waveform resemble?" and explore various possibilities. Furthermore, we look at patterns in kana combinations to explore "which word is most likely?"

Although keyboard input and voice recognition may look similar at first glance, the processes they perform are quite different. Roughly speaking, they can be seen in the diagram below.

Now that we have roughly explained the difficulties of speech recognition, we will explain how these difficulties are overcome. The key point in speech recognition is that it mainly uses two types of data: an acoustic model and a language model.*3.

About "acoustic models" in speech recognition

The acoustic model is a model that can recognize sounds such as "a," "i," and "u."*4...etc. It is used to compare input speech with the acoustic model and estimate "which sound is most likely to be this?"

AmiVoice absorbs the fact that waveforms vary from person to person by training its acoustic model on the sounds of many different people. It also absorbs differences in noise and recording environments by training it on various types of noise and sounds from various microphones. Training a wide variety of sounds allows for the creation of a general-purpose acoustic model that can handle a greater number of patterns.

However, there are cases where it is not necessary to train all sounds. For example, ACP provides specialized acoustic models for the following two types:*5.

• Conversational Acoustic Model
  It is intended for person-to-person conversation.

• Acoustic model for "voice input"
  It is intended for use in voice input to machines.

When inputting voice to a machine, many users speak clearly and crisply, but when talking to a person, they tend to speak in a more casual manner. ACP provides acoustic models specialized to match these differences in speaking style, achieving higher speech recognition accuracy.

About "language models" in speech recognition

As mentioned above, speech recognition analyzes the spoken voice to determine "which kana is most likely," and searches for "which word is most likely" from among the kana combination patterns. However, if this is all done, all kinds of unrelated words may jump out, and the content of the sentence in the speech recognition results may become incoherent.

Therefore, speech recognition uses data called a "language model" to determine which words are appropriate to be spoken.

For example, after saying "What will the weather be like tomorrow?", you can predict that there is a high probability that the next response will be "sunny," "rainy," "cloudy," "good," "bad," or "don't know," while "curry" or "ramen" are less likely to be mentioned. The data used to make such predictions is a "language model."

When creating a language model, a large amount of text is used to calculate the likelihood of connections between words, and generally the more text there is, the more accurately the likelihood can be calculated, resulting in a higher quality language model. Therefore, a common approach is to collect a variety of texts and create a single, huge language model.

However, AmiVoice does not just use a single large language model, but also develops many language models that are domain-specific depending on the situation the user is speaking in. This is because what users say varies depending on the situation, so using models that are specialized for that situation will result in higher speech recognition accuracy.

How domain specialization can enable highly accurate speech recognition

In speech recognition, the two pieces of data, the acoustic model and the language model, explained so far, are used to output the most likely speech recognition result for ambiguous speech input, but there are two ways of thinking about this.

• By training acoustic and language models with a variety of voice and text data, it becomes possible to recognize a wide range of speaking styles and content.
• The purpose is limited, and the acoustic model and language model are trained using limited voice data and text data within that range.

Neither of these two is better or worse, and both have their advantages and disadvantages. If you train a system with a wide range of data, it will be able to cover almost anything, but the wider range will inevitably make it more prone to misrecognition. Also, if you limit the application, it will only be able to recognize speech within that range, but since unnecessary words will not appear, it will be possible to achieve very high speech recognition accuracy.

Generally, most of the voice recognition engines available on cloud services are trained on a wide range of data. AmiVoice provides engines trained on this wide range of data, but we also offer many engines trained for specific purposes.

To summarize, the table below shows the results.

Domain-specific engines that can be used with ACP

There are countless types of acoustic models and language models provided by AmiVoice. For example, in ACP, https://acp.amivoice.com/main/plan/ You can choose from the options available.

First, there are two main types: "conversation" and "voice input," which differ mainly in their acoustic models. Furthermore, even within the same acoustic model, there are other types such as "general-purpose," "medical," and "pharmaceutical," which differ mainly in their language models.*6We hope you will choose the most suitable one depending on your purpose.

Conclusion

This time, we talked about how speech recognition is prone to misrecognition because it deals with the ambiguity of speech, how acoustic models and language models are used to solve this problem, and how AmiVoice achieves high accuracy by using these acoustic models and language models depending on the application.

ACP also offers several types of speech recognition engines that combine these acoustic and language models, and we hope to continue expanding our lineup in the future. If you have any requests, such as "I want a speech recognition engine specialized for this kind of application," please feel free to contact us via comments or ACP inquiries. We cannot guarantee that we will be able to accommodate your request, but we will definitely consider it.