Automating Speech Analysis

"MATLAB provides powerful modeling and analytic tools, and it allows us to visualize what’s happening at each step in our analysis."

William Liggett

William Liggett retired from his role as a behavioral scientist at IBM several years ago, but he has retained his lifelong interest in the subject. Dr. Liggett, who has a Ph.D. in applied social psychology, uses MATLAB Home to process and analyze data on human speech. While speech analysis can provide researchers with important clues about a person’s emotional or physiological state, collecting the data is a time-consuming process. Dr. Liggett’s ultimate goal is to develop a MATLAB model to automate speech analysis.

I first became interested in using speech to measure an individual’s physiological activation while working on my Ph.D. at New York University. However, I found the process of recording, coding, and analyzing speech tremendously slow and labor-intensive. I realized that speech analysis was a potentially powerful tool, but without automation, it would never be commonly used in the behavioral sciences. I’ve wanted to explore methods of recording speech digitally and automating the coding rules ever since.

The Work

Our work involves analyzing microtremors in the larynx to determine an individual’s level of physiological activation. All voluntary muscles in the human body, including those associated with the larynx, tend to have a tremor of about 8-12 Hz when relaxed. Researchers believe this to be the result of a feedback loop that prepares muscles for rapid movement. This microtremor disappears when the muscle is tense. We are analyzing speech for evidence of the microtremor appearing and disappearing as a person speaks. We will use this as an indicator of how tense or alert that person is.

My colleague Dr. Phillip Gilley, a neuroscientist, constructed an interface in MATLAB that manages all the raw audio data. Huy Le, an engineering student who has since graduated, wrote much of the original code. We chose MATLAB because we needed a program that offers tools for both processing data and applying coding logic. With MATLAB we can document and display the results of applying filters, which avoids creating a “black box” process where users must simply trust that the analysis is doing what we claim it does.

The Results

We’ve made huge strides in our work, but there’s still a lot to be done. For example, we are analyzing speech samples for evidence of physiological changes and then validating our results using simultaneous measurements of other physiological responses, such as heart rate. We are currently analyzing the frequency modulation (FM) of speech and comparing it with the amplitude modulation (AM) approach to look for differences between the two. All of this work can be done in MATLAB.

The Future

Our long-term goal is to make speech analysis a reliable, efficient, and powerful research tool in the behavioral sciences. We also want to support the many practical applications of speech analysis, such as psychotherapy, pilot workload monitoring, and computer-based instruction. In the immediate future, we plan to analyze speech for evidence of physiological changes and then validate our results with simultaneous measurements of other responses to change. We want to apply our techniques to databases containing emotional speech, such as emergency calls.

Learn More

Project made with

  • MATLAB ${product:ML use:home}

Expand the project with digital signal processing products.

Share your ideas and projects with the MATLAB community.

File Exchange

Explore how others are using MATLAB and Simulink products.

user story