Establishment of Confidence Thresholds for Interactive Voice Response Systems Using ROC Analysis

Authors

Oredola A. Soluade, Iona College

Abstract

An Interactive Voice Response (IVR) System is a platform for man-machine interaction. It is used for collecting and analyzing human voices so as to provide the desired response. The algorithm for collecting these utterances, analyzing them correctly, and providing the desired response to a caller, has been studied extensively (Allen, 1995). Whenever one calls most large organizations, their initial encounter is with a machine that will prompt the caller for their intent. Usually, such machines will give you options to choose from (Directed Dialog), or it may ask for your input (Open Dialog). This paper focuses on Open Dialog where the caller is free to indicate their intent. The problem is that the Voice Recognizer may misinterpret the caller intent; thereby providing the caller with the wrong information. This is because the recognizer has a threshold for recognizing any utterance, and traverses the part of the Call Flow that corresponds to what the engine recognizes. This threshold can be calibrated for optimal performance by undertaking a statistical analysis of a random sample of utterances, and based on the result, set the threshold that will be used to discriminate between caller utterances. The criteria that are used for establishing this threshold include, among others, Sensitivity, Accuracy and Specificity. The optimal threshold will be the one that optimizes the majority of these parameters.

Recommended Citation

Soluade, Oredola A. (2010) "Establishment of Confidence Thresholds for Interactive Voice Response Systems Using ROC Analysis," Communications of the IIMA: Vol. 10: Iss. 2, Article 4.
DOI: https://doi.org/10.58729/1941-6687.1135
Available at: https://scholarworks.lib.csusb.edu/ciima/vol10/iss2/4