A system for field performance assessment of a speech recognition based telephone service

A data collection and analysis system has been designed, developed, and deployed as an integral part of VoiceDialing service, in order to measure the "true" recognition performance of the service in the field and other statistics, and to identify potential areas of improvement. The data collection system collects relevant training, recognition data and timing information, in addition to utterance recordings, on selected lines. Complementing the collection system, the data analysis system provides the capability to study, analyze and classify the collected data, to display and listen to collected utterances, generate statistics and enter analysis results in a database. A graphical user interface enhances the analysis system providing easy access to the database, and simplifying the analysis process. Methods used in measuring field recognition performance are presented, along with field results. A system, using field collected and analyzed data to test recognizers is proposed.<>