Desktop speech products specific evaluation criteria

OUR EXPERIENCES with desktop speech recognition products have led us to look for several specific items as instances of bugs, usability errors or limitations on expandability in these products. This list changes from time to time as clients and testers discover new items of importance.

Installation:

  • no gratuitous images
  • handles out of sequence user responses
  • doesn't leave temporary installation files lying around
  • self-contained - does not require additional installation of external components from other vendors
  • does not conflict with other products from major vendors which have the same purpose
  • keeps its program files in its own directory rather than Windows\System
  • Enrollment process:

  • no gratuitous images
  • handles out of sequence responses
  • user can re-dictate misspoken phrases
  • multiple choices for training text
  • Training process:

  • speed of learning new words and phrases
  • handles special words like "3Com" and "Y2K"
  • can delete a mistrained word from vocabulary
  • can train alpha characters used for spelling
  • Dictation:

  • properly capitalizes beginning of sentences for insertions
  • status of recognizer shown - listening, processing commands, processing text, overrun
  • messages not truncated on status bar
  • availability of history list of commands and phrases
  • multilevel undo
  • availability of "computer attention word" to force next phrase to be treated as a command
  • availability of an "escape" word to prevent command interpretation of next phrase
  • right click on highlighted phrase brings up correction and training verbs
  • existence of playback facility
  • playback facility handles insertions/deletions correctly
  • playback facility doesn't drop phrases
  • accuracy of single word dictation, i.e. a one word phrase surrounded by pause
  • no bizarre capitalization errors
  • assignable key to toggle microphone on and off
  • "Select <text>" command works in multiple windows
  • Text to speech performance:

  • good phrasing, pitch and tempo
  • trainable pronunciation for new words
  • easy to stop and continue while audio proofreading
  • works in long documents
  • works outside of native word processor
  • Macro capability:
  • will accept parameters to the macro
  • facility to copy and paste from other macros
  • facility to make an alias for any official command including "wake up" and "go to sleep " and "computer attention word"
  • macros can handle combinations of keystrokes, program initiation, reference to other macros
  • keystroke capture facility
  • export/import macros by user or global context
  • internal macro format is text editable
  • macro can be made aware of current window focus in order to behave differently
  • Transcription process:

  • short time to change between transcription device input and regular user microphone input
  • doesn't drop phrases without warning
  • ability to abort from keyboard a transcription translation that is going badly wrong
  • transcription errors trainable
  • Quality of manuals and on line help:

  • no "tips" or "hints" in manuals
  • no manual references to "see on-line help for more details"
  • help available in all contexts with context sensitive command list
  •  

    More on bug testing...
    More on usability...
    More on expandability...


     
     
     
     
     
     
     

     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     




    You are here::
    home->testing->desktop


    Technology Review
    What's Out There
    Executive View
    Developer View
    FAQ
    At This Site
    Who We Are
    Consulting Services

    Other Stuff
    How We Test
      Bugs
      Usability
      Expandability
    Copyright © 1999-2012 eWyzard Inc.
    Comments? Questions? Contact us.

    Page Last Updated: 05/20/00