Using the Wizard-of-Oz technique
       to gather data for natural language interfaces

                       Paul Mc Kevitt

               Computing Research Laboratory
                   Dept. 3CRL, Box 30001
                New Mexico State University
              Las Cruces, NM 88003-0001, USA.

              E-mail: INTERNET: paul@nmsu.edu
                   Fax: (+1 505)-646-6218
               Phone: (+1 505)-646-5109/5466

                          ABSTRACT

     One of the most pressing problems in developing natural
language interfaces is scaling up the system. There are many
accepted theories  and  techniques  for  developing  natural
language  interfaces  but  there  is  a  bottleneck  in  the
acquisition  of  data  for  any  application  domain.   Many
natural  language  interfaces  fail because they do not have
enough knowledge about language use in the domain.  We  call
this   the   brittleness  problem.   It  has  recently  been
discovered by empirical studies that natural  language  sys-
tems fail 45% of the time because of inadequate knowledge of
the ongoing dialogue. We call this the reduced context prob-
lem.


     There are many ways in which data can be gathered for a
domain  and  it  is argued here that a useful methodology is
the Wizard-of-Oz technique. The  Wizard-of-Oz  technique  is
one  where  subjects  interact with a computer through typed
dialogue and are told that they are conversing with the com-
puter.   Subjects  utterances  are  sent  to another monitor
where a ``Wizard'' sends back a reply to the  subject  moni-
tor.   Results are shown for a Wizard-of-Oz experiment which
was conducted in the domain of computer  operating  systems.
The results verify that the Wizard-of-Oz technique is indeed
useful in the development of  dialogue  systems.  Also,  the
technique  possibly  provides  a solution to the brittleness
and reduced context problems.


     It is possible that the wizard technique can be used in
conjunction with other mechanisms to formulate automatic and
iterative methods for the development  of  natural  language
dialogue systems, rather than the subjective construction of
such models.