Article
Authors: Andrew Wilson, Nuria Oliver
ICMI '03: Proceedings of the 5th international conference on Multimodal interfaces
Pages 211 - 218
Published: 05 November 2003 Publication History
Metrics
Total Citations22Total Downloads838Last 12 Months3
Last 6 weeks0
New Citation Alert added!
This alert has been successfully added and will be sent to:
You will be notified whenever a record that you have chosen has been cited.
To manage your alert preferences, click on the button below.
Manage my Alerts
New Citation Alert!
Please log in to your account
Get Access
- Get Access
- References
- Media
- Tables
- Share
Abstract
Perceptual user interfaces promise modes of fluid computer-human interaction that complement the mouse and keyboard, and have been especially motivated in non-desktop scenarios, such as kiosks or smart rooms. Such interfaces, however, have been slow to see use for a variety of reasons, including the computational burden they impose, a lack of robustness outside the laboratory, unreasonable calibration demands, and a shortage of sufficiently compelling applications. We address these difficulties by using a fast stereo vision algorithm for recognizing hand positions and gestures. Our system uses two inexpensive video cameras to extract depth information. This depth information enhances automatic object detection and tracking robustness, and may also be used in applications. We demonstrate the algorithm in combination with speech recognition to perform several basic window management tasks, report on a user study probing the ease of using the system, and discuss the implications of such a system for future user interfaces.
References
[1]
A. Azarbayejani and A. Pentland. Real-time self-calibrating stereo person tracking using 3-D shape estimation from blob features. In Proceedings of 13th ICPR, Vienna, Austria, August 1996. IEEE Computer Society Press.
Digital Library
[2]
F. Berard. The pereptual window: head motion as a new input stream. In IFIP conf. on human-computer interaction, 1999.
[3]
W. Buxton and B. Myers. A study in two-handed input. In Proc. of CHI'86, pages 321--326, 1986.
Digital Library
[4]
T. Darrell, G. Gordon, M. Harville, and J. Woodfill. Integrated person tracking using stereo, color, and pattern detection, 1998.
[5]
S. Franconeri and D. Simons. Moving and looming stimuli capture attention.
[6]
W. T. Freeman and C. Weissman. Television control by hand gestures. In Intl. Workshop on Automatic Face and Gesture Recognition, pages 179--183, 1995.
[7]
Y. Guiard. Assymetric division of labor in human skilled bimanual action: The kinematic chain as a model. Journal of Motor Behavior, 19(4):486--517, 1987.
[8]
B. Horn. Robot Vision. MIT Press, 1988.
Digital Library
[9]
E. Horvitz. Principles of mixed-initiative user interfaces. In Proc. of CHI '99, 1999.
Digital Library
[10]
E. Horvitz and T. Paek. A computational architecture for conversation. In Proc. of the Seventh International Conference on User Modeling, pages 201--210, 1999.
Digital Library
[11]
N. Jojic, B. Brumitt, B. Meyers, and S. Harris. Detecting and estimating pointing gestures in dense disparity maps. In Proceed. of IEEE Intl. Conf. on Automatic Face and Gesture Recognition, 2000.
Digital Library
[12]
J. Jonides and S. Yantis. Uniqueness of abrupt visual onset in capturing attention. Perception and Psychophysics, 43(4):346--354, 1988.
[13]
J. Junqua. The lombard reflex and its role on human listeners and automatic speech recognizer. J. Acoustic Soc. Amer., 93:510--524, 1993.
[14]
T. Kanade. Development of a video-rate stereo machine. In Proc. of ARPA Image Understanding Workshop (IUW'94), pages 549--558, 1994.
[15]
F. Kjeldsen. Visual Interpretation for Hand Gestures as a Practical Interface Modality. PhD thesis, Department of Computer Science, Columbia University, 1997.
Digital Library
[16]
J. A. Long, J. Landay, and L. Rowe. Implications for a gesture design tool. In Proc. CHI'99, pages 40--47, 1999.
Digital Library
[17]
P. Maes, T. Darrell, B. Blumberg, and A. Pentland. The alive system: wireless, full-body interaction with autonomous agents. ACM Multimedia Systems, Special Issue on Multimedia and Multisensory Virutal Worlds, 1996.
Digital Library
[18]
C. Mignot, C. Valot, and N. Carbonell. An experimental study of future 'natural' multimodal human-computer interaction. In Proc. of INTERCHI93, pages 67--68, 1993.
Digital Library
[19]
M. Moyle and A. co*ckburn. Gesture navigation: An alternative 'back' for the future. In Proc. of CHI02, pages 822--823, 2002.
Digital Library
[20]
A. Oh, H. Fox, M. Van Kleek, A. Adler, K. Gajos, L. Morency, and T. Darrell. Evaluating look-to-talk: a gaze-aware interface in a collaborative environment. In CHI'02, pages 650--651, 2002.
Digital Library
[21]
A. Peleg and U. Weiser. Mmx technology extension to the intel architecture. IEEE Micro, 16(4):42--50, 1996.
Digital Library
[22]
A. Worden, N. Walker, K. Bharat, and S. Hudson. Making computers easier for older adults to use: area cursors and sticky icons. In Proc. of Conf. on Human factors in computing systems, pages 266--271, 1997.
Digital Library
[23]
I. Yoda and K. Sakaue. Utilization of stereo disparity and optical flow information for human interaction. In Proc. of ICCV'98, 1998.
Digital Library
[24]
Z. Zhang. A flexible new technique for camera calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11):1330--1334, 2000
Digital Library
Cited By
View all
- Vuletic TMcTeague CCampbell GHay LGrealy M(2023)Effects of activity time limitation on gesture elicitation for form creationJournal of Engineering Design10.1080/09544828.2023.227177334:11(963-985)Online publication date: 26-Oct-2023
- Lian SHu WWang K(2014)Automatic user state recognition for hand gesture based low-cost television control systemIEEE Transactions on Consumer Electronics10.1109/TCE.2014.678093260:1(107-115)Online publication date: Feb-2014
- Dhule CNagrare T(2014)Computer Vision Based Human-Computer Interaction Using Color Detection TechniquesProceedings of the 2014 Fourth International Conference on Communication Systems and Network Technologies10.1109/CSNT.2014.192(934-938)Online publication date: 7-Apr-2014
https://dl.acm.org/doi/10.1109/CSNT.2014.192
- Show More Cited By
Index Terms
GWindows: robust stereo vision for gesture-based control of windows
Computing methodologies
Artificial intelligence
Computer vision
Computer vision tasks
Human-centered computing
Human computer interaction (HCI)
Interaction devices
Touch screens
Recommendations
- TouchLight: an imaging touch screen and display for gesture-based interaction
ICMI '04: Proceedings of the 6th international conference on Multimodal interfaces
A novel touch screen technology is presented. TouchLight uses simple image processing techniques to combine the output of two video cameras placed behind a semi-transparent plane in front of the user. The resulting image shows objects that are on the ...
Read More
- Occlusion based interaction methods for tangible augmented reality environments
VRCAI '04: Proceedings of the 2004 ACM SIGGRAPH international conference on Virtual Reality continuum and its applications in industry
Traditional Tangible Augmented Reality (Tangible AR) interfaces combine a mixture of tangible user interface and augmented reality technology, complementing each other for novel interaction methods and real world anchored visualization. However, well ...
Read More
- Tablets, tabletops, and smartphones: cross-platform comparisons of children’s touchscreen interactions
ICMI '17: Proceedings of the 19th ACM International Conference on Multimodal Interaction
The proliferation of smartphones and tablets has increased children’s access to and usage of touchscreen devices. Prior work on smartphones has shown that children’s touch interactions differ from adults’. However, larger screen devices like tablets ...
Read More
Comments
Information & Contributors
Information
Published In
ICMI '03: Proceedings of the 5th international conference on Multimodal interfaces
November 2003
318 pages
ISBN:1581136218
DOI:10.1145/958432
- Conference Chair:
- Sharon Oviatt
Oregon Health & Science University
, - Program Chairs:
- Trevor Darrell
Massachusetts Institute of Technology
, - Mark Maybury
MITRE
, - Wolfgang Wahlster
DFKI, Germany
Copyright © 2003 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [emailprotected]
Sponsors
- ACM: Association for Computing Machinery
- SIGCHI: ACM Special Interest Group on Computer-Human Interaction
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 05 November 2003
Permissions
Request permissions for this article.
Check for updates
Author Tags
- computer human interaction
- computer vision
- gesture recognition
- speech recognition
Qualifiers
- Article
Conference
ICMI-PUI03
Sponsor:
- ACM
- SIGCHI
ICMI-PUI03: International Conference on Multimodal User Interfaces
November 5 - 7, 2003
British Columbia, Vancouver, Canada
Acceptance Rates
ICMI '03 Paper Acceptance Rate 45 of 130 submissions, 35%;
Overall Acceptance Rate 453 of 1,080 submissions, 42%
More
Contributors
Other Metrics
View Article Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations
22
Total Citations
838
Total Downloads
- Downloads (Last 12 months)3
- Downloads (Last 6 weeks)0
Reflects downloads up to 09 Aug 2024
Other Metrics
View Author Metrics
Citations
Cited By
View all
- Vuletic TMcTeague CCampbell GHay LGrealy M(2023)Effects of activity time limitation on gesture elicitation for form creationJournal of Engineering Design10.1080/09544828.2023.227177334:11(963-985)Online publication date: 26-Oct-2023
- Lian SHu WWang K(2014)Automatic user state recognition for hand gesture based low-cost television control systemIEEE Transactions on Consumer Electronics10.1109/TCE.2014.678093260:1(107-115)Online publication date: Feb-2014
- Dhule CNagrare T(2014)Computer Vision Based Human-Computer Interaction Using Color Detection TechniquesProceedings of the 2014 Fourth International Conference on Communication Systems and Network Technologies10.1109/CSNT.2014.192(934-938)Online publication date: 7-Apr-2014
https://dl.acm.org/doi/10.1109/CSNT.2014.192
- Jeong SJin JSong TKwon KJeon J(2012)Single-camera dedicated television control system using gesture drawingIEEE Transactions on Consumer Electronics10.1109/TCE.2012.641497758:4(1129-1137)Online publication date: Nov-2012
- Gadea CIonescu BIonescu DIslam SSolomon B(2012)Finger-based gesture control of a collaborative online workspace2012 7th IEEE International Symposium on Applied Computational Intelligence and Informatics (SACI)10.1109/SACI.2012.6249973(41-46)Online publication date: May-2012
- Jeong SSong TKwon KJeon J(2012)TV remote control using human hand motion based on optical flow systemProceedings of the 12th international conference on Computational Science and Its Applications - Volume Part III10.1007/978-3-642-31137-6_24(311-323)Online publication date: 18-Jun-2012
https://dl.acm.org/doi/10.1007/978-3-642-31137-6_24
- Prasad SSawant AShettigar RKhokale RSinha SMishra B(2011)iCruxProceedings of the International Conference & Workshop on Emerging Trends in Technology10.1145/1980022.1980162(646-651)Online publication date: 25-Feb-2011
https://dl.acm.org/doi/10.1145/1980022.1980162
- Ni TBowman DNorth CMcMahan R(2011)Design and evaluation of freehand menu selection interfaces using tilt and pinch gesturesInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2011.05.00169:9(551-562)Online publication date: 1-Aug-2011
https://dl.acm.org/doi/10.1016/j.ijhcs.2011.05.001
- Gustafson SBierwirth DBaudisch PPerlin KCzerwinski MMiller R(2010)Imaginary interfacesProceedings of the 23nd annual ACM symposium on User interface software and technology10.1145/1866029.1866033(3-12)Online publication date: 3-Oct-2010
https://dl.acm.org/doi/10.1145/1866029.1866033
- Pan GWu JZhang DWu ZYang YLi S(2010)GeeAirPersonal and Ubiquitous Computing10.1007/s00779-010-0287-714:8(723-735)Online publication date: 1-Dec-2010
https://dl.acm.org/doi/10.1007/s00779-010-0287-7
- Show More Cited By
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in
Full Access
Get this Publication
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderMedia
Figures
Other
Tables
