Enter the maze

Interacting with all the senses: Multimodal Design:

Touch it, feel it, hear it!

This area is devoted to showcasing technology designed to help us experience the world using computers, but through different senses (cross modal design) or more than one sense at once (multimodal design). Multimodal design is also a great way to support people with limited senses who can't hear, or see, or smell, or taste or touch. Some people find lots of sounds, sights, textures, tastes just too much, and we look at how computer science might one day help them too.

With the HapticWave you can hear and feel a sound wave, with the oPhone you smell a coffee and see a coffee even when there’s no coffee anywhere near, the baby birthing monitor lets mums know when to push based on a beep and a graph. We also focus on a research project called DePIC (‘Design Patterns for Inclusive Collaboration’) that is looking for the best ways of using combinations of senses. It is finding the best ways to help people with different senses available work together.

Issue 19 of the cs4fn magazine is all about multimodal design. Download the pdf now. UK teachers can subscribe for free copies of the magazine - just fill out the request form

Sights, sounds...and smells

Blind driver filches funky feely sound machine!

A Blue SoundWave: Copyright www.istockphoto.com  26316636 Haptic Wave: Touchning sound waves

Dreams, sticky tape and pass me a soldering iron!

Touching a sound wave: copyright www.istockphoto.com 61740466 Haptic Wave: How it ws designed

Beep, beep mummy here I come!

A new born baby held by doctors: copyright www.istockphoto.com 12676907 Babies to be saying when to push

The Tactful Watch

Clock hand Telling the time by touch

Wanting to scream and scream and scream

Hands reaching out for a scared woman: copyright www.istockphoto.com 44661334 Designing for people with autism

On from Smell-O-Vision

Fish Eyes / Heads: copyright www.istockphoto.com 9642800 Smelling things remotely

Digital lollipop: no calories, just electronics!

Lots of multicoloured sweets: copyright www.istockphoto.com 17013855 Tasting things remotely

Can you feel it?

nose and glowing sinus: copyright www.istockphoto.com 41361116 Feeling surgery

Putting your hand up a cow's bottom

A suspicious cow's bottom: copyright www.istockphoto.com 54014442 Haptics for a vet and a cow

Magic: Holding a card to your heart

A scribbled heart: copyright www.istockphoto.com 21696662 A magic trick and multimodal design

Designing robots that care

A robot hand holding a smartphone showing a smiley: copyright www.istockphoto.com 50529790 How to make robots show empathy

Scratching when next board

Multicoloured Chalk: copyright www.istockphoto.com 20441872 An experiment about your senses

Shh! Can you hear that diagram

Node and Link Diagram with music: From www.istockphoto.com The blind working on diagrams

A soundly engineered waterfall

The Cascade at Chatsworth A cascade of water designed to be heard

This area and much of the research described in it has been supported by EPSRC. This issue and/or the research in it has been supported by the EPSRC project : DePIC on research grants (EP/J017205/1 and EP/J018120/1). It has also been supported by the Department of Education and Mayor of London through LSEF project teachinglondoncomputing.org and Google.