based on the notes from Bert Bongers during the development of Project Txoom (based on Brussels version, 9 September 2002). edited/annotated by Nik Gaffney

(loosly) related topics: Wireless Network, Mobile Computing, Sensor Technology, HCI, Haptic Feedback, Motion Tracking, etc+

In this version of the report an overview (Background chapter) is given of the requirements for hardware for developing human interfaces for computer technology, and a proposal is worked out for such a system (Human Interface System chapter) which can be applied in a variety of user interface applications in the arts and other interaction research. A section describing technical scenarios of appoaching such an ideal system with a minimum of engineering efforts in order to meet the time constraints for the current phase of Txoom, using of the shelf components and OEM solutions, has been worked out based on the presentation and discussions during the workshop in Visby. Case studies of relevant existing projects and technical systems were presented during the workshop, and are described in this report, as well as a more detailed overview and description of existing sensor interfaces commonly used in these situations. A selection of sensors relevant for the current application (Txoom) has been discussed during the workshop, and is further documented here.

The interfaces through which humans communicate with and control computers have been a severe bottleneck for a long time. The mainstream computer interface paradigm (mouse, keyboard and windows) is certainly not adequate, and generally there is a severe mismatch between the human's capabilites (both in perception as well as in action) and the computer's input and output capabilities. In diverse research fields such as musical instruments and Virtual Environments research has been carried out that can lead to the emergence of new interaction paradigms.

The computer system needs input from the real world sensors, and to address the senses of the users through output devices displays. Inside the computer system sensor data has to be analysed, behaviours are programmed, worlds are generated, so that a closed loop between the input of the system and the output is established, and the system can interact with the user(s) as shown in the diagram below

From “Physical Interfaces in the Electronic Arts, Interaction Theory and Interfacing Techniques for Real-time Performance”, A.J. Bongers, chapter for the IRCAM e-book “Trends in Gestural Control of Music”, edited by M. Wanderley and M. Battier.

Potential movements of any point in space, such as a body or parts of the body, can be described in six Degrees of Freedom (DoF) in three-dimensional space; the lateral movements and rotational movements along the three axes.

Human movement can be measured for each degree of freedom in a range, with a certain_precision_, and with a certain _haptic feedback_.

Movement can be categorised in three ranges: the intimate (from 0, isometric pressure to a few centimeters), the bodysphere (within range of the body, from a few centimeters to a meter), spatial (the position of the body in architectural space, in practice from 1 - 10 meters)

See “The Physical Interface Design Space”, paper in progress by A.J. Bongers, and for a short summary “A Structured Instrument Design Approach: The Video-Organ”, paper by A.J.Bongers and Y.C.Harris for the NIME conference, Dublin 2002.

Human movement is guided by haptic feedback, such as the internal kinaesthetic feedback or externally by perceiving forces from inertia or static objects. In the case of active haptic feedback, the system displays forces or vibrations (through motors etc..) that can be felt by the human.

In order to sense the intimate range of human movement, the system of an interactive space has to be extended with on-body sensors in most cases (especially when measuring biosignals such as EMG or heart rate). Bodysphere and spatial movements can be tracked by on-body sensors as well as by underline in-space sensors, such as cameras and movement trackers that monitor the people.

Modalities are communication channels between humans or, as in this case, between humans and computer systems. These channels are usually described reflecting the human senses and actions, for instance there are visual modalities (text, colours, moving images), auditory modalities (speech, music) and tactual modalities (touch). Note that within each of these sensory modalities separate communication channels can be described. Machine input modalities (through sensors) register human output modalities (actions), and machine output modalities (through displays) address the human senses

The standard human interface options through built-in hardware of most computers and systems are limited, additional hardware is required to enable a rich interaction. There are a few solutions in hardware interfaces, an overview is given below, but they all have there limitations in a variety of issues such as number of I/O ports, speed, resolution, price and availability. This is due to the fact that they are generally developed with only one or a few application domains in mind. The solution proposed and described in this report aims to fit all the needs of the main research areas in the field of Human-Computer Interaction.

Overview of hardware interfaces: STEIM SensorLab, Infusion iCube, IRCAM Atomic Pro, Mindtel Thing box, Bodysynth, U. York MIDI Creator, U. Pisa 'Smart' MIDI Interface, La Kitchen Kroonde, DIEM digital dance system, Beehive ADB box, PAiA MIDI Brain, CNMAT Connectivity Processor, various others.

There are many different research areas involved in the development of better human interfaces. The resulting technologies often overlap, but still bare many characteristics of the original application domain or research strand, both in the arts as well as human sciences. For instance new musical instruments, Virtual Reality, Ubiquitous Computing, Wearable Computing, Dance, Installations, networking, measurement of human behaviour.

From the oldest sensor based instrument, the radio-antennae equiped Theremin (invented around 1917) to the more recent inventions by pioneers such as Robert Moog, Don Buchla and Michel Waisvisz, musical instruments have played an important role in the development of fine and sensitive human interfaces. A variety of sensors are mounted on elements worn, such as a glove (Laetitia Sonami in Oakland, CA, and several others at the Institute for Sonology in The Hague, Netherlands) or the wooden and metal frames of Michel Waisvisz' “Hands”. These signals are read by a box worn on the performer's back, typically the STEIM SensorLab?, which communicates with any assembly of synthesizers, samplers and computers (possibly off stage) via the MIDI protocol. The cable that connects performer to the system also carries the power. In other situations traditional instruments are extended with sensors, such as the Meta-Trumpet of Jonathan Impett (UK) or the Cello++ of Frances-Marie Uitti (US/NL).

The sensor interface in these situations is small enough to be worn, sometimes integrated with the instrument, and most performers can live with the cable though it is not perceived as optimal. There is usually no hardware support for actuators. All hardware has to be light enough to be carried yet strong enough to survive on stage - still, like any traditional instrument, it is often accepted that they are fragile and only handled by the player. Connectors have to be chosen (if used at all) to be strong, light, and durable.

The sensor interface hardware has to be extremely low latency, real time in musical defenitions.

The sensor interface in the case of interactive installations is usually out of sight or reach of the audience, sensors are connected with longer wires. Connections are often only made once when the installation is set up, and most of the system remains stationary during use. The elements that the audience reaches however have to be extremely rugged, including all sensors and control elements.

A certain latency of the sensor interface hardware is often tolerated, especially when visuals are controlled.

To analyse human motion, action and perception parameters hardware interfaces are used for sensor input and actuator output. For instance for low level psychometric experiments repsonse times can be measured which requires a high timing precision.

See “Investigating the Parallel Use of the Sense of Touch in Multimodal HCI”, MSs Thesis A.J. Bongers, 1999

“Threshold Levels of Tactual Sensitivity of Motion-Impaired Users”, A.J. Bongers University of Cambridge internal report.

In cases where human movement is studied, sensor systems often have to be carried but analysis of the data can be and often is done later so latency is not an issue.

4th International Conference on Methods and Techniques in Behavioral Research, http://www.noldus.com/mb2002

To improve the interaction and control of mainstream computers or special applications a lot of research is carried out to investigate novel ways of interaction. Test set-ups are often partly simulated (or entirely - the Wizard of Oz technique) in order to focus on the user experience and observations. A variety of technologies and techniques are applied in the research phase of these developments, and final products are mass produced so hardware is developed specifically for the application.For instance a mouse equipped with gyroscopic sensors that works in mid air (Gyropoint / Philips), a game devices that sense movement using accelerometers (MacAlly Airstick, Logitech Wingman (Using an ADXL202!), devices that are now on the market.

[…]

Over the last years within the research field of Human-Computer Interaction a specialised field of research has emerged that focusess on wearable computers. MIT has been engaged in ongoing research http://www.media.mit.edu/wearables/

To make a computer wearable is not such a big deal, the hardware is now small enough to fit in a pocket or to be worn on a belt. The main interest of the field is the problem of the human interface with such a system. Output solutions are tiny visual displays worn on spectacles, audio display of information through headphones, somtimes tactile displays, and on the input side often speech is used (which has similar problems as with mobile phones!), gesture sensors and portable keyboards (chording keyboards such as the Twiddler). The computers are usually connected with networks through wireless ethernet (802.11).

In the Txoom application it is often not necessary to wear the whole computer as the computing powerand processing is distributed over the rest of the system. So all that is needed is to sense the user's movements and perhaps give local feedback (on any modality), and a wireless link with the system to exchange this information.

With computers becoming smaller and networked, they are dissapearing from out everyday life altogether - only the interface remains.

It has been described as the the 'third trend in computing', after the mainframe and PC models. http://www.ubiq.com/hypertext/weiser/UbiHome.html

The term was coined by pioneer Mark Weiser in the late eighties at Xerox Parc.

The audience experience in a responsive environment such as Txoom is not about the computers either, they are in the background only manifest themselves through displays and are controlled through the sensor systems.

Computer generated environments that interact with the people that are in it, can be called Interactivated Spaces, which involves research and developments from many other fields. In order to sense the human actions, and to address the human senses, interface systems are a combination of on-body (wearable) and in-space (ubiquitous).

The specifications for a hardware interface system for this application area is a mixture of many of the other catagories described above. It has to be real-time, low latency, it has to be wearable by any user and therefore robust, lightweight and wireless, in combination with remote sensing, so the system has to be modular. In addition to in-space audio (loudspeakers or PA systems) and visuals (video projections), personal address may be required especially in the case of actuators that address for instance the human sense of touch which is in most cases inherentlyin the personal range.

Computers (including their subsystems such as graphic cards, mass storage etc.) are getting faster and more powerful. The system proposed in this report is only needed to fill in the gap for extending the user interface.

In contrast to the field of Wearable Computing, in most of the applications the user's electronics become part of a larger system of computers, peripherals, networks (even Internet) and most of the computing / processing / mapping etc. can take place in these stationary areas of the system. The only part that has to be worn is the actual interface, in addition to stationary (in-space) sensing. In cases where it is felt necessary to wear the whole system, wearable computer solutions (PDA's, or OEM computers such as the Cerf board) can be used and interfaced in thesame way as a stationary system.

In order to suit the wide range of needs as indicated in the previous chapter, it is important to set up the Human Interface System in a modular way. The modules, described in this chapter, communicate with each other through a bus system that can in some cases be wireless.

To communicate with the real world, computer systems have to transduce physical quantities into electrical signals and vice versa. The sense organs of a computer system are called \textit{sensors}, and the transducers through which the system can act or make things happen in the real world are called actuators.

In the sections below these elements will be further described.

In principle any physical quantity (eg. light, force, air speed) can be transduced into electricity, though some things are more easy to sense (and therefore cheaper) then others. Anything that humans can sense can be sensed by the computer, and often at a greater range (ultrasound, infrared light) but with complex phenomena computer systems have quite a lot to catch up with compared to the capabilties of the human perception system. This is why in most practical cases it is attempted to measure single parameters through each sensor, an approach which then can be easily multiplied. There are also sensors that sense phenomena that can not be perceived by human beings.

Sensors can be active or passive , in the latter case they need an electrical signal to be sent through them and the subsequent changes to this signal can be measured. Most active sensors need a supply current so from the system point of view there is not much difference. The resulting range of signal changes can vary however, and the input stage of the hardware interface needs to accomodate this. The list below gives an overview of systems that use an A/D (analog to digital) convertor or digital signals, some of which need further processing or drivers in the embedded software.

Analog voltage (or current): most A/D convertors have an input range from 0 - 5 volts, many sensor output signals match this but some would only range from for instance 1.5 to 3.5 volts (typical output of the Sprague UGN3503 magnetic sensor) or just in the millivolt range (the ICSensors 3031 acceleration sensor).

Digital a switch (or switch matrix) can be read by the interface hardware through digital lines. If a switch is read through an analog input, transition conditions have to be filtered out preferably with a hysteresis circuit (such as a Schmitt trigger)

Pulse count Rotary encoders, which for instance can be found in a standard mouse, translate a rotational movement of a wheel in a series of pulses, the amount of which is proportional with the angular movement speed.

Pulse Width Modulation (PWM) some sensors, such as the widely used ADXL202 acceleration sensors from Analog Devices, have a digital pulse output where the pulse width is proportional to the measured value.

Timing Typically for distance measurement systems based on ultrasonic waves, where the time of flight from emitter to receiver (this can be one transducer that measures the reflection of the sound waves, such as the Polaroid sensors), need a start pulse and stop pulse directly interfaced to a timer of the processor in the hardware interface. These applications need driver circuits and can be among the more complicated but very useful subsystems.

Serial Some sensor subsystems, such as the Motorola BiStatix tag reader system, communicate through a serial protocol.

As indicated above, most computer systems have powerful graphics and sound output capabilities so the focus of the Human Interface System is on kinetic and haptic displays and some simple actuators.

LED simple optical actuation can be obtained using Light Emitting Diodes which usually directly interface to processor output ports.

Tactual actuators electromagnetic coils can be used to generate palpable cues. If the current drawn by these actuators is small (miniature relay coils can be used) they can be connected directly to processor output pins, if the currents are bigger then an extra circuit around a (solid state) relay may be necessary. Micromotors can be used to, for instance with an eccentric weight attached (a common technique to create a palpable vibration - mobile phones often use this) or to create a larger movement. The motors can be controlled by a digital signal (just on/off or time modulated - PWM ), or with an analog voltage in which case a D/A (digital-analog) convertor is required. Touch feedback typically should be delivered there where the input takes place - in the intimate range of action of the user.

LCD to convey small text or graphic messages to the user small alfanumeric Liquid Crystal Displays can be used. Some versions of the PIC chips have built-in driver hardware for this (the STEIM supports a four digit display as well).

The main function of the actual interface unit is to get the sensor information in (in digital form by the means as described above), drive real world outputs through the actuators as described above, and communicate with the rest of the digital world. Some processing and signal conditioning may take place here, but mainly in order to avoid useless data to be sent. Simple mapping may take place here (from sensors to actuators) for efficiency reasons (Note: that the human nervous system supports this approach as well - through the reflexes) but the main part of the mapping, programming, making sense of the data etc. should take place in the higher levels of the system.

The diagram below shows the signals and busses. Not all functions mentioned in the diagram have to be part of the microcontroller. They can (and often have to, like in the case of the ultrasound) be seperate sub systems or, depending on the application, entirely omitted.

The resolution of the A/D convertor should be 10 - 12 bits, preferably with an auto ranging routine programmed at this level.

All sensor and actuator data should be given an ID, for sensors this has to be done in the interface unit.

The microcontroller has to be fast enough to meet the requirements set for this project and more - a latency of maximal 30 ms. This is the crucial stage. The delay and latence can be predicted from the design of the system, depending on the processor's clock speed, efficiency of the software of this embedded stage, and for instance in the case of analog inputs the speed of the A/D convertor.

The engineering efforts to develop this part of the system range from practically off-the-shelf, OEM type of single board PC solutions (Cerf, Rabbit, etc.), OEM microcontroller units, or at the lowest (and therefore most flexible) level specifially designed boards based on PIC or other microcontroller chips.

Microchip has a large range of PIC controllers, which differ in number of inputs / outputs, clock speed, memory size and type, programmabiltiy, etc. Texas Instruments has the TMS series of Microcontrollers.

Microcontrollers with a higher level language interpreter (Parallax Basic Stamp, BSX24, etc.) are easier to use but only sufficiently fast enough for a very limited range of applications.

The supply voltage for the system is usually 5 volts DC, and is best to be kept seperate from the power systems for sensor and actuator subsystems. The supplypower to sensors is often 5 volts (to Ground), but in some cases a symmetrical power supply (eg. +12/-12 volts) is needed especially when OpAmps? are connected for signal conditioning, although single supply OpAmps? exist too. The output voltage from a sensor is often related to the supply voltage, in order to reach the full range of 0 - 5 volts some sensors need a higher supply voltage. However, this can result in analog input signals above or below the A/D convertor window and not all converters like that.In such cases extra protection (using diodes) are necessary.

Power for heavier actuators such as motors and solenoids have to come from seperate power supplies depending on the specific needs, only the switching logic and circuits will be powered from the main supply.

The power supply itself should be a switched one, which is more reliable, smaller and has a more flexible AC input range (typically 100 - 240 volt).

Low power chips can be used in combination with power saving routines such as sleep modes(although there can be a trade-off with the speed here). For wearable applications, the human interface unit needs to be battery powered. Rechargable batteries are more environmentally friendly than (long life) Alkaline batteries, but to build reliable charging hardware is complicated. We can however rely on off the shelf solutions here, using seperate rather than in-circuit charging hardware. It is best to use Lithium-Ion batteries as used widely these days in digital cameras and laptop computers, as they have long life, short recharge times, and no memory effect. Nickel-Metal Hydride (NiMH) batteries are quite good too.

Battery powered logic usually runs off 3.3 volts, it has to be investigated how available Li-Ion batteries fit in this range (most logic has a tolerance in supply voltage range) or what voltage stabelisers are available which don't loose power.

Another thought is to generate the power from the movements of the players, but this involves cleaver charging cricuits that keep the power when there is not enough movement available.

One of the sensors I use a lot at the moment for measuring angular movement is actually a small motor, which generates its own power because it acts as a dynamo.

The bus links all the components in any configuration of the Human Interface System, which can be called the hi-bus. The hi-bus has to be bi-directional, the wiring has to be simple but relatively tolerant for noise, and allow for flexible topologies. Every component should have its own unique ID number (which also renders the connected sensor / actuator data identification unique). The bus may be used to distribute supply power.

Existing protocols and standards may be used, but this has to be investigated. Most microcontrollers have built-in hardware support for a number of standards such as CAN, I2C, SPI. Also a one-wire bus may be a possibility. Due to the modular nature of the Interface System, effectively we need a PAN: Personal Area Network

MIT MediaLab has pioneered this idea using high-frequency (~80kHz) electrical signals that travel through the body.

A low cost and very flexible solution that is widely used for connecting individual or small groups of sensors are the three pin (power, signal and ground) headers with 0.1“ spacing. Normally used for interconnecting circuit boards, the iCube system set this standard by having a large three row female sensor input connector. For demanding stage applications however these headers are often unsuitable when they are in exposed postions, or often (dis)connected, and they are often too big (but there are smaller versions available). In the case of instruments such as the Lady's Glove or The Hands, multipole D-connectors are used to connect all the sensors at once at the lowest possible pin count.

In the ideal system robust connectors such as available from Lemo (Swiss made) are preferable, they are round, are available with many different numbers of pins and sizes, and have a clever locking system.

In the past incarnations (and possibly in a near future version) of Txoom the wireless link was in the Ethernet connection. However, in line with the previously stated approach of having the least processing / computing on the body only the actual sensor / actuator data has to be transmitted ('transceiver' is a contamination of transmitter and receiver indicating that it is a two-way channel). In the whole chain of technical steps from sensor signal to meaningful output the wireless link can be made at various stages, each with their own advantages and disadvantages. The main point of consideration is that transmitting digital data is easier than analog. Only in situations where one or a few analog sensors signals need to be transmitted an analog transmitter can be used.

I have investigated this for a project at ZKM - in the end only two sensor signals needed to be wireless and this was done by bringing the \tab sensor signal in the audio range and using general purpose instrument tranmitters. Call this the Spinal Tap solution.

Wireless MIDI, although commercially available (MIDIMan Inc ) makes a set that piggybacks on a commercial analog instrument transmitter (eg. Nady). Still pretty Spinal Tap, and the speed and \tab resolution go down even further than the already poor MIDI spec.}, is quite a non starter because MIDI (however clever at the time and still very robust but limited in speed and precision) has no means of error detection, let alone error correction.

From an engineering point of view the best place to put the wireless link is just after the data acquisition stage (or output stage going the other way). This is where the data is digital, and it needs only to be packed in further protocols to employ error detection and correction. What is needed therefore are units (probably based on microcontrollers) that interface the hi-bus with a wireless link.

Popular digital wireless links are circuits from Linx Technologies Inc. (~ 900 MHz), Radiometrix (~ 400 MHz) and Aurel.

There are regulations that differ between countries, the American FCC for instance doesn't allow the 400 MHz band at certain ranges of power. Another issue of course is the overlap between the local mobile phone standards (GSM at 900 MHz).

Like with the wired bus, a scheme needs to be developed to enable multiple channels of information exchange.

Microchip is developing a new line of PIC chips with built in wireless link (the rfPIC), which is very interesting to make compact designs. The PIC specs are limited at present (no A/D for instance) but under constant development.

To interface the HIS to the other parts of the system, convertors have to be developed. Depending on the application, we can have convertors for hi-bus to Ethernet (for Txoom), hi-bus to MIDI to directly control synthesizers, hi-bus to USB for certain low cost applications, etc.

During the workshop in Visby, Sweden (19 - 25 August 2002) several case studies were presented after the presentation of the Background chapter of this report, in order to enable the group to discuss the design of the Human Interface System presented in this report.

From the author's own practice several examples were presented using the electronic portfolio (constructed in Macromedia Director) ranging from instruments to interactive architecture.

• The Hands - Michel Waisvisz, STEIM. An example of a new instrument
• The Lady's Glove, Laetitia Sonami. A number of sensors sewn on a lycra glove - very close to the skin.
• Chromasone gloves, Walter Fabeck. Gloves with the sensors inside.
• Belt & Shoes, Marie Goyette. Mainly shown for the bended circuit board, and the Lemo connectors used
• The Water Pavillion, Kas Oosterhuis and Lars Spuybroek. An example of interactive architecture.

Usually called 'interface' but during and after the workshop it was noted that this caused unnecessary confusion in the context of human interfaces Therefore the term Sensor Converters is preferred.

After a general discussion of (semi-)commercially available sensor hardware converters, a few examples were shown in reality and further discussed:

[ based on the Microchip PIC 17C73 microcontroller. It has multiplexed (32) analog inputs, 16 (matrix) switch inputs and ultrasound circuitry, converted to MIDI

The now discontinued Mindtel TNG-1 box, the smallest interface ever, four analog inputs to RS-232 serial based on a PIC 16C710.

Diana Young from the Hyperinstruments Group MIT MediaLab? developed a sensorised violin bow, described in detail in papers at several conferences such as the NIME (Diana Young: “The Hyperbow Controller: Real-Time Dynamics Measurement of Violin Performance”, Proceedings of the NIME (New Instruments for Musical Expression) conference, Dublin Ireland May 2002. See http://www.nime.org) and the ICMC. It incorparates a relative position sensing system (electric field), bow strain measurement (strain gauges) and acceleration sensors, al; sensor signals were converted through a microcontroller (PIC 16LF877) and wirelessly transmitted (Linx). Relevant issues such as weight and power consumption were very well dealt with.

The MIT Media Lab developed a shoe (Nike) equiped with almost every thinkable sensor, connected through a wireless link. A number of people in the group of Joe Paradiso have been involved over the years such as Ari Benbasat, and Eric Hu. The project was chosen to describe because of its technical merits, all sensors (earth gravity accelerometer and gyroscope, earth magnetic compass, ultrasound position, electric field position, pressure), the PIC 16C711 and Radiometrix transmitter are all fitted on a PCB in the size of a stack of cards

J. A. Paradiso, K. Hsiao, A.Y.Benbasat and Z. Teegarden: “Design and Implementation of Expressive Footwear”, IBM Systems Journal vol 39, no' s3&4, 2000. Downloadable from http://www.research.ibm.com/journal/sj/393/part1/paradiso.pdf

Research at Berkeley University in California has been carried out to make very small wireless sensing units. A operating system has been developed with a very small memory footprint, TinyOS, which is public domain http://today.CS.Berkeley.EDU/tos/

Many hardware and circuit board designs are public domain too, and there are commercial versions available through companies like Crossbow http://www.xbow.com, the MICA wireless measurement system… There is a lot of information available on the web that the workshop group looked at but some questions still remain. See below at the Scenario 2.

Another example that was discussed of distributed computing is the Filament project by Gershenfield, Hancher, Shri, of the Things That Think consortium, MIT Media Lab. Small sensor and actuator convertors units (based on a PIC16F876) are networked with RS-485 (simple twisted pair wiring) and using a stripped down version of IP. An ethernet bridge makes all Filament units available on a larger network, units can be addressed through a web site. All this was (attempted) to be used in a joint project between the Metapolis architects of Barcelona and Gershenfeld's group from the Media Lab, in the Media House http://www.metapolis.com

During the workshop a number of actual sensors were shown and described, to make the issues a bit more tangible. Examples were: a small accelerometer (a circuit board with SMD based on a ICSensors 3031 sensor and LM324 OpAmp), Interlink finger pressure sensors, a PIR movement sensor, the Sharp ranger distance sensor (10 - 90 cm), a Photocell, etc.

In this section some sensors are described that are of particular interest for the application in responsive environments, or interactivating spaces.

Currently in Txoom and Tgarden accelerometers from Analog Devices are used. Todor Todoroff designed a circuit board for the ADXL210J bi-axial sensor, two boards can be easily coupled tocreate a 3D sensing assembly. The sensors are read through the digital output (PWM signal) for optimal precision.

The older Analog Devices chip ADXL05 (soon to be discontinued) was shown too, on a small board with analog output and adjustable sensitivity up to ~5 g. (The Infusion Systems http://www.infusionsystems.com GForce is based on this chip too.)

The designs of STEIM and Sonology (both shown and discussed in the workshop) use an analog signal. The ICSensors 3031 design also has an analog output, and is single axis. In applications in musical instruments it was found that a sensitivity of ~2g worked best, but in these cases the signal was always read through an analog input (STEIM SensorLab? or Sonology MicroLab?).

In the FoAM projects a sensitivity of ~10 g was chosen to track bigger shocks and sudden changes, and if read with enough precision the fine range is included too.

All the designs mentioned based on the bi-axial Analog Devices sensors are based on the 14-pin Cerpak case SMD. There is a smaller version out (E-suffix) in a 8-pin leadless chip carrier (LCC) package with the dimensions 5x5x2 mm - even smaller then the single axis ICSensors 3031

Also the circuit board can be thinner,.75 mm instead of 1.5 mm - the epoxy based board will still be strong enough.

Tags are coils that react to radio waves send out by a sensing antenna coil. They can be passive (responders) or active (transponders).

Passive tags oscillate on their resonant frequency when brought into the emitted field, and this resonance is detected by the sensing coil. A passive tag can be very small, even a strip of metal with certain properties.

Transponders use the energy from the field they are in to charge a capacitor until they have enough power to respond with a transmission of information stored in the device, such as an indentification code. There are versions available that can be written into, to change the parameters (such as the ID) stored in the device. Transponder coils can be round, or flat as stickers, or even be printed on paper.

Tag technology can be used for the localisation and identification of objects and people. There are many different systems and applications, well known examples are

systems that prevent shoplifting: a tag is embedded in an article such as a piece of clothing or a book, which is detected if the tag passes the sensing antennas at the exit of the store

Drawing tablets such as the Wacom use a transponder inside the stylus to detect its position (an ID)

Recent systems for transponder technology are Texas Instruments TagIt and Motorola Bistatix. (Though the Motorola system seems to be taken over by Indala http://www.indala.info)

An older system such as Trovan can't distinguish between multiple objects brought into the same field. For an interesting discussion of these technologies and their application in many human interfaces and household objects see the article describing someof the research at Xerox PARC

Bridging Physical and Virtual Worlds with Electronic Tags. Roy Want, Kenneth P. Fishkin, Anuj Gujar and Beverly L. Harrison. Proceedings of the CHI conference 1999.

Several companies have introduced condictive and electrically sensitive materials, such as SoftSwitch http://www.softswitch.co.uk in England and Elektex http://www.elektex.com

Weaving electrically conducting wire in fabric is not so difficult, but the problem is this material never stretches. Also the connections remain a problem.

In the meeting at FoAM on 3 September we also looked at Electro Luminescent wire, as applied by Rachel Wingfield in fabrics and she described a process of printing a visual display in layers using silkscreen printing. Further experimentation with this will take place, also looking at how to connect it to a transducer convertor unit.

These sensors are common for burglar detection; when an intruder inters a space a signal is send out. Generally they are white plastic boxes, and are available from electronics stores. More choice will be available from specialised shops that sell burglaralarm systems.

The actual sensor chip, the PIR sensor, is a small and simplified CCD camera sensitive to infrared light which looks out through a multi-faceted fresnel lens, and combined with some circuitry it detects the movement of warm objects such as (living) people and animals.

They were used in the Trans\_Ports installation of Kas Oosterhuis for the Venice Architecture Biennale 2000 http://www.trans-ports.com} They act as a switch, and can be read at a distance. In some situations it can be a problem is that they often give a longs trigger of several seconds. Another issue is there large range of sensitivity, but this can be altered by making the window smaller with tape.

In the workshop a small type was shown.

Photocells come in a wide variety, ranging from cheapo DIY kits 30EUR to industrial quality versions that may cost around 200EUR. Basic ones are available from electronic stores, the industrial types have to be ordered from companies like Farnell or RS, directly from the respective distributors, or found in special stores.

They use an infrared light beam (the more expensive ones modulate the light to obtain a lower sensitivity to environmental light and other disturbances).

There are three types:

Separate: a single transmitter and a single receiver, creating a light path between the two elements. These have typically the longest range (10, 20 meters or more)

Reflex: Single with reflector: a combined transmitter and receiver that looks at a special reflector plate (that reflects mainly the light coming straight from the transmitter). These have an typical range of about 6 - 10 meters, and were used in the Water Pavilion of Lars Spuybroek.

Diffuse: Single, these are very versatile because they don't require a reflector; they detect the beam reflected of any object such as a person. Range is limited to about 1 meters typically, and they are quite expensive.

Good brands of industrial types are Honeywell, Omron, DL Tech, Sick.

To detect the presence of people on a specific location in 2D space (generally the floor) industrial switch mats canbe used. These switches close a contact when someone steps on them.

Tapeswitch makes mats and ribbons with switches inside for people detection and industrial applications. They are extremely durable, and come in a variety of sizes http://www.tapeswitch.com, the European distributor is the German Company Pepperl + Fuchs

For an earlier project FoAM has used similar switches, originally manufactured as driver seat sensor (White Products Ltd. Desford, Leicestershire, UK)

A human body has an influence on a high frequency electric circuit, such as a radio through approaching its antenna. This has been known for a long time, and in fact one of the earliest electronic musical instruments, the Theremin, used this principle.

This principle is based on the shunting of an electric field from an antenna to ground, where a body part acts as shunt (conductor), resulting in a decrease in electric field strength in the receiving antenna and a lower current, which can be measured. Another method is to make the human body act as an antenna, by applying a low power electrical signal (of about 80 - 100 kHz) to the body. This way more precise measurements are possible.

The translation of signal strength as received on multiple antennas to a coordinate in 3D space is difficult, as many factors have to be taken into account.

The MIT Media Lab is using this principle for sensing of movement for a variety of applications (HCI, musical instrmuments, smart furniture). The developed hardware is called the Fish board, after the ability of some species of fish to use electric fieldsas sense organs (Applying Electric Field Sensing to Human-Computer Interfaces).

Thomas G. Zimmerman, Joshua R. smith, Joseph A. Paradiso, David Allport and \tab Neil Gershenfeld. CHI Conference Proceedings 1995. http://www.acm.org/sigchi/chi95/Electronic/documnts/papers/tgz\_bdy.htm

The operating range is typically about 30 centimeters, and the accuracy at short distances is high (microns). The range is spherical around the antenna.

A company called Quantum makes special chips and everything else you need to make proximity sensing systems in all sorts of ranges www.qprox.com

This Qprox technology is available through Farnell, it would be good to get some developer and evaluation kits of this to see how it works.

A very interesting paper is written by people from Xerox PARC about an application of this technology in a hybrid electronic book

Listen Reader: An Electronically Augmented Paper-Based Book. Maribeth Back, Jonathan Cohen, Rich gold Stve Harrison and Scott Minneman. CHI Conference Proceedings 2001.

During the workshop small proximity sensor was showm, the Sharp ranger sensor GPD2D12. It sends out a beam of infrared light, and measures the angle of which it comes back (triangulation) through a small linear CCD, and this is proportional with the distance of the object. It is not sensitive to environmental light, or object colour (only slightly to object texture) and works well with the hand.

During the Visby workshop three scenarios or technical tracks were presented for developing a sensor system for Txoom. One was chosen for its short term feasibility (Scenario 1), based on off the shelf solutions and close to the original situation. Scenario 2 is using off the shelf components too, further away from the existing situation but closer to the ideal situation as researched and outlined in Chapter 2. Scenario 3 proposes to develop the ideal system from the ground up using versatile microcontrollers. This scenario involves the most work and is not feasible in the current stage of developments.

This is the solution closest to the existing solution and therefore requires the least alterations to the system, based on a single board computer running Linux. The information is passed on wirelessly using off the shelf 802.11b ethernet units, at a different point in the chain then outlined and supported in Chapter 2. Ethernet has never been developed for real-time applications (collitions and re-sends) and wireless ethernet uses error correction based on blocks. Practical experience has shown that it works well in the Txoom application most of the time.

The other argument is that the processing power of a wearable PC is not neccesary to be worn on the body - it only has to act as a converter to pass on the sensor information to the system.

The Cerf board http://www.intrinsyc.com is a small single board computer based on the Intel StrongARM processor, and runs Linux so that the existing software can be easily ported (from the iPaq's). The program can be stored in an on board CompactFlash? card, and a wireless ethernet card can be attached. Most I/O is collected in one multipin header and includes serial ports, and direct I/O pins which should be able to directly read the accelerometers' PWM signals. The size of the board is 57×69 mm. The supply voltage is 5 V DC and the current up to 800 mA.

The technology is described above in the Case Studies section. Further research is necessary, the main question is whether this system is fast enough with a low latency to suit our (very) real-time purpose. Demonstrated applications we found so far are mainly in the field of data logging, which has less strict timing constraints.

see also the article in the Sensors Magazine “The Commercialization of Microsensor Motes”, available on-line at http://www.sensorsmag.com/articles/0402/40/

But the specifications are very compelling, the units are small, and the network is able to dynamically reconfigure itself. The combination of open source / public domain software and hardware designs, and commercially available hardware (several sensors are already available as well as a convertor and wireless transceiver unit) is very strong.

The most flexible solution is to use standard microcontrollers such as the Microchip PIC series to design a system as outlined in Chapter 2. Each unit would consist of several (types of) sensor inputs and actuator outputs, connected in a network and with wireless links where needed. This is a longer term solution as the knowledge and skills in programming (specific assembly code) and hardware design has to be improved and / or acquired.

The Accelerometers are developed and built by Todor Todoroff. The wiring needs to be done, as well as the encapsulating (for protection) of the accelerometer units and the Cerfboards. The switches in the floor can be the FLD (Fat Lorry Driver sensor, see section on Switch mats) sensor, an amount between 20 and 100, wired to the electronics of a standard USBkeyboard. This keyboard kan be read through any computer of course.

For optimal reliabiltiy Lemo connectors should be used, 6 pin versions (2 x Xout and Yout, Vcc and GND) for double sensors and 4 pin connectors for single sensors. (Or 6 pin for single sensors too to be uniform.) It is suggested to hardwire a cable to each accelerometer unit, with a male cable mount Lemo connector at 30 cm. A similar cable with a matching (female cable mount Lemo) connector is attached to the motherboard. Separate extension cables should be made, at various lenghts to accomodate the diversety in application - taking into account the cable lenght limit determined by the sensor specifications. This is only a few meters, if larger distances need ot be covered buffer circuits (line drivers) have to be developed and inserted.

People that have been consulted / have given input / advice in relation to the project:

• Lex van der Broek, Institute for Sonology, Royal Conservatory, The Hague Netherlands
• Sukandar Kartadinata, ZKM Zentrum fur Kunst und Medientechnologie Karslruhe, and Berlin Germany
• Yolande Harris, Barcelona Spain
• Sha Xin Wei, Topological Media Lab, Georgia Tech, US
• Todor Todoroff, Brussels Belgium
• NikGaffney, FoAM, Brussels Belgium
• MajaKuzmanovic, FoAM, Brussels Belgium
• Rachel Wingfield
• Kamagurka.

Everyone at the Visby workshop: Tim Boykett, Tina Auer and Gerd Trautner (TimesUp, Linz), Martin Schlingmann (Interactive Institute), Yon Visell, Pix, Hiaz, Cocky Eek and Evelina Kusaite (FoAM, Brussels).