Introduction

Hybrid systems combine hardware and software technologies for the implementation of applications in the areas of multimedia and mixed reality. Our projects consist of components for computational perception of audio and video data streams, intelligent processing by machine learning and other forms of classification, physical interaction with sophisticated hardware prototypes and proper visualization and sonification methods for the blending of hardware and software.

Below, you can find a selection of projects that were developed by staff and students since 2015. Earlier projects, such as the automated video soccer table Vizler can be found on the website of the former Interactive Media Systems group (link to the static snapshot). If you would like to participate in one of the open projects, please go to the list of current topics for student projects. If you have any questions or comments, please send me an e-mail.

Base Dosimeter

2018-2020, audio project. Fair psychoacoustic measurement of noise in indoor environments. State of the art measurement of noise inside buildings suffers from the unfair aspect that the norm (ISO 226) over-emphasizes frequency bands that are irrelevant: Noise is created by bass sounds, yet the loudness is mostly a function of the (absent) higher bands. The Base Dosimeter compensates this insufficiency by combining a high quality recording circuit with a Raspberry processor that acts as dosimeter and processor of psychoacoustic noise indicators.

Details

• Hard- & Software: Sonarworks microphone, Raspberry Pi 4 processor, GPS receiver, GPRS sender module, custom amplifier and power supply, programmed in Python
• Solution: Records 11025 samples per second, computes RMS and loudness in Phone as defined by the norm, but limited to <=1000Hz, transmits measurement + GPS data to remote server
• Performance: With hardly any noise from the power supply, the results are competitive to top level noise measurement hardware
• Contributions: 95% by Simon Weber (hardware design, implementation and programming of software)

If you would like to borrow the noise dosimeter for testing, please send me an e-mail.

Bilderstutzen

2017-2020, multimedia project. Skilled hunting, no hurting. The Bilderstutzen is a device designed for old-fashioned hunting of animals, yet in contrast to a gun, animals are not shot at but just photographed. The shape of the Bilderstutzen has the typical dimensions and proportions of a hunting rifle. The trigger runs stiffly and has the typical characteristics of a gun lock. Aiming is done by a nothc and bead sighter. Once fired, the Bilderstutzen creates a sound that resembles an explosion and is loud enough to shy the target away. Technically, the system is based on an embedded computer as well as a self developed autofocus camera with zoom lens. Images of shots are processed through deep learning on the edge in order to identify (1) if an animal is present in the scene and, if yes, (2) if it would have been killed or wounded. In the latter case, a hash value is created that can be transfered via a mobile app to a server, where a trophy of the animal is 3D printed. If the Bilderstutzen is fired at human beeing (detected by the deep net), the Bilderstutzen locks the operating system and renders itself unusable. Hence, the Bilderstutzen gathers all the skills of the art of hunting without its negative aspects.

Details

• Hard- & Software: Welded steel frame, 20mm barrel, Raspberry Pi 4 for processing and CNN classification, zoom lens + step motor as autofocus, various sensors and actors, OpenCV and TensorFlow for image recording and image analysis + classification
• Solution: State transaction machine for gun control, convolutional neural net for object detection, transfer learning for the classification of the shot success (kill/wounded/missed)
• Performance: F1 score for the animal recognition of 92%, F1 score for the shot success estimation of 84%, both values are work in progress
• Contributions: -

If you would like to test the Bilderstutzen, please send me an e-mail.

Noisometer

2012-2015, multimedia project. User your smartphone as a noise measurement device. Strictly speaking, the Noisometer app was not a hybrid system. Yet it combines smartphone hardware with psychoacoustic transformations and location-based services: Audio data is recorded over the microphone, filtered, processed and coefficients are computed. The resulting loudness values are forwarded to a server, aggregated and displayed on a map.

Details

• Hard- & Software: Any Android smartphone, implementation in Java
• Solution: The major issue is computing reliable noise values from different types of phone hardware. Achieving this goal required detailed testing and evaluation of filtering methods.
• Performance: The final prototype produced reliable measurements & could be used as a dosimeter for the long-term evaluation of environmental noise.
• Contributions: Alexander Schatten (project management), Thomas Teufel & Johannes Hartl (implementation)

Please note that the Noisometer cannot be found in the Play store anymore. If you are still interested in the app, please send me an e-mail.

Vragrancer Smell Engine

2016-2018, mixed reality project. Enabling the sense of smell in VR applications. The Vragrancer is a system that allows the usage of up to eight different smells in mixed reality. It is based on the system ScentController by the German company Scent Communication. However, we had to develop our own controller and propagation system. Eventually, the smells are provided over a system of electric valves & tubes directly to the nose of the player.

Details

• Hard- & Software: ScentController.6, Raspberry Pi 3, Teflon tubes, electric 12V valves & a compressor
• Solution: The smell cartridges are also provided by Scent Communication and can be configures in almost any possible form. The palette includes, for example, the smell of a Diesel motor, coffee, ham & a meadow in spring.
• Performance: Scents are delivered with a latency of ca. 200ms over 5m from the Vragrancer to the HMD
• Contributions: Scent Communication, Cologne (hardware system, scents), Juri Berlanda (controller)

If you would like to borrow the Vragrancer for evaluation, please send me an e-mail.

Vreeboulder Projection System

Since 2020, mixed reality project. Gamefull mixed reality bouldering. The Vreeboulder Projection System employs the Vreeclimber wall for bouldering. User play old style arcade games while climbing on the rotating and tilting wall. Touch all targets before time runs out in oder to achieve a new high score.

Details

• Hard- & Software: Our self-made climbing treadmill, a self-made CNC control system for the BenQ projector, Python, OpenCV and the PyGame library.
• Solution: Computer vision methods are employed for position tracking of the gamer, the GRBL library is used for the positioning of the projector (we are not able to afford a bigger projector), and PyGame is used for game design and rendering of sprites. Based on the plattform, arbitrary 2D level games can be implemented.
• Discussion: Currently, since we do not have a proper mattress for bouldering (though the Vreeclimber was designed to be able to integrate one), we use the Vreeclimber rope system for securing the climbing person. However, all game elements are designed for bouldering applications and since no VR headgear is involved, could be directly switched to real bouldering.
• Contributions: -
• Sponsors: Many thanks to Hannes Kaufmann for donating a projector to the project!

If you would like to try the Vreeclimber of Vreeboulder system, please send me an e-mail.

Contact

• E-Mail: horst.eidenberger@tuwien.ac.at
• Phone: +43 1 58801 18853
• Mail: Favoritenstrasse 9/19401, 1040 Vienna, Austria


• Office: HE0150 - Enter Favoritenstrasse 11, go to stair 3, ascend to the first floor, take the first aisle on the right, once inside follow the aisle down to the meeting area, the office is right in front of you. Important: If the door on the first floor is closed and nobody opens, it means that the secretary is away. In this case, often, the next door to the left ist kept open - which leads to the same aisle.