Hybrid systems combine hardware and software technologies for the implementation of applications in the areas of multimedia and mixed reality. Our projects consist of components for computational perception of audio and video data streams, intelligent processing by machine learning and other forms of classification, physical interaction with sophisticated hardware prototypes and proper visualization and sonification methods for the blending of hardware and software.
Below, you can find a selection of projects that were developed by staff and students since 2015. Earlier projects, such as the automated video soccer table Vizler can be found on the website of the former Interactive Media Systems group (link to the static snapshot). If you would like to participate in one of the open projects, please go to the list of current topics for student projects. If you have any questions or comments, please send me an e-mail.
2014-2019, audio project. An internet-based radio that filters ads and speech. The FM tuner can be configured with the three buttons in the middle of the front panel. One filters ads, the second music, the third speech. Hence, if you like the music of your favorite station yet not the comments of the DJ, press a button and the filtered speech is replaced by silence or music from your USB stick.
Details
2018-2020, audio project. Fair psychoacoustic measurement of noise in indoor environments. State of the art measurement of noise inside buildings suffers from the unfair aspect that the norm (ISO 226) over-emphasizes frequency bands that are irrelevant: Noise is created by bass sounds, yet the loudness is mostly a function of the (absent) higher bands. The Base Dosimeter compensates this insufficiency by combining a high quality recording circuit with a Raspberry processor that acts as dosimeter and processor of psychoacoustic noise indicators.
Details
If you would like to borrow the noise dosimeter for testing, please send me an e-mail.
2020, audio project. Automated leveling of feature films with high levels for action scenes/music. Often, modern feature films (e.g. Hollywood productions) use very unequal levels for dialogue and music/environmental sounds. In settings where the needs of third parties (sleeping children, neighbors, etc.) have to be respected, this can be very annoying. Hence, the Antidolby Loudness Balancer is able to learn remote control IR patterns, measures the sound pressure coming from your TV and - if necessary - reduces the level by sending the adequate commands.
Details
If you would like to make your own Antidolby, please send me an e-mail. I am happy to share the bill of material, software and 3D models of the case.
2017-2020, multimedia project. Skilled hunting, no hurting. The Bilderstutzen is a device designed for old-fashioned hunting of animals, yet in contrast to a gun, animals are not shot at but just photographed. The shape of the Bilderstutzen has the typical dimensions and proportions of a hunting rifle. The trigger runs stiffly and has the typical characteristics of a gun lock. Aiming is done by a nothc and bead sighter. Once fired, the Bilderstutzen creates a sound that resembles an explosion and is loud enough to shy the target away. Technically, the system is based on an embedded computer as well as a self developed autofocus camera with zoom lens. Images of shots are processed through deep learning on the edge in order to identify (1) if an animal is present in the scene and, if yes, (2) if it would have been killed or wounded. In the latter case, a hash value is created that can be transfered via a mobile app to a server, where a trophy of the animal is 3D printed. If the Bilderstutzen is fired at human beeing (detected by the deep net), the Bilderstutzen locks the operating system and renders itself unusable. Hence, the Bilderstutzen gathers all the skills of the art of hunting without its negative aspects.
Details
If you would like to test the Bilderstutzen, please send me an e-mail.
Since 2018, multimedia project. Can you control a robot arm better if you cannot just see it but also hear the distance of the manipulated object? In this project, we tried to answer this question. Hence, a setup was developed that includes a state of the art training robot, a joystick and a stereo camera. Sonification is based on beep sounds describing distance from and orientation towards the object in question.
Details
2012-2015, multimedia project. User your smartphone as a noise measurement device. Strictly speaking, the Noisometer app was not a hybrid system. Yet it combines smartphone hardware with psychoacoustic transformations and location-based services: Audio data is recorded over the microphone, filtered, processed and coefficients are computed. The resulting loudness values are forwarded to a server, aggregated and displayed on a map.
Details
Please note that the Noisometer cannot be found in the Play store anymore. If you are still interested in the app, please send me an e-mail.
Since 2014, mixed reality project. A unique physical-virtual skydiving experience. The Jumpcube enables humans to jump out of a virtual plane while performing a real physical jump. This combination creates a kick that is quite unique in mixed reality systems. Due to its popularity, the system was tested by more than 2700 people since 2015.
Details
For a more detailed description of the project including images, videos & a list of events, please refer to the project website.
2016-2018, mixed reality project. Enabling the sense of smell in VR applications. The Vragrancer is a system that allows the usage of up to eight different smells in mixed reality. It is based on the system ScentController by the German company Scent Communication. However, we had to develop our own controller and propagation system. Eventually, the smells are provided over a system of electric valves & tubes directly to the nose of the player.
Details
If you would like to borrow the Vragrancer for evaluation, please send me an e-mail.
Since 2018, mixed reality project. Physio-virtual rock climbing through space. The Vreeclimber wall combines a climbing treadmill with virtual reality hardware that allows for virtually climbing through space while the physical wall enables endless climbing with varying grips and inclinations of the wall.
Details
For a more detailed description of the project including images, videos & a list of events, please refer to the project website.
Since 2020, mixed reality project. Gamefull mixed reality bouldering. The Vreeboulder Projection System employs the Vreeclimber wall for bouldering. User play old style arcade games while climbing on the rotating and tilting wall. Touch all targets before time runs out in oder to achieve a new high score.
Details
If you would like to try the Vreeclimber of Vreeboulder system, please send me an e-mail.