Institute of Software Technology and Interactive Systems
Information & Software Engineering Group
Music Information Retrieval
With the creation of large audio collections, we need to devise ways to make those collections accessible to the users. Currently, access to music repositories is mostly limited to query-based retrieval based on textual meta-data, with some advanced systems supporting acoustic queries. What we would like to have additionally, is a way to facilitate exploration of musical libraries. We thus need to automatically organize music according to its sound characteristics in such a way that we find similar pieces of music grouped together, allowing us to find a classical section, or a hard-rock section etc. in a music repository.
We thus research various methods of indexing and structuring audio collections, as well as providing intuitive user interfaces for a wide range of devices. Feature extraction from audio signals, incorporating psychoacoustic information, is combined with textual features. Machine learning techniques are used to e xtract semantic information, group audio by similarity, or classify it into various genres. Advanced visualization techniques are employed to provide intuitive interfaces to audio collections on standard PC as well as mobile devices.
Contact: Andreas Rauber.
Audio Feature Extraction
Using methods from digital signal processing
and psycho-acoustics we are extracting semantic information from
music. The features extracted from the audio signal are able to
describe the stylistic content of the music, e.g. beat, presence
of voice, timbre, etc. Thus, a system using audio feature extraction
is able to tell about the content of a piece of music without the
need of annotated labels such as artist, song title or genre. Moreover,
it is able to find similar music automatically. Different kinds
of feature sets (Rhythm Patterns, amongst others) are the basis
to many subsequent tasks, such as automatic music organization or
classification into genres. [Details] |
 |
Project Members: Thomas Lidy, David Laister, Alexander Trinkl
Music Classification
Applying machine learning methods and employing
the features calculated from audio signals we built a system that
is able to learn the classification of music pieces into a pre-defined
taxonomy, corresponding to the user's likes. The system has to be
trained with a number of examples and is then able to categorize
music into different classes, e.g. music genres (classical, jazz,
hip hop, electronic, ...) or can identify artists. In another
project we combine textual with audio information for music classification.
A new project focuses on the recognition of mood (emotions) in music.
[Details] |
 |
Project Members: Thomas Lidy, Alexander Trinkl, Robert Neumayer, Doris Baum
PlaySOM - Organisation of Music Archives
Contrary to Music Classification, this project
is about unsupervised organisation of whole music archives, which
means that no training of the system is necessary. We apply the
Self Organizing Map (SOM) algorithm on audio features and are thus
able to organize a music archive on an intuitive map, grouping audio
tracks by their perceived acoustic similarity. Music with a similar
style is located close to each other building clusters (or islands),
while music with different style is located farther away. Different
visualization techniques enable various browsing metaphors, such
as geographic maps or weather charts. As the name says, the system
is compeletely self-organizing, and thus overcomes traditional genre
boundaries and adapts individually depending on the music archive
used. [Details] |
 |
Project Members: Michael Dittenbach, Robert Neumayer, Rudolf Mayer, Thomas Lidy
Browsing Music Spaces on Mobile Devices
Large music collections are becoming more and
more important also for the mobile user. People are able to carry
their complete audio collection with them on small portable devices.
We believe, that intuitive and intelligent access to music spaces
is also needed on portable devices, such as PDAs and mobile phones.
Manually searching through directory structures and sorting music
into playlists is no way to enjoy music, especially when one wants
to listen a music of a particular mood. Thus, we are making effort
to implement our metaphors from the SOM enhanced JukeBox also on
state-of-the-art portable devices. [Details] |
 |
Project Members: Robert Neumayer
Map of Mozart
We extracted acoustic features from the complete works of Wolfgang Amadeus Mozart
and created an interactive Map of Mozart. The map provides an immediate
overview of all works by Mozart. The shape of the map was chosen to resemble Mozart's
silhouette by using the MnemonicSOM algorithm. A semi-transparent layer enables the
visualization of both Mozart's head and the Islands of Music metaphor for intuitive
browsing and selection of the music. All pieces of music ever composed by Mozart can thus be
retrieved and listened to by using the PlaySOM application.
[Explore] |
 |
Project Members: Thomas Lidy, Rudolf Mayer
Visualization Enhancements
Our SOMeJB system already provides a number
of intuitive visualization methods making browsing through music
a pleasure. We implemented the U-Matrix, showing distances between
neighbouring SOM-clusters, Smoothed Data Histograms, showing a methaphor
of Islands of Music, as well as Weather Charts, and more. Research
is continuing to provide the most enjoyable means of access to the
wealth of music possible. |
 |
Project Members: Georg Pölzlbauer, Rudolf Mayer
SOMeJB - The SOM-enhanced JukeBox
SOMeJB, the SOM-enhanced JukeBox, commenced
in 1999, established the basis for our browsing interfaces to music
archives. The SOMeJB Music Digital Library Project aims at creating
a browsable music archive by combining a variety of technologies
from the fields of audio processing, neural networks, and information
visualization, to create maps of music archives. It has its roots
in the SOMLib Digital Library for text archives. It is based on
the self-organizing map (SOM), a popular unsupervised neural network,
and its extension, the Growing Hierarchical Self-Organizing Map
(GHSOM). The resulting maps of the music archive can be explored,
and new, unknown pieces of music similar to ones personal likings
can be discovered, with Islands of Music and Weather Charts providing
an intuitive interface to the system. [Details]
|
 |
(Former) Project Members: Markus Frühwirth, Elias Pampalk, Thomas Lidy
| top | last edited 11.04.2006 by Thomas
Lidy |