MUSCLE Network of Excellence

Inventory of
Audio Description and Recognition Tools

RP_extract Music Feature Extractor

TU Vienna - IFS, Thomas Lidy

Content-based access to audio files, particularly music, requires the development of feature extraction techniques that capture the acoustic characteristics of the signal, and that allow the computation of similarity between pieces of music. At TU Vienna - IFS three different sets of descriptors were developed:

• Statistical Spectrum Descriptors: describe fluctuations by statistical measures on critical frequency bands of a psycho-acoustically transformed Sonogram
• Rhythm Patterns: reflect the rhythmical structure in musical pieces by a matrix describing the amplitude of modulation on critical frequency bands for several modulation frequencies
• Rhythm Histograms: aggregate the energy of modulation for 60 different modulation frequencies and thus indicate general rhythmic in music

The algorithm considers psycho-acoustics in order to resemble the human auditory system. The feature extractor is currently implemented in Matlab and processes au, wav, mp3 and ogg files. Feature vectors are output in SOMLib format.

Download (V 0.621) : RP_extract_0.621.zip

Usage Guide: http://www.ifs.tuwien.ac.at/mir/howto_matlab_fe.html

Audio Feature Extraction Web Service 

TU Vienna - IFS, Jakob Frank

GenChords - Automatic Chord Detection

TU Vienna - IFS, Veronika Zenz

Music Audio Segmentation

TU Vienna - IFS, Ewald Peiszer

Phase 1: Boundary detection

This phase tries to detect the segment boundaries of a song, i.e., the time points where segments begin and end. The output of this phase is used as the input for the next phase.

Phase 2: Structure detection

This phase tries to detect the form of the song, i.e., a label is assigned to each segment where segments of the same type (verse, chorus, intro, etc.) get the same label. The labels themselves are single characters like A, B, C, and thus not semantically meaningful.

Details: http://www.ifs.tuwien.ac.at/mir/audiosegmentation.html

Bayesian Extensions to Non-negative Matrix Factorisation for Audio Signal Modelling

Cambridge University, Tuomas Virtanen, Ali-Taylan Cemgil, Simon Godsill

Non-negative matrix factorisation is applied here as a decomposition where spectrogram matrix X is approximated as a product of excitation matrix E and template matrix V. It is an efficient machine learning tool that can be applied in audio signal analysis in supervised or unsupervised manner. Bayesian extensions of the method enable assigning priors for the parameters. In the case of templates V, they allow defining prior information about spectra of sources and when used for excitations E, they can model correlation between adjacent frames, or sparseness of the excitations.

The software package contains non-negative matrix factorisation algorithms targeted for the analysis audio signals. It contains an implementation of the algorithms by Lee and Seung [1], which minimise either the Euclidean distance or the divergence function. It also contains extensions, which include sparseness or temporal continuity terms [2]. Bayesian extensions described in [3] which enable setting priors for the matrices to be estimated are implemented in the package as well. It also includes algorithms for minimising two other reconstruction error measures which are based on the probabilistic models described in [4].

Multi-Object Tracking of Sinusoidal Components in Audio

Cambridge University, Daniel Clark, Ali-Taylan Cemgil, Paul Peeling, Simon Godsill

The tracker identifies individual sinusoidal tracks from audio signals using multi-object stochastic filtering techniques. It can estimate target states when observations are missing and can maintain the identity of these targets between time-frames. The objective to find parameters of a set of damped sinusoidals is achieved through a multi-object generalisation of Bayes filtering. The Probability Hypothesis Density Filter is a recursion of the first-order moment of the multi-object Bayes filter.

This is an example of the tracker, which has been programmed for a two-dimensional constant velocity model. The library is not restricted to this case and can be used for other models and different dimensions. The algorithm is described in the following papers:

• D. E. Clark, K. Panta, V. Ba-Ngu (2006) The GM-PHD Filter Multiple Target Tracker. Proc. International Conference on Information Fusion, Florence, Italy, pp. 1-8.
• D. Clark, A. T. Cemgil, P. Peeling, and S. Godsill (2007) Multi-object tracking of sinusoidal components in audio with the Gaussian mixture probability hypothesis density filter. Proc. of IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, pp. 339-342.

 ./tracker

Support Vector Machine Re-Scoring Algorithm of Hidden Markov Models with Applications to Speech Recognition

Tel-Aviv University ( TAU – SPEECH), A. Sloin, A. Alfandary, D. Burshtein

According to the statistical approach to automatic speech recognition, each linguistic unit is typically assigned a hidden Markov model (HMM), the parameters of which are estimated using the Maximum Likelihood (ML) approach. However, when the assumed model is not sufficiently accurate or when there is not enough training data, it is possible to significantly improve the recognition rate using discriminative training methods, such as support vector machine (SVM)-type classification. In our work we proposed a new SVM re-scoring algorithm of HMMs with applications to speech recognition. This algorithm utilizes a variable to fixed length transformation of the speech data. The application of the algorithm to isolated and connected digit and word-spotting tasks provides a significant error rate reduction compared to standard ML-trained HMM systems.

Figure 1 shows a comparison between our method and standard ML training (baseline) on a noisy isolated digit task (TIDIGITS database). The error rate reduction is 56.1% compared to the baseline.

For a connected digit experiment we obtained a reduction of 51% in the error rate. This is shown in the figure 2.

Figure 3 shows the ROC curves of the baseline and new approaches for a keyword spotting task.

Figure 3

Software: The research programs for the tasks described above (isolated speech, connected speech and word spotting) can be obtained by contacting David Burshtein (available to MUSCLE members only).

Related Journal Publications:

• A. Sloin and D. Burshtein, “Support Vector Machine Training for Improved Hidden Markov Modeling,” IEEE Transactions on Signal Processing, vol. 56, no. 1, pp. 172-188, January 2008.
• A. Alfandary and D. Burshtein, “Improvements and Generalization of the SVM Re-Scoring Algorithm of Continuous HMMs”, to be submitted to IEEE Transactions on Signal Processing.