DOC PREVIEW
SWARTHMORE CS 97 - A Large-Scale Evaluation of Acoustic and Subjective Music Similarity Measures

This preview shows page 1-2-3-4-5 out of 14 pages.

Save
View full document
View full document
Premium Document
Do you want full access? Go Premium and unlock all 14 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 14 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 14 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 14 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 14 pages.
Access to all documents
Download any document
Ad free experience
Premium Document
Do you want full access? Go Premium and unlock all 14 pages.
Access to all documents
Download any document
Ad free experience

Unformatted text preview:

A Large-Scale Evaluation of Acoustic and Subjective Music Similarity Measures Adam Berenzweig1, Beth Logan, Daniel P.W. Ellis1, Brian Whitman2 Cambridge Research Laboratory HP Laboratories Cambridge HPL-2003-193 September 8th , 2003* E-mail: [email protected], [email protected], [email protected] music similarity, acoustic measures, evaluation, ground-truth Subjective similarity between musical pieces and artists is an elusive concept, but one that must be pursued in support of applications to provide automatic organization of large music collections. In this paper, we examine both acoustic and subjective approaches for calculating similarity between artists, comparing their performance on a common database of 400 popular artists. Specifically, we evaluate acoustic techniques based on Mel-frequency cepstral coefficients and an intermediate ‘anchor space’ of genre classification, and subjective techniques which use data from The All Music Guide, from a survey, from playlists and personal collections, and from web-text mining. We find the following: (1) Acoustic-based measures can achieve agreement with ground truth data that is at least comparable to the internal agreement between different subjective sources. However, we observe significant differences between superficially similar distribution modeling and comparison techniques. (2) Subjective measures from diverse sources show reasonable agreement, with the measure derived from co-occurrence in personal music collections being the most reliable overall. (3) Our methodology for large-scale cross-site music similarity evaluations is practical and convenient, yielding directly comparable numbers for different approaches. In particular, we hope that our information-retrieval-based approach to scoring similarity measures, our paradigm of sharing common feature representations, and even our particular dataset of features for 400 artists, will be useful to other researchers. * Internal Accession Date Only Approved for External Publication 1 Columbia University, New York, NY 2 Music Mind & Machine Group, MIT Media Lab, Cambridge, MA To be published in and presented at ISMIR 2003: Fourth International Conference on Music Information Retrieval, 26-30 October 2003, Baltimore, Maryland.  Copyright ISMIR 20031 IntroductionTechniques to automatically determine music similarity have attracted much attention in recentyears [8, 7, 12, 10, 1, 6]. Similarity is at the core of the classification and ranking algorithmsneeded to organize and recommend music. Such algorithms will be used in future systems toindex vast audio repositories, and thus must rely on automatic analysis.However, for the researcher or system builder looking to use similarity techniques, it is difficultto decide which is best suited for the task at hand. Few authors perform comparisons acrossmultiple techniques, not least because there is no agreed-upon database for the community.Furthermore, even if a common database were available, it would still be a challenge to establishan associated ground truth, given the intrinsically subjective nature of music similarity.The work reported in this paper started with a simple question: How do two existing audio-basedmusic-similarity measures compare? This led us in several directions. Firstly, there are multipleaspects of each acoustic measure: the basic features used, the way that feature distributions aremodeled, and the methods for calculating similarity between distribution models. In this paper,we investigate the influence of each of these factors.To do that, however, we needed to be able to calculate a meaningful performance score for eachpossible variant. This basic question of evaluation brings us back to our earlier question ofwhere to get ground truth [6], and then how to use this ground truth to score a specific acousticmeasure. Here, we consider five different sources of ground truth, all collected via the Webone way or another, and look at several different ways to score measures against them. We alsocompare them with one another in an effort to identify which measure is ‘best’ in the sense ofapproaching a consensus.A final aspect of this work touches the question of sharing common evaluation standards, andcomputing comparable measures across different sites. Although common in fields such asspeech recognition, we believe this is one of the first and largest cross-site evaluations in mu-sic information retrieval. Our work was conducted in two independent labs (LabROSA atColumbia, and HP Labs in Cambridge), yet by carefully specifying our evaluation metrics,and by sharing evaluation data in the form of derived features (which presents little threat tocopyright holders), we were able to make fine distinctions between algorithms running at eachsite. We see this as a powerful paradigm that we would like to encourage other researchers touse.This paper is organized as follows. First we review prior work in music similarity. We thendescribe the various algorithms and data sources used in this paper. Next we describe ourdatabase and evaluation methodologies in detail. In Section 6 we discuss our experiments andresults. Finally we present conclusions and suggestions for future directions.2 Prior WorkPrior work in music similarity has focused on one of three areas: symbolic representations,acoustic properties, and subjective or ‘cultural’ information. We describe each of these belownoting in particular their suitability for automatic systems.1Many researchers have studied the music similarity problem by analyzing symbolic represen-tations such as MIDI music data, musical scores, and the like. A related technique is to usepitch-tracking to find a ‘melody contour’ for each piece of music. String matching techniquesare then used to compare the transcriptions for each song e.g. [8]. However, techniques basedon MIDI or scores are limited to music for which this data exists in electronic form, since onlylimited success has been achieved for pitch-tracking of arbitrary polyphonic music.Acoustic approaches analyze the music content directly and thus can be applied to any musicfor which one has the audio. Blum et al. present an indexing system based on matching featuressuch as pitch, loudness or Mel-frequency cepstral coefficients (MFCCs) [3]. Foote has designeda music indexing system based on histograms of MFCC features derived from a discriminativelytrained vector


View Full Document

SWARTHMORE CS 97 - A Large-Scale Evaluation of Acoustic and Subjective Music Similarity Measures

Documents in this Course
Load more
Download A Large-Scale Evaluation of Acoustic and Subjective Music Similarity Measures
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view A Large-Scale Evaluation of Acoustic and Subjective Music Similarity Measures and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view A Large-Scale Evaluation of Acoustic and Subjective Music Similarity Measures 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?