All posts in Blog

Dear all,

After ramping down development already in 2013, Spectralmind now fully closed the company and stopped all operations.

We hugely thank the entire former team of Spectralmind, all collaborators and partners and generally all people who have given us the trust and interest over all these years for the good times we had together!

This was an exciting journey! As exciting as the music domain always has been, is and ever will be! :-) Good luck to all the other exciting and adventurous music companies out there!

And yes, there is good news: We open-sourced almost our entire software stack, from spectral audio analysis and audio-based music similarity to all our music discovery apps! Try it, hack it, make it better!

All the best,
Thomas Lidy, Ewald Peiszer, Johann Waldherr
former CEO, CTO and BizDev of Spectralmind

Both companies present the future of navigating digital music collections with a new Spotify app prototype at the ISMIR conference.

Porto/Vienna/Emeryville, 8th October, 2012. Spectralmind ( und Gracenote ( will give an insight into the future of selecting and listening to digital music at the 13th International Society for Music Information Retrieval Conference (ISMIR) in Porto, Portugal. Both companies will present a demo-app for the Spotify player, which personalizes music for every user, based on their musical tastes. Navigation is done intuitively using a multi-level graphic display, making lists obsolete. The app’s additional “Discovery Mode” takes the consumer on a musical journey. The user will not only hear songs he or she already knows, but will be offered music from a catalog comprising 18 million music titles – all adapted to the user’s individual tastes and moods. The presentation of the demo app for Spotify is the first time that Spectralmind has made an app for notebooks and desktop stations, following a series of music apps made for iOS.

The world is getting a glimpse into the (near) future of music consumption, through the cooperation between Gracenote, a Sony subsidiary and the world’s largest provider of music metadata, and Spectralmind, the leading provider of intuitive graphic media interfaces. Rigid, random music categorization in lists is history. Instead, the user’s music collection will be interpreted intelligently using software and intuitively arranged in personalized clusters that are based on their musical tastes.

The application first analyzes the user’s listening history and preferences, and then clusters them into multiple tastes using a rich array of musical attributes provided by Gracenote, including genre, era, mood, artist type, and more. These taste clusters will be named according to the musical attributes represented within them (such as “Energetic Anxious Indie Dance” or “Arousing Groove Hip-Hop/Rap by Male Artists), and will provide an intuitive and easy way to navigate ever-expanding music catalogs.

These personalized musical taste clusters will be displayed as different-colored bubbles, and not in lists. These bubbles provide several levels into which users can zoom smoothly. Each level provides more information and more refined music recommendations – always based on the initial musical taste cluster. The songs can be played at every level of the player, of course.

Musical Journey in “Discovery Mode”

Once the user activates the app’s “Discovery Mode”, he or she will go on a musical journey based on the favored taste cluster. The app takes music from a Spotify-catalogue that includes millions of songs – an almost inexhaustible supply of new, undiscovered music for the listener.

“The colorful bubbles that provide more details at every level are perfect for choosing music depending on one’s feeling and mood,” says Thomas Lidy, CEO of Spectralmind. “The visually appealing display makes listening to music more emotional, intuitive and playful again – and we offer an entirely new experience in listening to and discovering music. The spreadsheet-like list displays seem really anachronistic in the age of touch screens and music from the cloud.”

Gracenote delivers the music identification technology and rich music attribute metadata, as well as the clustering technology used to analyze and group the user’s listening history into personalized taste clusters, and the recommendation technology to generate Spotify playlists based on these taste clusters.

The Spotify-app presented by Spectralmind and Gracenote at the ISMIR is a functional prototype, providing a preview into the current technological and visual possibilities of music consumption. The exact time when this technology and application will be available is still to be disclosed.


About Spectralmind:

Spectralmind is a technology leader in the field of media intelligence. Spectralmind offers fully fledged media discovery solutions in both consumer and professional markets. The solution stands out by its unique combination of intuitive visual interfaces and advanced content discovery technologies to provide the latest in media discovery and content recommendation. The company’s proprietary audio matching engine is capable of suggesting similar music titles based on a song’s acoustic content and a user’s preferences. The technology is used for media discovery in a wide range of domains, such as music and video portals, smartphone and tablet apps and professional media solutions.

The company’s vision is to revolutionize the media discovery space through its unique combination of intelligent content discovery and visual user experience.

About Gracenote

A pioneer in digital media, Gracenote, Inc. provides music and video content and technologies to the world’s hottest entertainment products and brands. The company’s partners in the entertainment community include major music publishers and labels, prominent independents and movie studios and television networks. A wholly owned, independent subsidiary of the Sony Corporation of America (SCA), Gracenote has offices in Tokyo, Munich, Berlin, Seoul, and Taipei with worldwide headquarters in Emeryville, California. For more information, follow us at @GracenoteTweets and

Spectralmind online:

Sonarflow App:

Sonarflow on Facebook:


For inquiries please contact:

i5comm for Spectralmind

Bernhard Lehner

Tel.: +43 664 439 86 09

Spectralmind and XroadMedia present the iPad app “MyChoice” for personalized TV programming and intuitive graphic navigation at the top TV trade fair IBC2012 in Amsterdam.

Amsterdam/Vienna, 6th September. Spectralmind (, the experts in intuitive graphic-based navigation through digital catalogues, will present “MyChoice”. MyChoice is an iPad app for personalized television programming catering to individual TV preferences using graphic navigation to browse through different detail levels. Users will no longer need to click their way through long lists thanks to Spectralmind‘s patented graphic media navigation. The user’s TV recommendations are visualized in so-called “bubbles” and organized hierarchically over numerous different levels. The app is based on the recommendation technology by XroadMedia, specialists in the discovery and personalization of media data. The initial presentation of MyChoice will take place during the IBC2012, the top TV industry trade fair which is currently underway in Amsterdam.

The MyChoice app combines all the advantages of the well-established music recommendation app Sonarflow with the innovative and unique recommendation solutions developed by XroadMedia. These solutions focus on finding and recommending any desired digital media content based on the personal tastes and preferences of the user. Sonarflow‘s intuitive, graphic user interface with its so-called ´”bubbles” compliments this functionality perfectly. Spectralmind is able to provide an innovative and intelligent media navigation solution for video content for the first time, thanks to its partnership with XroadMedia.

Here’s how the MyChoice app works

The start screen of the app greets the user with the three categories “My TV”, “Live TV” and “Program Guide”, which are visualized in Sonarflow’s well-known bubbles. The user gets deeper into each bubble with the typical iPad zoom movement. In MY TV, the user only gets TV program recommendations based on their established preferences. The user can go deeper and deeper into his/her categories such as “My Kids”, “My Shows” or “My Series”. Each level can be zoomed into to get further details. When they finally reach the title level all that is required is a tap on the title to get all the information on this program. The user can see which programs are running at present in the bubble “Live TV”. The Program Guide holds the entire TV program content.

Find programs and films using the Discovery Mode

It’s easy to discover new programs to suit your own personal tastes with “MyChoice”. All that you need to do is tap on a film title. You will get suggestions for similar, suitable films.

Content from social networks can be found in the bubble “Facebook”. Every user “Like” is reflected in this bubble. They are analyzed by the app and then the appropriate film content is offered from online sources such as Facebook, YouTube or other on-demand or TV sources.

“The colorful Sonarflow bubbles are perfect for visualizing TV programs. Every desired detail can be obtained thanks to smooth zooming. Navigation is finally list-free, visual and intuitive,” says Thomas Lidy, CEO of Spectralmind, the company behind Sonarflow.

Adolf Proidl, CEO from XroadMedia, added: „We are excited to work with Spectralmind on the myChoice app – an intuitive and fun to use experience that helps users to easily find and enjoy content they are really interested in.”

About Spectralmind:

Spectralmind is a technology leader in the field of media intelligence. Spectralmind offers fully fledged media discovery solutions in both consumer and professional markets. The solution stands out by its unique combination of intuitive visual interfaces and advanced content discovery technologies to provide the latest in media discovery and content recommendation. The company’s proprietary audio matching engine is capable of suggesting similar music titles based on a song’s acoustic content and a user’s preferences. The technology is used for media discovery in a wide range of domains, such as music and video portals, smartphone and tablet apps and professional media solutions.

The company’s vision is to revolutionize the media discovery space through its unique combination of intelligent content discovery and visual user experience.

About XroadMedia GmbH:

XroadMedia GmbH, pronounced “crossroad media”, was established by former management team members of APRICO Solutions – leading industry experts in the areas of content navigation, recommendation and targeting. XroadMedia has created an innovative and revolutionary recommendation and targeting solution to provide relevant content to consumers based on their viewing behavior, interests and social profiles. Our technology was developed to seamlessly integrate into existing and new content delivery and monetization platforms. By using our solution, our customers reduce complexity and resources required to distribute, recommend and target the right content to the right people… regardless the device and the platform.

Spectralmind online:

Follow Spectralmind on Facebook:


For inquiries please contact:

i5comm für Spectralmind

Bernhard Lehner

Tel.: +43 664 439 86 09
E-Mail: sonarflow [at]

From 28 million iTunes songs, Sonarflow’s Music Discovery App for iPhone and iPad finds the ones that really match your personal taste

Sonarflow (, a new iPhone and iPad app to discover new music, harmonizes perfectly with the personal music library in iTunes and the musical universe of about 28 million songs at the Apple iTunes Store (

With its visual and intuitive user interface, Sonarflow iTunes offers a new, list-free view of your music collection. The App is also the perfect tool to discover previously unknown music on iTunes: the majority of the current 28 million songs in iTunes lead a life well below the popular mainstream perception. In “Discovery Mode” you will be able to tap into this rich music source as Sonarflow recommends artists and songs based on your personal music taste. Sonar Flow for iTunes is available in a free version and in an ad-free Pro version for € 1.59 in Apple’s iTunes Store.

In today’s digital age, music lovers are confronted with a luxury problem. How do you filter out music you like from the sheer endless supply of songs and albums out there? A challenging task: 28 million songs are waiting out there to be discovered on iTunes, other providers such as the streaming service Spotify (for which Sonarflow has recently released a separate free app for premium users) offers close to 18 million songs in the cloud.

Find music in Sonarflow’s discovery mode

Sonarflow helps to discover music using either its intuitive visual representation of the user’s personal iTunes music library or switching to “Discovery Mode”:music is presented on iPad and iPhone using bubbles. The alignment of the bubbles visualises the proximity of the different music styles to another, the colour of each bubble represents the music-genre.

The musical journey through the personal music library starts in the genre overview. If you zoom into a bubble, it displays the several artists. One level below, you will find the albums of these artists. If you activate the “Discovery Mode”, Sonarflow will cross the boundaries of your iPhone or iPad. The App will now recommend appropriate music, which it finds in the iTunes universe. Sonarflow will now find new music in the depths of the music universe, limited only by your personal state of mind. New music can be listened to in Sonarflow for 30 seconds before you decide to buy.

“Lists simply fall short of the almost infinite variety of the iTunes music catalogue. The colourful Sonarflow Bubbles are much better suited for discovering new music, as they enable the user to zoom in and open up more and more music levels. Sonarflow for iTunes unlocks a whole new musical universe in an intuitive and interactive way”, says Thomas Lidy, CEO of Spectralmind, the company behind Sonarflow.

What makes Sonarflow iTunes so special?

- The iTunes music library is presented in an intuitive graphical interface. With zoom and swipe you can browse through music on many levels, displayed in coloured bubbles.

- Without Sonarflow and its “Discovery Mode”, the majority of the 28 million iTunes songs would most likely remain undiscovered.

- Sonarflow departs with the one dimensional level (Genre) and extends it with three others (Artist/Album/Song). Searching for music is substantially refined and song proposals become more specific.

- After finding appropriate music, play them immediately and create playlists by simply double-tapping on one of the bubbles.

- Share songs on Facebook, Twitter and via e-mail.


Sonarflow iTunes on iTunes Store:
Download press photos:

Copyright: Spectralmind


About Spectralmind:

Spectralmind (, established in Vienna, is a technology leader in the field of audio intelligence. The central subject is the semantic analysis technology that extracts the audio content that makes “search and selection” of media possible in a quick and easy way and in a visually appealing manner. The core competence of Spectralmind is in the automatic analysis of audio, especially music. With its focus on technology, Spectralmind is able to charge the content of music based on computer assistance which means describing the content automatically according to rhythm, timbre, etc. and establishing relations of similarities between music pieces.

With this knowledge about the semantics of a song together with common metadata such as title, artist, year, user evaluation, sales data, etc. their technology enables the rapid detection of desired music and audio data, both in end consumer area, as well as for professional firms in the area of music, advertising and film production.

Spectralmind on the Internet:

Sonarflow on Facebook:

Contact for editorial inquiries:

i5comm for Spectralmind

Bernhard Lehner

Phone: +43 664 439 86 09


Using an iPhone or iPad, music fans can now visually submerge into the Spotify universe of 18 million songs.

Sonarflow (, a free app available for iPhone and iPad (, offers a visual and intuitive way to browse through the catalogue of over 18 million songs available to Spotify Premium users.

The starting point of the Sonarflow visual music journey is its revolutionary new graphical  interface. It highlights artists and tracks in different coloured “Bubbles”. After opening the app, three bubbles, “Starred Spotify Tracks”, “Your Spotify Tracks” and “Country Top Tracks”, appear. Pinching one of these bubbles with two fingers leads you to the next level.

The “Sonarflow Magic”: Musical journey with “discovery-mode”

By enabling the discovery-mode, Sonarflow recommends music based on your personal taste as defined by Spotify. Sonarflow opens up a massive musical universe, ready to be discovered in an entirely new way.

“Some people may be overwhelmed by the massive amount of musical choice on Spotify,” The colourful Sonarflow Bubbles which allow you to easily zoom in and open up more and more details on any level, and are perfect for helping you to discover new music. Using intuitive and interactive browsing, Sonarflow for Spotify unlocks a whole new musical universe”, says Thomas Lidy, CEO of Spectralmind, the company behind Sonarflow.

How Sonarflow works on Spotify

The app connects to a Premium Spotify Account and visualizes music in different coloured “Playlist Bubbles”. By zooming into the bubbles, the songs of the playlist get visible. By activating the discovery mode, similar artists will be suggested in further bubbles. By touching one of those bubbles, you get information regarding the artist, cover art, videos and “Top Spotify Tracks” which can be played directly by the app. And what’s best – songs are not just sampled for 30 seconds like in other apps, Sonarflow on Spotify plays songs in full length.

And what exactly is so special about Sonarflow?

  • Even with no songs available on the iPhone or iPad, using the Spotify account, the app allows you to discover new music.
  • Sonarflow extends the one dimensional level (Artist) by two others (Album/Song). These three levels substantially improve the musical analysis and proposals become even more specific.
  • The app plays songs in full length.
  • By connecting to Spotify, you can discover music from a universe of 18 million songs.
  • Share songs on Facebook and Twitter.


Blog Sonarflow:

Download Sonarflow:

Download press photos:
Copyright: Spectralmind

About Spectralmind:

Spectralmind, founded in Vienna, Austria, is a technology leader in the field of audio intelligence. The central subject is the semantic analysis technology that extracts the audio content that makes “search and selection” of media possible in a quick and easy way and in a visually appealing manner.

The core competence of Spectralmind is the automated analysis of audio, especially music. With this technology, the content of music can be calculated based on computer assistance – describing the content automatically according to rhythm, tone, etc. and establishing similarities between songs.

Combining the knowledge about the semantics of a song with common metadata such as title, artist, year taken, customer evaluation, sales data, etc. enables this technology to detect desired music and audio data. This can be used both for the consumer market, as well as professional firms in the area of music, advertising and film production.


Spectralmind on the web:

For inquiries please contact:

i5comm für Spectralmind

Bernhard Lehner

Tel.: +43 664 439 86 09



*Number of tracks licensed globally. Catalogue size varies in each country.

Music distributors confront their customers with an overwhelming choice of music. Online music stores pride themselves with catalog sizes of 15 to 20 million tracks. It appears, as if the magnitude and completeness of music on stock is taken as a guarantee of the ability, to satisfy everybody’s listening desires, based on the idea, that potential customer should never turn their back on a particular music service, because of the unavailability of a particular piece of content. And record labels, once signed to make their content available, forcibly push their entire (back-) catalog into any distribution platform.

For customers, such masses of content are a like a huge, impenetrable jungle, which they hardly enter, or, if they dare, causes them a lot of effort to get around. Consumers restrict themselves to take note only of the freshest plants growing at the very edge of this jungle and, probably, a few of the old but bigger-than-the-rest landmark trees. Distributors know for long, that the increment of tracks made available not automatically translates into an increment of tracks sold.

Personally, I´m not much interested to see duplicates of the same recording released through endless numbers of compilations, re-issues or samplers. I find it a pain, to scroll through more of the same while discovering music. As a serious music listener (and buyer), a main driver of my personal music discovery and purchasing behavior is musical context: the stories behind a recording, the circumstances that influenced an artist to create or perform a piece. The particular era, which shaped an artist or which was shaped by an artist, whether it was italian Renaissance or 1970s hip-hop from the Bronx.
Unfortunately, contextual information in online music distribution seems to be largely neglected. At least, a few basics are in place:

Album art:

Most online music stores display album art. Probably used rather as colorful elements to break-up boring listings of tracks than intended to convey context. Album art are the photographs or graphical designs originally applied to the packaging of a physical sound recording. Why is this contextual information? The imagery on vinyl discs or CDs quite often relates directly to the conceptual idea behind the music. Album title, album art and the music in its particular sequence are a holistic synthesis of artistic expression. Much of this synthesis gets lost in online stores: e.g. customers see only the front cover, but never the back cover or the inside artwork. As well, the purchase of entire albums is on the wane, replaced by the cherry-picking of individual tracks.

Artist info and song lyrics:

What´s often lost as well is the gist of information related to the who, when, how etc. of a recording. This makes it very difficult to figure out the real recording date and place or the line-up of musicians in a particular performance. Some online stores present more or less accurate articles about artists or albums. These articles are helpful, but quite often I perceive them as isolated fragments of information, and, much worse, entirely unaffected by the principle blessing of the online age: the hyperlink. Extensive hyperlinking of artist- or album-related articles in online stores would represent a powerful driver of music discovery. Information of that kind is available online, but it resides in distinct, unrelated online silos. Same is the case with song lyrics.

Artist videos:

At least since the days of MTV we know about the importance of video for music. Music videos make YouTube probably the most frequented music destination site of the planet. Yet, the combined, well integrated display of artist videos next to the audio offering in online music stores is rarely to be found.

I guess, online stores still mimic to a large extent their physical predecessors. Only that online stores have a confusingly sized inventory on offer, but no helpful shopkeepers, capable to support shoppers with knowledgeable and supportive information. Online stores have not fully grasped the fact, that they could be more than marketplaces. Like physical malls, they could provide convenient and pleasurable, probably playful discovery experiences, which would turn out as stickiness attributes, purchase drivers, differentiators and reasons to come back. Deeper integration of contextual information, provided through carefully curated linkage and referencing of text, lyrics, videos, images and album art, should be the way to go.

In a recent blog-post, music analyst Mark Mulligan muses about a “Music start-up strategy 2.0“.

In the essence, he asks the question, whether or not a music startup necessarily needs to obtain music licenses from record labels. This is a question, which we discussed quite seriously at Spectralmind as well. Why would a music tech startup need music licenses?

Of course, we need large music catalogs to analyze them. The larger the better. Currently we work with sample libraries in the range of 100k items.  What we need is temporal access to music inventories for the sake of running the tracks in high-speed through our analysis software, without altering them in any way. But sole analysis does not necessarily presume to acquire music distribution licenses. This seems to be in line with what Mulligan says:

“…. in the more immediate term start-ups should look at ways to deliver their experiences without licenses.  No I’m not advocating the Groove Shark approach, but instead leveraging the content licenses of digital music services that are pursuing ambitious API strategies.  Music start-ups should think hard about whether they really need to own music licenses themselves to deliver a great user experience, or at least whether they need to right away ……. In the era of integrated music API’s it is no longer crucial for a music service to have its own licenses.  An investor wouldn’t expect a mobile app developer to own Android, iOS or Windows Mobile so they need not expect a music service to own music licenses.”

Mulligan addresses the era of “integrated music API´s”. In fact, there is a range of companies out there, some of them startups itself, which are striving to fuel a new wave of music applications by granting access to music, music metadata or other music related information.

In other words: the scope of upcoming music apps goes far beyond the creation of just another download storefront or just another streaming portal. Playback of music is certainly a central use-case, but there´s much more possible with music. The interaction of music consumers with their content is manifold, and with the broadening of digital listening experiences (e.g. through smartphones, cars, connected homes), new needs for contextual services to improve discovery, search or social interaction about music, emerge. This new breed of music apps does not only accommodate to consumer needs, they help to create differentiators for the established digital music distributors, each of them struggling to extend their footprint, if nothing else, than to generate the returns needed to cover the upfront payments for music licenses.

In the history of mobile music, content discovery has always been a challenge. Music stores represent a special kind of information overload. Exploring the depth of a super-sized content catalog, given the limited screen size of a mobile device, is a bit like doing the weekly shopping while looking through a matchbox cover.

Already in the days of the ringtone craze, music distributors thought of methods to improve the content exploration experience on small screens, hoping to create a discovery convenience that ultimately adds to the stickiness of the mobile storefront and that stimulates higher consumption. Since then, a flurry of content discovery approaches have been put in place.

music discovery on a mobile phone

Here’s a quick overview of some of the main discovery methods:

  • In the beginning was the browsing portal. Drop-off rates of more than 50% per menu level, even in popularity-based menu structures invalidated this model. The battle for main page presence was decided shortly after in favor of access categories like “new” (aka “novelties” or “latest additions”),  ”most wanted” (aka “charts”) combined with the display of lists of noteworthy, editorially selected albums or tracks.

  • Personalized storefronts. The idea is to rearrange a mobile storefront according to a user’s previous browsing history, assuming that the historic session would be indicative of the user’s preferences. Users appreciate personalization, but want popular content at other times.

  • Discovery through search. Valid idea, but only if the user knows exactly what to search for.

  • Recommendation engines. An attempt to infer a user’s music preference algorithmically from his/her past purchases, followed by suggestions of music bought by other users with similar preferences. Alternatively, recommendations are derived from human classification of content as the basis for the suggestion of matching titles. Such recommender systems have a permanent place in today’s music storefronts.

  • Social sharing and communities. This idea picks up the concept of “following” (another user and his/her purchases or music plays) or the sharing of playlists and their proliferation through social networks.

What we find in today’s music stores is a best-of-breed combination of all of the above. These are tried and tested methods. Still, we believe music discovery needs an innovative push. Music services are increasingly similar. They offer more or less the same content at the same price. They even look similar in terms of their user interfaces and they provide comparable user experiences. In short: music distribution needs differentiators to avoid commoditization.

Here at Spectralmind, we believe in data visualization as the new frontier of music discovery. Data visualization is an art, which attempts to turn even very big data sets into visual patterns, structures and elements, in order to make the data readable and understandable. There is no doubt that music represents an enormous body of data. The leading digital music distributors pride themselves on managing catalog sizes in the range of 15-20 million tracks. Visualization methods can repackage such volumes into easily accessible formats.

Our approach goes beyond the static visualization of data. In sonarflow, our visual music browser, graphical catalog visualization is the interface to navigate, operate and explore vast arrays of musical content and to expose music recommendations in a spacial and gestural environment.

This interface is capable of embracing core user needs for content discovery:

    • browsing through large stocks of content in an intuitive and seamless way

    • discovery through serendipitous expedition, ready to encounter music of unexpected relevance

    • personalization through playlist creation

    • social sharing

So hey, if you are in music distribution, don’t fall into the commodity trap. Get in touch, we would love to show you our approach.



Search Ain’t Misbehavin’

Searching music offside of the mainstream can be tedious. Recently i fell for a particular jazz piano genre, called “Harlem Stride Piano” while listening to a radio broadcast. Stride piano developed in the 1920s and 1930s in New York as an advancement from Ragtime. It is characterized by a rhythmic left hand play, where the pianist alternates a bass note or octave on the first and third beat with chords on the second and fourth beat, while the right hand plays the melody line.  This causes the left hand to leap great distances on the keyboard, often at neck-break speed. Back then, pianists like Fats Waller, James P. Johnson or Eubie Blake were famous stride virtuosos.

Louis Mazetier introduces harlem stride piano

Today, only a few pianists are capable to play stride, and I was curious to find out about contemporary ”Harlem Stride Piano” interpreters and recordings.

The textual search for “Harlem Stride Piano” in iTunes led to zero results. Even in the advanced search of iTunes, you can only search for artists and interpreters, title- or track names, but not for genres. A search just for “stride piano” brought up one album, fortunately carrying both terms in its title. Similar, Spotify´s search for “Harlem Stride Piano” did not match anything, whereas a search for “stride piano” returned a few albums because of the use of the terms “piano” and “stride” in their titles or tracks.

Still unsatisfied, i continued the search for contemporary stride players in Google, YouTube and Wikipedia to find out about artists like Louis Mazetier, Günther Straub or Bernd Lhotzky. Knowing their names finally helped me to find the desired tunes in iTunes and Spotify.

This little research clearly depicts the limits of text based music search. It´s results depend largely on the coincidental presence of the chosen search terms in the title or artist name. If you have nothing but a tune, search is often impossible. What´s missing is search for music based on the sounds of a sample track.

While chasing contemporary “Harlem Stride Piano” records through Spectralmind´s audio intelligence platform, I certainly would have used Fats Wallers “Ain´t Misbehavin“. For sure, a sound-similarity search would have brought up more and better results in far less time.

Musing About Music Similarity

When we demo Spectralmind’s SEARCH by Sound, a similarity search engine for music, we often realize how different the focus is on certain aspects of “similarity” among listeners. The similarity results calculated by the Spectralmind platform appear “similar” to one listener, but are judged as “not similar” by another or “somewhat similar” by a third.

Musical similarity is a very complex area and the reason for the deviations in judgement stems from the fact that similarity has so many dimensions. This raises the question, to which dimension do people relate when asked about the similarity of music?

Personally I observe that people try to exemplify similarity first of all from melody. The particular succession of higher and lower tones that form a melody is clearly a distinctive feature, which allows the listener to determine the degree of likeness or even closeness between two musical works.

Trombone Shorty at the Jazzfest Wien, 2011

Trombone Shorty at the Jazzfest Wien, 2011

But there are other dimensions of similarity as well:

  • Timbral similarity: timbre refers to the the tone color of a sound, which varies significantly among the characteristics of the sound-creating device, such as voice, string or wind instruments. As a listener we are able to identify the kinds of instruments playing, even in an ensemble like a band or an orchestra. The same melody played by a piano or a saxophone or a guitar makes a big difference in terms of timbral similarity.

  • Rhythmic similarity: rhythm is made up of a repeating pattern of sounds and silences. We perceive rhythm as fast or slow. Through rhythmic beats alone, we can set apart musical genres from each other, like rock from reggae.  Music, dance and even spoken language rely on rhythm as a main and defining element. Different rhythms can be put underneath the same melody (which can be highly entertaining or massively disturbing). This practical example of melodic similarity combined with rhythmic dissimilarity highlights the difficulty to assess an overall measure of similarity between two pieces of music.

  • Structural similarity: this refers to the occurrence of specific sections within a piece of music. Common sections are intro, verse, chorus (also known as refrain), interlude and outro among many more. These are formal criteria, which can be applied to describe constructive or sequential similarities of e.g. pop music songs or symphonic compositions.

There are many more dimensions of similarity beyond the ones mentioned. Some of them are even inaccessible to human perception, but very perceptible to musical data-mining programs such as the Spectralmind Audio Intelligence Platform.

Similarity decisions need to be judged by the rationale of the similarity search. Sometimes, melodic resemblance is the searched-for attribute. In other cases it might be rhythmic conformity or timbral affinity.  Or a mix of multiple qualities. The crucial factor is the intended use of the similar-sounding music. Having this intention in mind helps to escape a possible bias.

We are striving to improve our software in a way that makes its similarity opinion more comprehensible and transparent. Users have a desire to understand which dimensions of similarity the software uses to suggest something as similar.