For customers, such masses of content are a like a huge, impenetrable jungle, which they hardly enter, or, if they dare, causes them a lot of effort to get around. Consumers restrict themselves to take note only of the freshest plants growing at the very edge of this jungle and, probably, a few of the old but bigger-than-the-rest landmark trees. Distributors know for long, that the increment of tracks made available not automatically translates into an increment of tracks sold.

Personally, I´m not much interested to see duplicates of the same recording released through endless numbers of compilations, re-issues or samplers. I find it a pain, to scroll through more of the same while discovering music. As a serious music listener (and buyer), a main driver of my personal music discovery and purchasing behavior is musical context: the stories behind a recording, the circumstances that influenced an artist to create or perform a piece. The particular era, which shaped an artist or which was shaped by an artist, whether it was italian Renaissance or 1970s hip-hop from the Bronx.
Unfortunately, contextual information in online music distribution seems to be largely neglected. At least, a few basics are in place:

Album art:

Most online music stores display album art. Probably used rather as colorful elements to break-up boring listings of tracks than intended to convey context. Album art are the photographs or graphical designs originally applied to the packaging of a physical sound recording. Why is this contextual information? The imagery on vinyl discs or CDs quite often relates directly to the conceptual idea behind the music. Album title, album art and the music in its particular sequence are a holistic synthesis of artistic expression. Much of this synthesis gets lost in online stores: e.g. customers see only the front cover, but never the back cover or the inside artwork. As well, the purchase of entire albums is on the wane, replaced by the cherry-picking of individual tracks.

Artist info and song lyrics:

What´s often lost as well is the gist of information related to the who, when, how etc. of a recording. This makes it very difficult to figure out the real recording date and place or the line-up of musicians in a particular performance. Some online stores present more or less accurate articles about artists or albums. These articles are helpful, but quite often I perceive them as isolated fragments of information, and, much worse, entirely unaffected by the principle blessing of the online age: the hyperlink. Extensive hyperlinking of artist- or album-related articles in online stores would represent a powerful driver of music discovery. Information of that kind is available online, but it resides in distinct, unrelated online silos. Same is the case with song lyrics.

Artist videos:

At least since the days of MTV we know about the importance of video for music. Music videos make YouTube probably the most frequented music destination site of the planet. Yet, the combined, well integrated display of artist videos next to the audio offering in online music stores is rarely to be found.

I guess, online stores still mimic to a large extent their physical predecessors. Only that online stores have a confusingly sized inventory on offer, but no helpful shopkeepers, capable to support shoppers with knowledgeable and supportive information. Online stores have not fully grasped the fact, that they could be more than marketplaces. Like physical malls, they could provide convenient and pleasurable, probably playful discovery experiences, which would turn out as stickiness attributes, purchase drivers, differentiators and reasons to come back. Deeper integration of contextual information, provided through carefully curated linkage and referencing of text, lyrics, videos, images and album art, should be the way to go.

In the essence, he asks the question, whether or not a music startup necessarily needs to obtain music licenses from record labels. This is a question, which we discussed quite seriously at Spectralmind as well. Why would a music tech startup need music licenses?

Of course, we need large music catalogs to analyze them. The larger the better. Currently we work with sample libraries in the range of 100k items.  What we need is temporal access to music inventories for the sake of running the tracks in high-speed through our analysis software, without altering them in any way. But sole analysis does not necessarily presume to acquire music distribution licenses. This seems to be in line with what Mulligan says:

“…. in the more immediate term start-ups should look at ways to deliver their experiences without licenses.  No I’m not advocating the Groove Shark approach, but instead leveraging the content licenses of digital music services that are pursuing ambitious API strategies.  Music start-ups should think hard about whether they really need to own music licenses themselves to deliver a great user experience, or at least whether they need to right away ……. In the era of integrated music API’s it is no longer crucial for a music service to have its own licenses.  An investor wouldn’t expect a mobile app developer to own Android, iOS or Windows Mobile so they need not expect a music service to own music licenses.”

Mulligan addresses the era of “integrated music API´s”. In fact, there is a range of companies out there, some of them startups itself, which are striving to fuel a new wave of music applications by granting access to music, music metadata or other music related information.

In other words: the scope of upcoming music apps goes far beyond the creation of just another download storefront or just another streaming portal. Playback of music is certainly a central use-case, but there´s much more possible with music. The interaction of music consumers with their content is manifold, and with the broadening of digital listening experiences (e.g. through smartphones, cars, connected homes), new needs for contextual services to improve discovery, search or social interaction about music, emerge. This new breed of music apps does not only accommodate to consumer needs, they help to create differentiators for the established digital music distributors, each of them struggling to extend their footprint, if nothing else, than to generate the returns needed to cover the upfront payments for music licenses.

Already in the days of the ringtone craze, music distributors thought of methods to improve the content exploration experience on small screens, hoping to create a discovery convenience that ultimately adds to the stickiness of the mobile storefront and that stimulates higher consumption. Since then, a flurry of content discovery approaches have been put in place.

Here’s a quick overview of some of the main discovery methods:

• In the beginning was the browsing portal. Drop-off rates of more than 50% per menu level, even in popularity-based menu structures invalidated this model. The battle for main page presence was decided shortly after in favor of access categories like “new” (aka “novelties” or “latest additions”),  ”most wanted” (aka “charts”) combined with the display of lists of noteworthy, editorially selected albums or tracks.

• Personalized storefronts. The idea is to rearrange a mobile storefront according to a user’s previous browsing history, assuming that the historic session would be indicative of the user’s preferences. Users appreciate personalization, but want popular content at other times.

• Discovery through search. Valid idea, but only if the user knows exactly what to search for.

• Recommendation engines. An attempt to infer a user’s music preference algorithmically from his/her past purchases, followed by suggestions of music bought by other users with similar preferences. Alternatively, recommendations are derived from human classification of content as the basis for the suggestion of matching titles. Such recommender systems have a permanent place in today’s music storefronts.

• Social sharing and communities. This idea picks up the concept of “following” (another user and his/her purchases or music plays) or the sharing of playlists and their proliferation through social networks.

What we find in today’s music stores is a best-of-breed combination of all of the above. These are tried and tested methods. Still, we believe music discovery needs an innovative push. Music services are increasingly similar. They offer more or less the same content at the same price. They even look similar in terms of their user interfaces and they provide comparable user experiences. In short: music distribution needs differentiators to avoid commoditization.

Here at Spectralmind, we believe in data visualization as the new frontier of music discovery. Data visualization is an art, which attempts to turn even very big data sets into visual patterns, structures and elements, in order to make the data readable and understandable. There is no doubt that music represents an enormous body of data. The leading digital music distributors pride themselves on managing catalog sizes in the range of 15-20 million tracks. Visualization methods can repackage such volumes into easily accessible formats.

Our approach goes beyond the static visualization of data. In sonarflow, our visual music browser, graphical catalog visualization is the interface to navigate, operate and explore vast arrays of musical content and to expose music recommendations in a spacial and gestural environment.

This interface is capable of embracing core user needs for content discovery:

• browsing through large stocks of content in an intuitive and seamless way

• discovery through serendipitous expedition, ready to encounter music of unexpected relevance

• personalization through playlist creation

• social sharing

So hey, if you are in music distribution, don’t fall into the commodity trap. Get in touch, we would love to show you our approach.

Louis Mazetier introduces harlem stride piano

Today, only a few pianists are capable to play stride, and I was curious to find out about contemporary ”Harlem Stride Piano” interpreters and recordings.

The textual search for “Harlem Stride Piano” in iTunes led to zero results. Even in the advanced search of iTunes, you can only search for artists and interpreters, title- or track names, but not for genres. A search just for “stride piano” brought up one album, fortunately carrying both terms in its title. Similar, Spotify´s search for “Harlem Stride Piano” did not match anything, whereas a search for “stride piano” returned a few albums because of the use of the terms “piano” and “stride” in their titles or tracks.

Still unsatisfied, i continued the search for contemporary stride players in Google, YouTube and Wikipedia to find out about artists like Louis Mazetier, Günther Straub or Bernd Lhotzky. Knowing their names finally helped me to find the desired tunes in iTunes and Spotify.

This little research clearly depicts the limits of text based music search. It´s results depend largely on the coincidental presence of the chosen search terms in the title or artist name. If you have nothing but a tune, search is often impossible. What´s missing is search for music based on the sounds of a sample track.

While chasing contemporary “Harlem Stride Piano” records through Spectralmind´s audio intelligence platform, I certainly would have used Fats Wallers “Ain´t Misbehavin“. For sure, a sound-similarity search would have brought up more and better results in far less time.

Musical similarity is a very complex area and the reason for the deviations in judgement stems from the fact that similarity has so many dimensions. This raises the question, to which dimension do people relate when asked about the similarity of music?

Personally I observe that people try to exemplify similarity first of all from melody. The particular succession of higher and lower tones that form a melody is clearly a distinctive feature, which allows the listener to determine the degree of likeness or even closeness between two musical works.

Trombone Shorty at the Jazzfest Wien, 2011

But there are other dimensions of similarity as well:

• Timbral similarity: timbre refers to the the tone color of a sound, which varies significantly among the characteristics of the sound-creating device, such as voice, string or wind instruments. As a listener we are able to identify the kinds of instruments playing, even in an ensemble like a band or an orchestra. The same melody played by a piano or a saxophone or a guitar makes a big difference in terms of timbral similarity.

• Rhythmic similarity: rhythm is made up of a repeating pattern of sounds and silences. We perceive rhythm as fast or slow. Through rhythmic beats alone, we can set apart musical genres from each other, like rock from reggae.  Music, dance and even spoken language rely on rhythm as a main and defining element. Different rhythms can be put underneath the same melody (which can be highly entertaining or massively disturbing). This practical example of melodic similarity combined with rhythmic dissimilarity highlights the difficulty to assess an overall measure of similarity between two pieces of music.

• Structural similarity: this refers to the occurrence of specific sections within a piece of music. Common sections are intro, verse, chorus (also known as refrain), interlude and outro among many more. These are formal criteria, which can be applied to describe constructive or sequential similarities of e.g. pop music songs or symphonic compositions.

There are many more dimensions of similarity beyond the ones mentioned. Some of them are even inaccessible to human perception, but very perceptible to musical data-mining programs such as the Spectralmind Audio Intelligence Platform.

Similarity decisions need to be judged by the rationale of the similarity search. Sometimes, melodic resemblance is the searched-for attribute. In other cases it might be rhythmic conformity or timbral affinity.  Or a mix of multiple qualities. The crucial factor is the intended use of the similar-sounding music. Having this intention in mind helps to escape a possible bias.

We are striving to improve our software in a way that makes its similarity opinion more comprehensible and transparent. Users have a desire to understand which dimensions of similarity the software uses to suggest something as similar.

Based on its VIEW by Sound visual interface, Spectralmind has released an iOS app called sonarflow for the iPhone, iPad and iPod touch. Download it from the App Store.

Sonarflow offers a beautiful user-interface, which reads the music from your iOS device and displays it as a series of interactive, dynamic bubbles. Sonarflow uses most of the iOS gestures and touch events to navigate your music content space.

Touch Navigation

• On the top level, the bubbles represent the various genres derived from your music assets

• Zoom in and out with a pinching gesture across a bubble. Pinch navigation lets you dive into a bubble to smoothly transition from genre level down to artist, album or track view and back again.

• Pan or drag bubbles in any direction, to move horizontally through the active content layer

• Tap a bubble once to reveal the tracks inside

• Double tap a bubble to play its items

• Tap and hold (long press) an item, to add it to a playlist

Features

In it’s current release (1.6.4), sonarflow offers the following features:

• Access content by pinch, tap, double tap and tap & hold gestures

• Playback music and create instant playlists from any zoom level

• Preview tracklist for all bubbles

• Playback history

• On-screen playback controls

• Visual cues on touch interactions

• Create multiple playlists

• See album artwork

• Read artist biographies provided by Last.fm

• Watch band videos on YouTube

• Share on Facebook and Twitter

• Supports scrobbling to Last.fm

• Support for iOS multitasking

• Support for Apple Airplay

• Support for Apple iCloud and WiFi sync

 ”Music is an art form whose medium is sound and silence. Its common elements are pitch (which governs melody and harmony), rhythm (and its associated concepts tempo, meter, and articulation), dynamics, and the sonic qualities of timbre and texture.”

In fact, these described elements of music are the ingredients Spectralmind uses for the creation of music tech products. Music is the base material from which we explore, analyze and extract information:

Algorithms, packaged into software, “listen” to music. What the algorithm “hears”, are music properties, including rhythm, timbre and many more.

Of course, a computer does not perceive music like humans do. Computers just calculate, they cannot take into consideration the cultural heritage, emotions and interpretations human listeners feel or are aware of.

 ”The border between music and noise is always culturally defined—which implies that, even within a single society, this border does not always pass through the same place; in short, there is rarely a consensus … By all accounts there is no single and intercultural universal concept defining what music might be.” (musicologist Jean-Jacques Nattiez, quoted in Wikipedia).

Applying a uniform algorithmic evaluation across a large number of music titles creates an objective mathematical description of each piece of analyzed music and, derived from here, an approach of comparability. We call it “music intelligence”. Such intelligence can be exploited in various ways like identifying music, determining similarities between music titles or organizing music. Still, there will always remain a gap between ”human understanding” and ”machine understanding” of music, as there will always be a gap in the understanding of music between human listeners.

“The creation, performance, significance, and even the definition of music vary according to culture and social context.”

Ever increasing sophistication of algorithms and availability of computational power lets us apply the music intelligence approach on large catalogs of music, thus eliminating great portions of cost and manual labor for large inventory music classification.

In comparison to such sensual searching in real life, searching for something on a computer is a poor experience. Search as we know it is limited to entering textual terms into a form, hoping for a result that is straightforward enough to get what we need. This works reasonably well with all things written. Computerized search is mainly search for written stuff.

But how do you search for something unwritten, like an image, a color, or a song? Search engines would require us to describe these things through words, using a language, forcing us into conventions of search terms and search operators. But how do you describe a piece of music within the narrow confines of a search engine’s syntax? How do you express these deeply subjective impressions a song leaves behind in your mind? What is not described in words, is hard to find. What can’t be described in words, remains hidden.

This problem is the baseline of what Spectralmind does: searching, finding and discovering music in addition to and beyond what can be expressed in words. As a result, Spectralmind brings seeing and hearing, the visual and acoustic senses, back into the digital search and discovery of music.

You’ve made it to this blog and we are happy to have you here with us. We would love to see you come back from time to time to learn more about Spectralmind, the way we approach music over and above bare tunes. Music is a carrier of rich and universal information, which we believe we can unleash through our technology, creativity and passion to give it the attention, it deserves.