Binary Elysium

Project: MP3tunes + Amarok Integration

Total Commits: 51 Weekly Commits: 12

Past 7 Days

It was another productive week in #amarok with over 150 commits! In the 12 of those that were mine I managed to do several things.

• Added elegant session handling to the MP3tunes service
• Fixed the collection search box, so you can filter your MP3tunes Locker.
• Enabled Copy To Collection functionality, so you can now copy (read: download) mp3tunes tracks to some other collection!

Of course, implementing those items wasn’t as simple as it sounds, but the features are essential and basic.

The search box (filtering) could use some improvement as currently it only filters via the artist field, but that is a limitation of the MP3tunes API. When I say it
“only filters via the artist field” I mean that it only matches against artists, so searching for a particular track name will not work. To fully support the filtering feature the MP3tunes API would need to allow you to do something like get a list of artists based on a partial track name in a single request. That is just one example, and yes, I could workaround it by doing multiple queries, however that would slow the entire operation significantly. Users expect the search fields in Amarok to be snappy, not take ~5 seconds per. token they supply. It is certainly not a showstopper, and it functions well enough for now, but hopefully MP3tunes will be open to expanding their API later on down the rode. To be fair I have never come across a web API that supported that sort of complex searching. The Ampache service in Amarok suffers from the same lack of functionality.

Interjection: Major props to my GSoC mentor, Nikolaj, for attempting to explain various parts of Amarok’s innards to me, not only once, but the several times it took to get the concepts through my thick skull. Also, he’s helped me track down several childish mistakes I’ve made when I was at my wits end trying to locate them. I can’t thank him enough. Hands down he’s the best GSoC mentor.

With the addition of “Copy to Collection” Amarok has taken a large step towards being fully integrated with MP3tunes. Up till this week all you could do was browse and stream your MP3tunes Locker. That is fine and dandy, but you could do that from the MP3tunes web player, their mobile player, your PS3, or any other number of their supported devices. However, none of those options allow you to seamlessly download and organize your stored music into your local music collection at the click of a button.

There is one shortcoming that needs to be addressed at some point before I’m satisfied: there is no progress indicator of any kind when you download tracks. The only way to see if tracks are being downloaded after you press Go is to watch the destination directory for changes. Thankfully this affects all collections you can “copy to/from”, not just MP3tunes, so perhaps someone else will feel inclined to whip up a progress indicator. There’s no such thing as a free lunch.

Upcoming 7 Days:

I have one big goal this week:

• Add MP3tunes Upload features

By Monday next week, you will be able to do Copy tracks from your local collection, Ampache collection, and the Magnatune database, to your MP3tunes locker.

There is quite a bit of work to be done before this can happen, but I will spare you the gritty implementation details until next week after I’ve committed the code where my mouth is (?).

Of course my weekly predictions wouldn’t be complete without a task to fall back on if I happen to complete the aforementioned task in a Ballmer-Fueled rage. After upload is in place there is only one major item left: Syncing. I need to break “Syncing” into manageable actionables (quite a term, eh?) and then lay out some mid-level designs for the process. Later on during the week I will dedicate an entire post to this topic.

Project: MP3tunes + Amarok Integration

Total Commits: 35 Weekly Commits: 15

Past 7 Days

This was a busy week. I migrated the existing MP3tunes service in Amarok 2 from making REST calls and parsing XML manually to use libmp3tunes. I created an object oriented encapsulation framework in C++ for libmp3tunes, which is written entirely in c. This means instead of mucking about crafting http queries and worrying about parsing data from XML one can manipulate the Locker in an OO fasion. Here’s a little snippet of how libmp3tunes saves work.

Without libmp3tunes if you wanted to fetch a list of artists this is what it would look like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

    QString urlString = "http://ws.mp3tunes.com/api/v1/lockerData?sid=<SESSION_ID>&partner_token=<PARTNER_TOKEN>&output=xml&type=artist";
 
    urlString = "http://ws.mp3tunes.com/api/v1/lockerSearch?output=xml&sid=<SESSION_ID>&partner_token=<PARTNER_TOKEN>&type=artist&s=" + m_artistFilter;
 
    urlString.replace( "<SESSION_ID>", m_sessionId);
    urlString.replace( "<PARTNER_TOKEN>", CENSORED);
 
    debug() << "url: " << urlString;
 
    m_storedTransferJob =  KIO::storedGet(  KUrl( urlString ), KIO::NoReload, KIO::HideProgressInfo );
        connect( m_storedTransferJob, SIGNAL( result( KJob * ) )
            , this, SLOT( artistDownloadComplete( KJob *) ) );
 
    ArtistList artists;
 
     //so lets figure out what we got here:
    QDomDocument doc( "reply" );
    doc.setContent( m_storedTransferJob->data() );
    QDomElement root = doc.firstChildElement( "mp3tunes" );
    root = root.firstChildElement( "artistList" );
 
    QDomNode n = root.firstChild();
    while( !n.isNull() )
    {
        QDomElement e = n.toElement(); // try to convert the node to an element.
 
        QDomElement element = n.firstChildElement("artistName");
        ServiceArtist * artist = new ServiceArtist( element.text() );
 
        element = n.firstChildElement("artistId");
        artist->setId( element.text().toInt() );
 
        ArtistPtr artistPtr( artist );
 
        artists.push_back( artistPtr );
 
        m_collection->acquireWriteLock();
        m_collection->addArtist( artistPtr );
        m_collection->releaseLock();
 
        n = n.nextSibling();
    }

With libmp3tunes it looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

    Mp3tunesArtistFetcher * artistFetcher = new Mp3tunesArtistFetcher( m_locker );
    connect( artistFetcher, SIGNAL( artistsFetched( QList<Mp3tunesLockerArtist> ) ), this, SLOT( artistDownloadComplete( QList<Mp3tunesLockerArtist> ) ) );
 
    ThreadWeaver::Weaver::instance()->enqueue( artistFetcher );
 
    ArtistList artists;
 
    foreach(Mp3tunesLockerArtist artist, artistList) {
        ServiceArtist * serviceArtist = new ServiceArtist( artist.artistName() );
 
        serviceArtist->setId( artist.artistId() );
 
        ArtistPtr artistPtr( serviceArtist );
 
        artists.push_back( artistPtr );
 
        m_collection->acquireWriteLock();
        m_collection->addArtist( artistPtr );
        m_collection->releaseLock();
 
    }

Both of those code samples produce this:

Notice that instead of looping through XML and ripping out data, I was able to call getter methods to retrieve the same data. Of course the XML parsing has only been moved to libmp3tunes, but by hiding the MP3tunes API implementation from Amarok it creates more maintainable code.

If none of that made much sense, no worries, the important bit to grasp is that libmp3tunes does these important things:

• Encapsulates the MP3tunes API so it is separate from the rest of Amarok.
• Provides an Object Oriented interface to the mp3tunes API. Creating a new session is as easy as Mp3TunesLocker locker = new Mp3tunesLocker();
• Ensures Amarok is officially supported by mp3tunes as long as they support libmp3tunes.

Upcoming 7 Days:

There are a few libmp3tunes shortcomings. One is the lack of a means to detect when a session has expired. Each MP3tunes API request requires a valid session (except of course the initial session-establishing request), and each session times out eventually. When using MP3tunes in Amarok it will be important to elegantly handle session timeouts, for the user does not care about sessions or timeouts. When the user clicks play on an artist they expect it to play, while currently, if the session has timed out Amarok doesn’t do anything. This week I will patch libmp3tunes to support detection of timed out sessions.

Another goal for this week is to fix the search box, so it actually searches.

Also, if you right click on an artist in the MP3tunes collection browser you get a “Copy to Collection” option. At the moment it doesn’t do anything. After this week is over, hopefully, selecting the “Copy to Collection” option will let you do just that.

Foreseeable Roadblocks:

Last week’s roadblock still stands. In the next few weeks I’ll be getting closer to the time when I will need to implement that syncing part of libmp3tunes into Amarok. The licensing issue won’t stop me from developing it on my own workstation of course, but it will have to be resolved before I can commit that part of library or code that implements it.

Project: MP3tunes + Amarok Integration

Prescriptum: These weekly reports will likely contain a bit of technical information that only other Amarok developers will understand. I do not like that idea, as I want these reports to be grokable by all, but since I haven’t yet decided on a format to present the info in a manner I like this will have to do for this first week.

Past 7 Days:

I started coding for GSoC last Tuesday (May 27th) beginning with a simplification of the ServiceCollection hierarchy by combining ServiceDynamicCollection and ServiceCollection into one class. I also started creating the ServiceCollectionLocation’s. It is not implemented anywhere yet, and won’t need to be for awhile, but I started it as at the time I was waiting on mp3tunes to deliver their c sdk.

Leeo kindly created mp3tunes icons, which I committed on Saturday.
and

During the week I kept nudging the mp3tunes developers to send me the c sdk, and Saturday the nudging paid off as I received a pre-release version of libmp3tunes. Even though it is a pre-release it is complete enough to match the current mp3tunes feature set in Amarok2. On Sunday I added libmp3tunes to the src tree, including the dependency detection. libmp3tunes is dependent on curl and libxml2, and if someone does not have them mp3tunes will be excluded from the build thanks to cmake.

Finally, today, I committed a ~750 line c++ wrapper for libmp3tunes to compartmentalize the unsightly c code.

You can see all my commits here:
http://kollide.net:8060/changelog/~author=link/Amarok

Upcoming 7 Days:

There are still a few functions left TODO in the c++ wrapper for libmp3tunes, so I’ll complete those this week. Then I plan to start migrating the existing service to use the library functions. I expect this will take all week, so my goal for the next report is to have the migration complete and a working Mp3tunes service utilizing the library. If my time estimation ends up being too long, and I finish the migration early, I’ll work on the Mp3tunesCollectionLocation functionality so you can copy tracks from Mp3tunes to the local collection in a manner similar to the Magantune service.

Forseeable Roadblocks:

The syncing part of libmp3tunes is not licensed and not complete. However, the MP3tunes developers have been particularly responsive the past several weeks, so I am optimistic they will pull through.

Reflections:

I am amazed at how much I learned this week. To keep this brief here is a list of some things I’ve learned:

• How to do basic cmake things like define optional deps and exclude portions of an app if they aren’t met.
• Before this week I had never even seen object oriented c before.
• Also I learned how to combine c and c++ code in the same program (extern!).
• KDE4 is a beast and breaking it is bad news.

Huzzah! Google announced the list of 1125 fortunate students that will be developing for Google’s Summer of Code program. I’m a member of this group, which means I will be writing code for an open source project and getting paid for it!

I was asked by my community mentor to write an introduction that was posted at various places, and I will re post it below.

Hey everyone! My name is Casey Link and I am an undergraduate Computer Science and Philosophy major at Virginia Tech in Blacksburg, VA. As a part of Google Summer of Code I will be tightly integrating the MP3Tunes service with Amarok 2 as well as implementing a generic synchronization framework between Collections. In Amarok 2, collections have been abstracted, allowing Amarok to treat a local SQL collection the same as an online Web Service based collection. Why is this awesome? With the addition of the service architecture (thanks to Nikolaj) it is now
possible to plug in remote, local, and soon portable collections without
having to differentiate between their interfaces.

Not convinced yet? Here is a real example of just how cool this is.

Enter MP3Tunes. MP3Tunes provides an “online music space” for its users to store their entire music collection so they can listen to their music anywhere. Using their API and client libraries, applications can be developed for just about any device, so you can literally listen to your music anywhere! This summer I will implement full MP3Tunes functionality into Amarok so you can seamlessly synchronize your local music collection with your MP3Tunes Locker. Not only will you be able to upload your local music collection, but if you add music to your MP3Tunes Locker via other methods – such as buying music from an online store – the new tracks will be added to your collection. This is just the beginning. Technology is seeing a trend towards ubiquitous connectivity and information storage in ‘the Cloud’. It won’t be long before “local” and “remote” are terms of the past as you are able to seamlessly access your information everywhere.

Amarok and MP3Tunes will be on the forefront of this technological shift. I hope you’ll be there with us.

In case anyone is interested I’ve posted my original project proposal that was a part of my GSoC application.