My Hack Week: The new Banshee

A couple of weeks ago I really started investing a lot of time into what will become the new Banshee. There are a few big goals for this effort, including a full client UI/services stack split, insane performance improvements, and greatly improved library management and playback support.

Banshee will be split into two core stacks: a “headless” services stack and the thick client UI stack (what one would traditionally call “Banshee”).

Headless Services

One of the biggest areas of focus for Banshee will soon be on platform integration and the ability to run different client UIs against Banshee’s core.

There will be a headless services stack, which will include the source manager (which in turn provides access to the library, playlists, and other sources) and the playback controller. Everything here will be exposed over D-Bus, and can be extended through Mono.Addins.

So for instance, F-Spot could integrate the Banshee library into its slide show functionality; PiTiVi could access the same to offer music to a user for video editing; Elisa could be extended to pull its content from Banshee; Joe’s Banshee Music Server will talk to this stack to serve a user their content from any browser in the world.

The services stack is highly efficient – serving data is extremely fast, and consumes very little memory – and yes, it’s running on Mono.

The Thick Client

Banshee’s primary UI won’t be changing too drastically, but there are some key things to note.

There will at last be a multi-select artist/album browser, even though I’m still not a fan of this feature. It is however the number one most requested feature. There will also be a play queue, which battles for the browser as the most requested feature.

Most importantly, I am dropping the GtkTreeView for our track view. The biggest problem in Banshee over the past two years has been the memory use and speed of the track view. This is because it is not possible to write custom tree models in Gtk# – which lacks GInterface support – and thus we have had to rely on the simple GtkListStore model. This is not good when you need to map directly to a database, for example. It requires loading everything from the database ahead of time, which means the startup time is O(N), and memory consumption the same.

Now, before you start jumping to conclusions or throwing out solutions to this problem that would still allow us to use the GtkTreeView, let me cover the available options:

  • Write the model in C, bind that. While this would work, it would mean we’d have to rely on a native library for Banshee (and I am trying to work away from libbanshee, which currently only does GStreamer work). Extending this model would be painful in the managed realm as well, as we really want this to be a strong extension point.
  • Fix Gtk# to support GInterfaces. Well this one is obvious. If Gtk# properly supported GInterfaces, I would have written a managed custom tree model long ago, but would then have been done with it. Read on, and I’ll tell you why I’m “glad” GInterfaces aren’t supported. However, this does seriously need to be fixed. It’s the biggest hole in our binding.
  • Use P/Invoke hacks to implement the custom model in C#. While possible, it’s quite ugly. It’s not so bad for simple models that aren’t meant to be reused, but when you need to implement sorting, filtering, drag and drop, and other features, this gets out of control quickly. It’s just a gross hack, not to be taken seriously. It says more for the native interop in Mono than anything else.

The solution? I started writing a new widget, similar to GtkTreeView, but only for lists at the moment. I have no intentions of supporting trees, as this widget is designed specifically with our Banshee needs in mind (but is meant to be used in other managed applications that need list-based data binding).

The first thing you will notice is that startup time is ridiculously fast. The view needs only to know two things from the model: a row count, and a method for fetching a row at an index. Simple.

Columns are managed by a ColumnController, which is not a part of the view (one of the things that irritates me about the GtkTreeView). Columns in the ColumnController provide cell renderers, much like those in core GTK. With this all pieced together, when a row comes into view, the model is queried. The object returned from the model is then used when the view iterates the columns in the ColumnController, which is then passed to each renderer for binding.

Objects returned by the model have attributes on either the class or its properties which binds them to certain columns. The renderer is then responsible for rendering the bound object or property thereof. Pretty simple, but wildly effective, and insanely fast.

Also, the width of the columns are percentage based, like ETable in Evolution. It’s slick. I hate horizontal scrolling. There is support for a minimum absolute width and completely fixed width columns.

I will save a more detailed technical overview for some developer doc on our Wiki. Now on to a screencast.

This first one shows the new view and browser operating on a very sane collection with a decent amount of cover art. But it’s not a very large collection and doesn’t do the performance of the view justice. It’s nice however. No comments on the music selection – this database was carefully crafted by hand from a collection of databases so I could have a “nice” database with full collections of albums and proper metadata – that is, it’s not mine!

Theora Screencast

This second one shows the startup performance. It really is O(1). However, sorting and filtering gets slow when you reach such a high value. With the use of indexes in sqlite and a cache table though, filtering and sorting works against the cache, not the primary tables, once an initial sort or filter is applied. That is, it gets faster as you narrow your results, as to be expected.

Theora Screencast

I don’t really think anyone has a million songs, but this new model/view can still cope, even if it is slightly sluggish. However, my goals for performance optimization are to be fast at 100k songs. I think that’s a pretty high average, and we’re almost there. Just some more SQL tuning and I think we’ll have it. The limitation currently is that sorting and filtering starts to get “sluggish” at about 75k rows on my machine.

Oh, and this database was generated by a tool I wrote yesterday that uses a dictionary to randomly create a music library. It came up with some rather interesting band names. Other projects might find it useful for performance testing.

I’ll close on this point just by leaving a few stats, all tests based on a 25,000 song database.

Old Banshee New Banshee
Startup Time 32 Seconds 0.8 Seconds
RSS Consumption 81 MB 8.5 MB

Now, these tests weren’t performed very scientifically, but the point is that the differences are so vast. What’s more is that the Old Banshee numbers are based off O(N) efficiency, while the New Banshee numbers are O(1) – the same startup time and memory consumption exists for the database with almost a million rows. It would probably take an hour or so to load that with Old Banshee, and I don’t think I have the memory for it.

Great, but where can I get this?

All of this work is being performed on a separate branch, currently called “list-view” in Banshee trunk. I do not recommend anyone use this new work yet, however. I can reasonably say that it will not munge your existing data, but it will migrate it into new tables.

There is still a lot of work to be done in the coming weeks, but I hope to merge this back into Banshee trunk proper sometime around GUADEC.

That is all for now.

41 Replies to “My Hack Week: The new Banshee”

  1. the new listview sounds a lot like the kind of thing TnyMail does to load huge numbers of emails.. I wish more applications used this approach.

  2. Anyone able to use that “rmcg” tool? It crashes with

    Unhandled Exception: Mono.Data.SqliteClient.SqliteSyntaxException: table CoreArtists has 2 columns but 5 values were supplied

    for me…

  3. @Eugenia: Nothing on the near horizon, but it’s something I would eventually like to do.

    @Justin: Yes, the concepts are similar. The difference is that TnyMail is written in C and can actually take advantage of custom tree models.

    @Tom: rmcg generates content for the database schemas used in “The New Banshee.” It will not work unless you run the new code first, which actually creates the tables. If you want to use this for your own application and testing, you will need to write a new writer class that accepts the tree from the generator.

    @Florian: as I said in the post, I am aiming to have this in main trunk sometime around GUADEC (3-4 weeks). That goal may be ambitious.

  4. Hey Aaron, this is amazing! But don’t forget to add Genre to the Artist/album browser, which is _really_ helpfull when you have soooo many bands.

    Any way, this is gonna be great!

  5. @Aaron: ah I see, I thought it would output a fresh test-database.

    I was wondering why I couldn’t run the demo app (list-view.exe) anymore… turned out the schema seems to be different. Didn’t know it migrated my banshee.db when I first ran it (about a week ago). Is there a way to convert it back to the old or to the very latest format?

  6. @Tom: When the new database code lands in the main trunk, any database schema revisions will be properly versioned and your database will upgrade itself across the diff of revisions. However, because the layout is currently so volatile, I am not versioning my changes yet.

    To fix this, all you need to do is run the following command if you notice that your database is no longer compatible with the new code:

    sqlite3 ~/.config/banshee/banshee.db “DROP TABLE CoreConfiguration”

  7. Pfft, this is just another lame attempt by Novell to try to gain street cred with open source people. I am pretty sure there is probably a M$ patent bomb in this somewhere.

  8. @Your Mom – Dude, just screw off. Seriously. Get your FUD out of here and keep your mouth closed while you have no idea what you’re talking about.

    Aaron – This is some great stuff you’re doing. I check out svn trunk nightly and I’m just simply in love with Banshee. Keep it up! :D

  9. Despite my continued reservations about mono, i think this is a great and fresh new approach to music playback and management on the desktop. I particularly like the multi client approach, similar to what telepathy is doing. It basically allows lots of helpful integration all over the place.

    Maybe it would be a good idea to start brainstorming uses for this – one I can think of it would be good if there was some integration into gnome-screensaver – perhaps showing artwork onscreen with the song meta data if there is a track playing. A desklet type thing would also be cool. A gaim plugin would be awesome as well, I’m not sure if any of these exist but just some thoughts :)

  10. It was about time someone dealt with the slugish performance of Banshee,
    good job, can’t wait for the final result !!!

  11. Does this mean that you will be able to use the headless server on one computer and connect multiple Banshee clients to it as “remote controls”?

    Most of the client/server systems available today are far behind Banshee and Rhythmbox in UI polish and features. If I was able to start using Banshee to pick music to be played on my stereo from any laptop or other computer in my house, that would be awesome.

  12. Will you be able to connect to MPD servers/use the Banshee frontend as a MPD client?

  13. Pingback: Sitzpisser
  14. The screencasts showing the ability to handle so many files would be greatly appreciated.
    BUT the new UI from Jakub is FREAKING AWESOME!!!

  15. Wow… fuckin frikin awesome, I can’t wait for it be in trunk.

    Aaron for president!!!…. or hacker of the week ;)

  16. Pingback: El nuevo Banshee
  17. Is it possible yet to use this code within Banshee, or do the C# files only compile to the UI described above?

    Either way, could you describe how to compile the current SVN code? (This is me learning C# and wanting to start contributing to Banshee at the same time.)

  18. Just a suggestion…

    Maybe make an option that deletes the old podcasts after you’ve listened or downloaded a new episode. Also, it would be nice for the deleted podcast episodes just to be greyed out in case we want to download them again.

    That and maybe the playlists in Banshee could sync with the iPod?

  19. Great job! .., the lasts weeks I was looking for some way to speed up filling a Gtk.Listview, but I found only some workarounds for C.
    I think this is a great achievement for Gtk#, something that needed to be done.
    At this time.., I’m using a Gtk.Treeview and filling it up to 90.000 sqlite retrieved records, obviously it takes several seconds..
    I’ll keep an eye on your work, for sure it will be useful for others applications that need to deal with large amount of datasets.


    Best, Pedro Guridi

  20. Great job so far on Banshee. It’s now my default player. Hey when will Banshee be able to load its library from a mounted drive (that is, over the network)?

Comments are closed.