Cross Platform thoughts through the lense of Banshee

Banshee running on OS X 10.5
Banshee 1.3.2 on Mac OS X 10.5

A huge amount of thanks goes to Eoin Hennessy for all of his efforts to bring Banshee to the Mac. Eoin singlehandedly fixed cross platform bugs in our core, worked out some kinks in the build, and implemented a Mac OS X platform backend to provide tight integration with the OS (i.e. the menu bar).

So how do I get it?

On Friday we will release Banshee 1.3.3, the last preview/developer release before we officially bless 1.4 as the new stable series in a couple of weeks.

As Miguel mentioned, we will be releasing for the Mac from here on. That means that along side our source code tarballs, RPMs for openSUSE, and packages published through other distributions, we will have a .dmg available to download.

If you are courageous, you can try to build Banshee on OS X today. However, our trunk does not yet run. Eoin has done all of his work in github, and while this branch runs, it’s slightly outdated, and does not use the new build environment I wrote this weekend.

Eoin is in the process of extracting patches that we will merge this week into trunk. These patches include the fixes to core, and the OS X integration.

Banshee running on OS X 10.5
More bling, before I ramble on about technical things and get teary-eyed and nostalgic

How does this cross platform stuff work?

Banshee is designed in a very modular way. We have a fairly small core, which is completely platform agnostic. It uses only ECMA/OSP .NET APIs and APIs of libraries which are themselves cross platform and open source (Mono.Addins, NDesk.DBus, Cairo, GTK, etc).

Additionally, all real features are designed as extensions: the Play Queue, video playback, the notification area, Last.FM, and so on. This means that we can easily pick and choose features by adding or removing assemblies from a package.

Finally, anything requiring a platform specific implementation is abstracted into an agnostic factory/interface, and implementations are loaded as extensions.

Cross platform frameworks give you the 90%, but it’s really the 10% that matters

It’s virtually impossible to write a cross platform application that does not suck if you just stick to what a single framework provides. By this I mean if we used only features in .NET or if a Java application used only features in the Java framework, the user experience would feel isolated and sandboxed.

  • In GNOME this means using GConf to store settings and working with DBus services like GNOME Settings Daemon. We interact with the screensaver, we support multimedia keys, integrate with Brasero for CD burning, and so on.

  • In the general UNIX world (Linux and BSD/OS X), we have a POSIX IO backend that performs much better than the System.IO .NET backend.

  • In Linux we have the HAL hardware backend (which I like to think of as HALAL, but don’t read too far into that).

  • On OS X we integrate with the dock and the system menu bar.

Okay, but what about Windows?

Ah, right. Windows. There’s one tiny little drawback to making a media player written in C# cross platform. Namely, we require GStreamer on all three platforms, and in turn we have a small library, libbanshee, written in C. Of course this pain is more than eased by the incredible power and functionality that GStreamer offers.

The truth is, I know next to nothing about Windows, and I find it incredibly frustrating to work in that environment. I am getting close however to getting a build of libbanshee on Windows. With this piece in place, we’ll be able to release on WIndows.

Currently the goal is that we’ll always distribute libbanshee as a binary on Windows, but the rest of Banshee can be built using “F5” in Visual Studio. This will be very interesting as it opens up Banshee to a whole new world of contributors.

Google Chrome has seen an incredible amount of contribution from external contributors simply because the source can be downloaded, opened with Visual Studio, and compiled. Users can tweak some code, and immediately see the results. This is where we want to be – Linux, Mac, and Windows.

The End Result

What is important with all of this cross platform work is simple: infiltrate the proprietary platforms, attract users and developers to free software, and grow the open source ecosystem. Ultimately, we’d like to think this will help Linux adoption. Applications are the key, not the platforms.

Banshee on Mac OS X is symbolic to me. It marks a period in our project where we step out of the realm of our comfortable and lovable GNOME community and into different communities, creating and bridging through the shared goal of open and free software. Users unite!

Finally, I’d again like to thank Eoin for his great efforts with our code, Imendio for their great work on GTK for OS X, the Mono team for the solid 2.0 release, and the GStreamer community for the best multimedia framework an application developer could ask for.

A few updates from the farm

Banshee Collection Indexer API

  • While Scott is working on all sorts of awesome, Gabriel and I are hard at work in mostly bug-fix mode to prepare for Banshee 1.4, to be released on November 10th.

    Tomorrow we will release Banshee 1.3.2, the third release in the development series leading up to 1.4.

  • Currently I am finishing up the new collection indexer API that allows other applications to index or monitor the Banshee library over DBus. Alex Launi is patiently acting as a guinea pig – implementing support in GNOME Do!

  • Check out the new Banshee Calendar that we’ll try to keep up to date and relevant, with new release information being added hopefully well in advance of actually making releases.

  • Mono 2.0 has finally been released – and the web site got a huge, long over due face lift!

  • And of course this weekend is the Boston GNOME summit! See you there!

Banshee 1.0 Released!

The Banshee logo

It is my immense pleasure to formally announce the release of Banshee 1.0. After nearly eight months of vigorous, non-stop work (since the last major Banshee release), it’s here – and we couldn’t be happier!

I’m not going to highlight much of the release in this post since we have written up some rather dashing release notes, full of pretty pictures and exciting detail.

Do yourself a service today, and try the release for yourself!

Install Banshee 1.0 – Binaries Ready Today

openSUSE Logo    Foresight Logo    Ubuntu Logo    Fedora Logo    Debian Logo    Mandriva Logo    Linux Logo

Release Highlights

I would however feel a little disappointed if I didn’t at least mention some of the awesomeness that Banshee 1.0 has in store, but really, you should read our release notes. We spent a lot of time on them!

  • Artist/Album browser
    Visually filter your collection with album art

  • Video playback and management
    Create video playlists, smart playlists, browse, search, and sort your video library — just like your music library

  • Powerful Podcasting
    Supporting video podcasts and stream content before or while you download — no waiting!

  • Rich Last.fm integration
    Create your own radio stations and discover new music

  • Play Queue Source
    You be the DJ

  • Lots of supported hardware
    MTP/PlaysForSure players, USB mass storage players, iPods, audio CD playback and ripping, CD burning

  • Performance improvements
    Better scalability, impressive speed, lower memory footprint

  • Compelling framework
    It’s easy to bring new features to life in Banshee through its powerful extension framework and rich APIs. In fact, most of the features in Banshee are extensions themselves. Drop into IRC and start hacking on your favorite feature today!

New Web Site — Finally

Gabriel and I have spent the last four days cranking out the new Banshee web site. We’ve still got tons of content to organize and migrate from the old wiki, but we think this new web site will become a strong asset to the Banshee Project in due time.

Why are you still reading this?

        Get It!

Digg It!

Mono.Zeroconf 0.7.4 and cross platform glory

I released Mono.Zeroconf 0.7.4 last night which is a minor bug fix release. Packages are available for openSUSE, SUSE Linux Enterprise, and Fedora. Sources and binaries are available for Linux, Windows and OS X.

What I have found interesting about Mono.Zeroconf is a growing adoption by the Windows crowd. I have been getting almost weekly emails from a new Windows developer using Mono.Zeroconf. A few bugs have been fixed in the process, though mostly in the Bonjour provider, which doesn’t affect modern Linux distros (because they use the Avahi provider, of course).

Since there’s been good feedback from the Windows world, I spent a little time making Mono.Zeroconf actually build on Windows in Visual Studio (yikes!). I am producing .zip files of the regular sources as well to make releases more accessible on Windows. Additionally, I generate a -binary.zip file which contains just pre-compiled assemblies.

The fun thing with all of this is validating how truly remarkable Mono is. I had never written any of Mono.Zeroconf specifically for Windows, never tested it on Windows, yet released the precompiled assemblies (built in Linux by Mono) for the hell of it. And without any changes, everything “just works” on Windows/.NET (and Mac OS X as well).

Leap
Andy made the leap to Linux.

So what’s so important about this? These new Mono.Zeroconf users on Windows inherit the fact that Mono.Zeroconf is truly cross platform, which means their applications are immediately more portable to Linux and other platforms that Mono supports. By luring Windows developers with delicious, easy to use libraries that are already cross platform, we make it easier and more enticing for those developers to bring their awesome applications to Linux.

Awesome. Mono rocks.

Announcing Mono.Zeroconf

I started working on Mono/.NET bindings to Bonjour/mDNSResponder almost two years ago. The code was quite useful but we found ourselves adding a lot of nasty preprocessor directives — yes, in C# — and configure junk to support both Mono.Zeroconf (Bonjour) and Avahi in software like daap-sharp and Banshee.

Earlier this year I found some time to start finishing my grand goal for Mono.Zeroconf — to support both Avahi and Bonjour through a simple, unified API.

Say what, exactly?

Today Mono.Zeroconf provides an easy to use API that covers the most common operations for mDNS (Browsing, Publishing, full TXT record support). It works either against Avahi (via the avahi-sharp bindings) or Bonjour on Linux, Windows, and Mac.

If your application needs to be cross platform (or even just cross-distro if you’re concerned about targeting older Linux distros) and perform Zeroconf networking, Mono.Zeroconf is the library for you. On Windows, the Mono.Zeroconf binaries work under both Mono and .NET.

It should be noted that Mono.Zeroconf does not support many of the more advanced mDNS features that Avahi in particular exposes. It is an intersection of the most common operations supported by both Bonjour and Avahi.

If your application needs advanced mDNS features, you most likely will care about the mDNS provider that you actually use (and I hope you choose Avahi). Therefore, I do not expect the functionality of Mono.Zeroconf to expand much and the current API is considered stable and complete. I may however add domain enumeration support in the future if it becomes a requested feature.

And how do I use it?

There are a couple of C# examples on the Mono.Zeroconf page that show how to browse, resolve, and publish services through Mono.Zeroconf. Here’s a quick adapted example on how to browse:

using Mono.Zeroconf;
...
ServiceBrowser browser = new ServiceBrowser ();
browser.ServiceAdded += delegate (object o, ServiceBrowseEventArgs args) {
    Console.WriteLine ("Found Service: {0}", args.Service.Name);
    // Connect to args.Service.Resolved event
    // Call args.Service.Resolve ()
};
browser.Browse ("_daap._tcp", "local");

There is also a small command line utility included in Mono.Zeroconf called mzclient. This works much like the traditional Bonjour command line tool, but it’s slightly nicer in ways I don’t remember. The source code to mzclient is probably the best example on what Mono.Zeroconf can do and how to do it.

Monodoc API documentation is also available.

Where is it?

Mono.Zeroconf will be part of the tarball set for the next Mono release (1.2.7). It is a separately maintained project and not bound to the Mono release schedule, but the upstream sources will be available in the same place. Packages will also be available through official Mono repositories.

For now, I have released sources on the Banshee web site. Packages (noarch) are available through the wonderful openSUSE Build Service for openSUSE Factory, 10.3, 10.2, 10.1; SLE 10; and Fedora 6, 7, and 8.

Sebastian Dröge is pushing it into Debian and Ubuntu, so packages should be available there soon. Will Farrington is working on a Conary recipe for Foresight.

Mono.Zeroconf is of course available in Mono SVN:

$ svn co http://anonsvn.mono-project.com/source/trunk/Mono.Zeroconf

Plugins, Addins, Extensions, oh my!

We are hard at work on the next generation of Banshee, and I’ll be posting details and juicy screencasts and screenshots soon. What I’m curious about now is what we should call Plugins in our UI. We use Mono.Addins in trunk now, and I have been thinking about renaming Plugins in the UI to something else, such as Extensions. My first thought was, “what does everyone else use?”

That wasn’t so helpful:

  • Banshee, Totem, gedit: Plugins
  • F-Spot, Epiphany, Firefox: Extensions
  • Tomboy: Add-ins

While this is just a small sampling of GNOME applications I thought of in 30 seconds, by far I think Tomboy loses. Add-ins, especially hyphenated, is weird. Note that Firefox seems to mix Extensions and Add-ons. Extra weird.

So the vote is between keeping it Plugins or switching to Extensions. I don’t care either way, but it would be nice to settle on some consistency within GNOME. Or maybe it doesn’t matter. Just a thought.

Some other names we have thought of include: Plugstensions, Bundleups, Snap-Ins, Snap-Ons, and snorp even recommended Butt-Plugs, but I don’t want to think too far into that one. I am sure Strap-Ons was in the running at some point too.

This is where you leave your €0.02. I hope it gets philosophical.

UPDATE: It seems like Extensions wins. Bugs filed against Totem, gedit, Rhythmbox, Tomboy, MonoDevelop, and Banshee. Other applications I’ve run across are already using Extensions, like F-Spot and Epiphany.

Introducing PodSleuth

PodSleuth is a new project I started last May that aims to probe, identify, and expose properties and metadata bound to iPods. It obsoletes libipoddevice, which had the same goals, but due to many reasons ended up being a mess of spaghetti code and hacks due to effectively being obsoleted overnight about a year ago when Apple made a particular firmware release.

PodSleuth takes the many lessons learned from libipoddevice and evolutions of the iPod itself and implements hardware support in a more future proof, yet backwards compatible and easy to maintain way.

The Technical

The core of its design is platform independent, although PodSleuth in its current and complete form is most usable in the context of HAL. A HAL layer is provided that is used now in Banshee, and we would like to see other certain iPod libraries at least optionally support PodSleuth. This layer is implemented as a HAL callout. Because the callout runs as the same user as the HAL daemon (typically root), it is allowed to cleanly perform operations that were implemented as hacks in libipoddevice (like reading from SCSI code pages).

The metadata that PodSleuth collects is then merged into the HAL device as properties. Once the PodSleuth callout ends, HAL finishes probing the device and exposes the iPod on the Global Device List (GDL) where the mount daemon (i.e. gnome-volume-manager) will then mount the iPod in a manner accessible to the user session. The key here is that by the time the iPod is exposed on the GDL, all PodSleuth properties will have been merged into the device. This makes application and user session library support for PodSleuth properties very easy and straight forward. One need only check for org.podsleuth.version on the device to check if PodSleuth ran against the device. If that property exists, other properties will be available that expose iPod metadata.

if (device.PropertyExists ("org.podsleuth.version")) {
    // We have a PodSleuthed iPod, it's ok to expect 
    // at least the minimum set of properties

    Console.WriteLine (String.Join (", ", device.GetPropertyStringList 
        ("org.podsleuth.ipod.capabilities")));
}

One of the most important changes that affects the end user directly is that the iPod will always be supported based on metadata extracted from the plist file located in the SCSI code pages. This means that even if the iPod is not found in the model table, no functional limitations will be imposed. That is, the model table is now only purely cosmetic (for presenting things like shell color and the right icon). This is in contrast to earlier versions of libipoddevice.

Additionally, PodSleuth sets the info.icon property in HAL which means that once we get new icons for the new iPods and update the existing ones to follow the new required naming specification, the proper iPod icon will finally show up in Nautilus and elsewhere (provided you are running a recent enough GNOME). A cute thing indeed.

The Files

PodSleuth will see its first official release within the next week, alongside a new ipod-sharp release, and Banshee 0.13.2. All of these releases spell out support for new iPods (Classics and Nanos), so stay tuned. PodSleuth is developed in GNOME subversion under the podsleuth module.

The Unfortunately Political

The fun part is that PodSleuth is written in C# of course, a detail that I like to think doesn’t matter, but it certainly and most unfortunately offends many for various reasons. There are two lovely facts that I like to point out when opposition to my language of choice for this project arises:

  • Because PodSleuth is not used as a library, any application written in any language with HAL support can benefit from PodSleuth. I could have written PodSleuth in shell, and it would not affect applications wishing to support it.

    Furthermore, PodSleuth requires only the ECMA approved portions of Mono (which is a 100% open source project and has a strict policy to keep patented code out) and Mono.Posix (which is an open source library wrapping POSIX system calls). Therefore there are no patent concerns (although I know the facts don’t matter to the noisy zealots).

    What matters on the technical side of this point is that most modern distributions ship at least this core subset of Mono these days, so I feel the dependency argument falls apart here as well.

  • My favorite argument against PodSleuth because it is Mono-based comes from the crowd that hates Mono for unfounded political reasons, yet require free software support for their iPod.

    Let’s see, the iPod is closed hardware, proprietary software, contains DRM, supports proprietary formats (doesn’t even have any open format support), and just recently added a hashing mechanism against the primary track database to lock out third party clients. I’m not sure if I’ve ever come across a more proprietary device, yet this zealot crowd has the nerve to scream about Mono!

    This theme is unfortunately all too common in our community (“Oh, it’s OK to run a Mac! Just don’t use Windows! The Mac is shiny too!”).

    In response to this very argument a few days ago in the #banshee IRC channel and mocking what could be seen as a double standard, Jorge Castro quipped,

    “I value freedom so I refuse to use Mono, oh look, a new iPod!”

    So true, so sad. On a somewhat related topic, I recently looked at the APSL next to the Ms-PL. Now that’s funny.

The Bottom Line

There is no reason an application or library focused around iPods shouldn’t support PodSleuth. Regardless of political alignment, PodSleuth is designed in such a way to make it very easy to optionally support. Supporting it technically places zero dependencies on a library or application. If a distribution happens to provide PodSleuth, then why not take advantage of the properties that would be readily available to you? They’re just strings in HAL! If the properties exist, use them. If not, fall back to what you’re doing today.

Think of your users! There’s a very small majority of users out there who will actually care about what a PodSleuth is, let alone what it’s written in. They just want their iPod to work, and PodSleuth provides a very solid hardware metadata layer in a clean language and platform agnostic way. Of course, these users should already be using or are already using Banshee, which has provided this experience from the get-go.

People who care about the political issue and oppose the platform shouldn’t be using iPods anyway, lest they be hypocrites.

My Hack Week: The new Banshee

A couple of weeks ago I really started investing a lot of time into what will become the new Banshee. There are a few big goals for this effort, including a full client UI/services stack split, insane performance improvements, and greatly improved library management and playback support.

Banshee will be split into two core stacks: a “headless” services stack and the thick client UI stack (what one would traditionally call “Banshee”).

Headless Services

One of the biggest areas of focus for Banshee will soon be on platform integration and the ability to run different client UIs against Banshee’s core.

There will be a headless services stack, which will include the source manager (which in turn provides access to the library, playlists, and other sources) and the playback controller. Everything here will be exposed over D-Bus, and can be extended through Mono.Addins.

So for instance, F-Spot could integrate the Banshee library into its slide show functionality; PiTiVi could access the same to offer music to a user for video editing; Elisa could be extended to pull its content from Banshee; Joe’s Banshee Music Server will talk to this stack to serve a user their content from any browser in the world.

The services stack is highly efficient – serving data is extremely fast, and consumes very little memory – and yes, it’s running on Mono.

The Thick Client

Banshee’s primary UI won’t be changing too drastically, but there are some key things to note.

There will at last be a multi-select artist/album browser, even though I’m still not a fan of this feature. It is however the number one most requested feature. There will also be a play queue, which battles for the browser as the most requested feature.

Most importantly, I am dropping the GtkTreeView for our track view. The biggest problem in Banshee over the past two years has been the memory use and speed of the track view. This is because it is not possible to write custom tree models in Gtk# – which lacks GInterface support – and thus we have had to rely on the simple GtkListStore model. This is not good when you need to map directly to a database, for example. It requires loading everything from the database ahead of time, which means the startup time is O(N), and memory consumption the same.

Now, before you start jumping to conclusions or throwing out solutions to this problem that would still allow us to use the GtkTreeView, let me cover the available options:

  • Write the model in C, bind that. While this would work, it would mean we’d have to rely on a native library for Banshee (and I am trying to work away from libbanshee, which currently only does GStreamer work). Extending this model would be painful in the managed realm as well, as we really want this to be a strong extension point.
  • Fix Gtk# to support GInterfaces. Well this one is obvious. If Gtk# properly supported GInterfaces, I would have written a managed custom tree model long ago, but would then have been done with it. Read on, and I’ll tell you why I’m “glad” GInterfaces aren’t supported. However, this does seriously need to be fixed. It’s the biggest hole in our binding.
  • Use P/Invoke hacks to implement the custom model in C#. While possible, it’s quite ugly. It’s not so bad for simple models that aren’t meant to be reused, but when you need to implement sorting, filtering, drag and drop, and other features, this gets out of control quickly. It’s just a gross hack, not to be taken seriously. It says more for the native interop in Mono than anything else.

The solution? I started writing a new widget, similar to GtkTreeView, but only for lists at the moment. I have no intentions of supporting trees, as this widget is designed specifically with our Banshee needs in mind (but is meant to be used in other managed applications that need list-based data binding).

The first thing you will notice is that startup time is ridiculously fast. The view needs only to know two things from the model: a row count, and a method for fetching a row at an index. Simple.

Columns are managed by a ColumnController, which is not a part of the view (one of the things that irritates me about the GtkTreeView). Columns in the ColumnController provide cell renderers, much like those in core GTK. With this all pieced together, when a row comes into view, the model is queried. The object returned from the model is then used when the view iterates the columns in the ColumnController, which is then passed to each renderer for binding.

Objects returned by the model have attributes on either the class or its properties which binds them to certain columns. The renderer is then responsible for rendering the bound object or property thereof. Pretty simple, but wildly effective, and insanely fast.

Also, the width of the columns are percentage based, like ETable in Evolution. It’s slick. I hate horizontal scrolling. There is support for a minimum absolute width and completely fixed width columns.

I will save a more detailed technical overview for some developer doc on our Wiki. Now on to a screencast.

This first one shows the new view and browser operating on a very sane collection with a decent amount of cover art. But it’s not a very large collection and doesn’t do the performance of the view justice. It’s nice however. No comments on the music selection – this database was carefully crafted by hand from a collection of databases so I could have a “nice” database with full collections of albums and proper metadata – that is, it’s not mine!

Theora Screencast

This second one shows the startup performance. It really is O(1). However, sorting and filtering gets slow when you reach such a high value. With the use of indexes in sqlite and a cache table though, filtering and sorting works against the cache, not the primary tables, once an initial sort or filter is applied. That is, it gets faster as you narrow your results, as to be expected.

Theora Screencast

I don’t really think anyone has a million songs, but this new model/view can still cope, even if it is slightly sluggish. However, my goals for performance optimization are to be fast at 100k songs. I think that’s a pretty high average, and we’re almost there. Just some more SQL tuning and I think we’ll have it. The limitation currently is that sorting and filtering starts to get “sluggish” at about 75k rows on my machine.

Oh, and this database was generated by a tool I wrote yesterday that uses a dictionary to randomly create a music library. It came up with some rather interesting band names. Other projects might find it useful for performance testing.

I’ll close on this point just by leaving a few stats, all tests based on a 25,000 song database.

Old Banshee New Banshee
Startup Time 32 Seconds 0.8 Seconds
RSS Consumption 81 MB 8.5 MB

Now, these tests weren’t performed very scientifically, but the point is that the differences are so vast. What’s more is that the Old Banshee numbers are based off O(N) efficiency, while the New Banshee numbers are O(1) – the same startup time and memory consumption exists for the database with almost a million rows. It would probably take an hour or so to load that with Old Banshee, and I don’t think I have the memory for it.

Great, but where can I get this?

All of this work is being performed on a separate branch, currently called “list-view” in Banshee trunk. I do not recommend anyone use this new work yet, however. I can reasonably say that it will not munge your existing data, but it will migrate it into new tables.

There is still a lot of work to be done in the coming weeks, but I hope to merge this back into Banshee trunk proper sometime around GUADEC.

That is all for now.

Surfing the Tubes

  • Novell and EFF team on patent reformAwesomeness

    “I think it’s fair to say that we’ve been talking a lot about patents within Novell in the last twelve months. More than we have in the past,” he said. Novell concluded that the patent system is broken and that patents are hobbling innovation and open standards, by putting companies on the defensive.
    –Nat Friedman

  • My good friend Jeff Tickle is working on an awesome Tomboy/Gtk# project called Virtual Paper, and has been making some wonderful progress. He could really use some expert Cairo advise regarding custom gradients on a stroke. I think Jeff should always blog from Virtual Paper itself and never again use the keyboard.

  • Late last week Miguel started white-boarding the drawing engine for Moonlight, and has made some progress since getting his canvas to draw a rectangle.

  • After a frantic brute-force apartment hunt in Boston last Thursday and Friday, I found a nice place in Kenmore/Back Bay, and will be moving out of my apartment in Raleigh at the end of next week. I’ll spend a week at my parents’ house visiting with family we have coming into town, and then will drive up to Boston on the 10th. I can’t wait!

  • I have been working on some major performance improvements in Banshee and will write more about this later. 0.13.x is going to be completely awesome and noticeably faster. Unfortunately I’ve spent so much time this month preparing for the move to Boston that I haven’t been able to spend as much time as I want hacking. The good thing is that I am in one of those mad-hacking moods where I stay up all night and get real work done. These come and go in cycles, and usually last a long time.