Banshee in Summer of Code (Not)

Banshee was rejected from the Summer of Code, as was GStreamer. I’ll just echo that having GStreamer rejected came as a bit of a surprise, and it would be nice to have some kind of justification. Some of the projects approved are rather “interesting” to say the least. I’m just curious as to what the selection criteria is, how selections are made, etc. Or maybe it’s not my place to wonder or care.

However, I have put a few of the more high-profile Banshee SoC ideas under Mono, and maybe one or two could go under GNOME as well – perhaps the more GStreamer related ones? Maybe there are already too many under GNOME? Maybe we’re not cool enough? I was hoping to put one or two under GStreamer’s SoC if Banshee was rejected – but we both lose! A little sad only because there were lots of existing Banshee contributor-students looking to hack on it with dedicated time this summer. I only applied because of their eagerness and community involvement.

If you have any exciting ideas not listed, we’re open to them, but the number of applications we can accept for Banshee under Mono will be limited compared to what we could have accepted if we were not rejected as our own organization.

Audio profile configuration for the masses

Welcome to the second part of the “Things you may not know about Banshee” series of posts, where I highlight some cool features about Banshee that have been introduced in the 0.11.x series. I’m making up for all the blogging I haven’t done in the last 4-5 months.

Just before the GNOME Summit last year, I started working on a user friendly way to perform audio profile configuration. For example, selecting the desired bitrate for an MP3 stream. I had a few goals in mind at the time:

  1. Must be audio framework agnostic. Banshee supports both GStreamer and Helix, and this needs to work with both frameworks without any issues. The user should be able to configure profiles using the same interface and not know which audio framework will be doing the heavy lifting. Essentially, the audio profiles framework should not actually need to know anything about specific audio frameworks. Ever.

  2. Must provide a straight-forward, sensible user interface for configuring complex pipelines. The primary point here is that the user should never have to edit a raw pipeline. A user should not have to “know GStreamer” to change their desired encoding bitrate.

    The current GNOME audio profiles editor
    The current GNOME audio profiles editor :-(

  3. Multiple configurations should be supported on the same profile. Profiles are things like “MP3”, “Ogg Vorbis”, “FLAC”. The profile contains the pipeline and interface description. Configurations are sets of values that can be merged into and saved from a profile. This allows a user to configure a 128 Kbps MP3 encoding setting for their iPod and a 192 Kbps encoding setting for ripping CDs to their local library. Each configuration uses the same base profile, but its settings are different.

  4. Never show profiles that the user won’t be able to use. Not all users have the necessary components installed to be able to encode AAC or MP3 for example. Profiles for these formats should be provided, but if they won’t be able to run, they should not be shown. This means profiles should be tested against their default configuration values before ever presenting a user interface.

With all this in mind, I set out to write the beast. The user interface is defined in XML. Variables define a UI control type, possible values, etc. “Processes” are also defined in the XML with an audio framework ID, for instance “gstreamer.” For GStreamer, the process is the pipeline definition.

However, as of early this morning, the process definition is now an S-Expression. Before, it was simply a pipeline string that had $variables in it, which would be expanded based on the user configuration.

Since the GNOME Summit, I have been working with this profile stuff on and off. It’s been functional since Banshee 0.11.2 (a few months ago), but has been evolving in various ways since then. During this time it was clear that more expressiveness was needed for generating the actual process/pipeline definition. For example, in GStreamer if a user chooses to use VBR in LAME, the xingmux element should be added to the pipeline. However, xingmux is in gst-plugins-bad, and chances are not many users actually have xingmux. This means xingmux should only be appended to the pipeline if VBR is enabled and xingmux is actually available. Other reasons for needing more expressiveness are arguments for GStreamer elements that may be mutually exclusive. If I use mode X, I must provide arguments A and B but not C. If I use mode Y, I must provide arguments C but neither A nor B.

Last night I decided I needed to write an S-Expression evaluator to make this expressiveness a reality. 10 hours later, we now have SExpEngine, and it can do some really cool things. Functions are very easy to add to it and there are a number of built-in functions for logic, conditionals, comparisons, casting, arithmetic, and strings. It also supports variables, which can either be value types or a callback method that returns a tree.

I added a function to allow process S-Expressions to test sub-parts of a pipeline before merging it in to the resulting/final pipeline (think xingmux, from above). Additional GStreamer functionality can be added to build variations of a pipeline based on available elements, differences in GStreamer versions, etc. S-Expressions mean configurability (woo – fake words), reliability, and compatibility.

The result is something I’m quite happy with. For example, here is the S-Expression for the GStreamer LAME process:

+ "audioconvert ! "
"audio/x-raw-int,rate=" $sample_rate ",channels=" $channels " ! "
"audioconvert ! "
"lame mode=" $mode " "
(if (= $vbr_mode 0)
  (+ "bitrate=" $bitrate)
  (+ "vbr-mode=" $vbr_mode 
    " vbr-quality=" (- 9 $vbr_quality)
    (if (gst-element-is-available "xingmux")
      " ! xingmux"
(if (gst-element-is-available "id3v2mux")
  " ! id3v2mux"
  " ! id3mux"

To make sense of the variables, take a look at the full XML LAME profile..

Now, getting back to the “okay, why should I, as a user, care” side of things, I’ll close the post with a screencast (ooooh, fancy, I’ve never done one of these!) that shows all of the profile stuff in action. For the sake of also demoing how the S-Expression evaluates into a proper GStreamer pipeline, I ran Banshee in debug mode for most of the screencast, which shows a text view and a “Test S-Expr” button. Rest assured, if you’re running Banshee like a normal user, you’ll never see this part of the profile configuration dialog :-).

Banshee's audio profiles
What I hope one day can replace the GNOME audio profiles editor so applications other than Banshee can take advantage of the sweetness. Click the screenshot to watch the screencast. (Ogg/Theora).

I’m still working a lot of things out with this, but it’s my hopes to some day make this work outside of Banshee. It’s written with that in mind. At the very least, I’d like to make the XML profile and S-Expression format some kind of standard.

openSUSE Build Service

The openSUSE Build Service is really rocking. A lot of great effort has gone into it, and I hope it’s close to being more publicly open for new packagers.

I tried a few weeks ago to start managing upstream Banshee builds within it, but there were a number of issues and it just didn’t work out too well.

However, after giving it another shot yesterday, I’m very pleased to announce now that the very latest release of Banshee is currently and will always be available on the openSUSE build service. I have added targets for openSUSE 10.1 and Factory (soon to be 10.2, which, by the way, is really shaping up to be an outstanding release).

So, openSUSE Banshee users, add the appropriate repository below for your distribution and enjoy the updates:

Note to SLED 10 users: the openSUSE 10.1 packages will work on SLED, but you will lose the Helix Banshee support (RealPlayer/AAC/MP3) that was shipped with the distribution. I am trying to figure out the best way to manage Helix Banshee/SLED specific builds. What’s in the openSUSE Build Service is strictly 100% open source/upstream and works only with GStreamer.

I also hope to soon be able to host builds/packages for other non-SUSE distributions such as Fedora, Mandriva, Debian, and Ubuntu. We’ll have the infrastructure in place within the build service to support this, so we’re definitely going to want to take advantage of it when the time comes! I think it’s important to note however that this would not replace packaging done by the excellent individual distribution packagers/contributors – it’d merely be a way to offer an official bleeding-edge (well, still stable) repository for users on a distribution that wish not to wait for the next distro release cycle to get a major update.

Finally, James Ogley also wrote about GNOME:Stable, GNOME:Unstable, and GNOME:Community pieces of the build service, effectively replacing his old repositories. I’m really pleased with this new direction of what I’d like to see become more community-driven packaging.

Multimedia across the board

Multimedia in openSUSE 10.2 should also be a pleasant experience for users. Aside from getting the latest and greatest Banshee in, last week I spent time fixing and updating GStreamer and Totem and making sure we had all the right components selected by default to give the the potential for an awesome in-browser multimedia video streaming experience.

I’ve been testing 10.2 on all the Fluendo Commercial Beta plugins, and it’s quite nice to just be able visit any site with embedded video and have it play in your browser. All you’ll need to do is drop in the right GStreamer plugins – which is another topic altogether.

Cool stuff now in the wild

Quite the release, Mono 1.2 is ready for indulgence! Fantastic news also from the DBus camp – 1.0 “Blue Bird” was finally released upon us! Eexxccellent.

A smörgåsbord of text and an erratum

The History Channel Presents: gst-sharp

On the heels of my last post regarding GStreamer C# bindings, Alp Toker, original author of the first generation gst-sharp binding, provided me with a little more history regarding his binding.

“The ‘valiant effort’ to bind GStreamer in C# wasn’t just a couple of years ago, it was in 2002, and it targeted not GStreamer 0.8 but 0.4.1, making it one of the earliest GStreamer language bindings in use.”

I had thought the original binding started against 0.6.x, not 0.4.1, which is really cool and just one more reason why the new binding must become on par with other language bindings (PyGst).

Oh, yes, and some photos

I have posted some of my [less incriminating] photos from GUADEC. It was an absolute blast, and was wonderful to meet everyone in person finally. You can find my photos in the guadec2006 group on Flickr or in my F-Spot exported gallery. There’s way too much to write about on this subject, so I’ll reserve that for my thoughts.

Has anyone had problems listening to “Jono’s World of Metal”? For the record, Jono, Ted had everything to do with it. (actually, that’s completely false)

Just one more question: just who will have the honor of removing it? :-D

On the platter today…

My new head Sporting the ultra-sexy Fluendo hat, Christian delivered my new head for planet use. I like it very much. To whomever updates hackergotchi’s on various planets: would you mind doing me the honor?

GStreamer C# Binding Update

As many may know, there was once a valiant attempt to bind GStreamer in C# a couple of years ago against 0.8. However, this binding was never complete and had a number of problems, but it was nonetheless promising. That is until it eventually found itself without a maintainer.

For over a year it sat in Mono subversion, and during that time, a number of third-party developers assumed it was a working GStreamer binding for their beloved development platform. Unfortunately this was not the case, and a number of times myself and others had to explain that the binding was not complete and that the best thing to do was to develop the required GStreamer functionality in C, expose it in a simple P/Invokable API, and privately bind that. That’s what I do in Banshee, and it works very, very well.

I eventually moved the gst-sharp module out of the main source trunk to keep it from being “discovered” by new developers to save them the pain and excitement of seeing the apparent binding, and that worked well. After doing that, I found myself no longer having to explain the situation. But still, this was sub-optimal.

Now fast forward to just a couple of months ago. After Peter Johanson and I started talking about it, he managed to find a few hours spread over a few weeks to resurrect the old gst-sharp binding, and began fixing up its GAPI component. We then agreed that it might be a good idea to make it a Google Summer of Code project.

Now enter Khaled Abdul Karim Mohammed (who needs a blog): enthusiastic SoC student destined to deliver working GStreamer CLR bindings. We laid out a few goals for the summer project including extensive unit tests, runtime-bound GObject signal support, GAPI cleanup, sample porting, and some other minor tasks.

#region technical mumbo-jumbo

So now the personally most exciting and compelling reason for writing today: after a few weeks of hard work by Khaled, we now have one of the biggest goals for the binding essentially complete. Khaled took two ideas of mine that I had been toying with a few months ago and turned the best one into a working solution for supporting GObject signal binding at runtime.

This may sound a little weird, but it’s the best phrase I can come up with to describe it. In C, to connect to a signal you do something like g_signal_connect(object, "signal", callback). This isn’t so possible in the CLR without a decent amount of work, and it’s a major requirement for a GStreamer binding because elements aren’t targeted for binding, and thus GAPI doesn’t touch them, and can’t know what signals to bind.

GAPI-bound signals introduce an insane (but necessary) amount of code overhead to transform CLR events into a GObject signal binding. It’s not the kind of overhead you want to pass on to consumers of your binding :-).

Now what I mean by “GObject signal binding at runtime” is this:

Element typefind = ElementFactory.Make("typefind", "typefind");
typefind.Connect("have-type", delegate(object o, DynamicSignalArgs args) {
 Console.WriteLine("MimeType: {0}", args[1] as Gst.Caps);            

In C#, connecting to a signal is very much like how you would do it in C. A number of things had to happen under the hood to make this possible, but it’s much like it’s done in Python (or so Edward says). There’s a default C handler that is connected with G_CONNECT_SWAPPED, and the argument stack is walked to collect the arguments as GValues. There’s one caveat with GstMiniObject however, but we handle this with minimal overhead because GstMiniObject-derived types in the CLR binding are GLib.Opaque-derived. Subsequent Connect/Disconnect calls on the same signal and object then take advantage of multicast-delegate support in the CLR instead of having to manage multiple callbacks for a object-signal in GObject land.

This also means that adding by-hand bindings of common/base elements like decodebin or typefind to get C# event syntax is much easier.

All of this “dynamic signal” stuff is fully generic to glib-sharp, so it is my hope that after we fine-tune it and test drive it in gstreamer-sharp that it will end up in glib-sharp itself. It is an immensely useful feature to have if you need quick signal support on a GObject-based binding and can’t bring GAPI into the equation.


That all being said, and technical jargon aside, we are hard at work (Khaled for the most part right now) on a working, usable, sexy, GStreamer 0.10 binding for Mono/.NET/C#/Boo/VB.NET/IronPython/WhateverLanguageThatTargetsTheCLR. Much of it is already working now, but there is still much to do. Eventually I will port all of the Banshee backends to this lovely new binding.

To avoid the messy situation we lived with for almost a year with the first iteration of the GStreamer CLR binding for 0.8, I am currently not going to directly disclose where the new bindings are located, but for the adventurous at heart, I’m sure they can be found :-). Expect a first release in the coming months.

And, oh boy… I just can’t *wait* to see GStreamer apps written in VB.NET! (ahhh!!!H!!OMG!)

GStreamer up the Banshee

Way down in native land…

In the last couple of days, I’ve committed some shiny new GStreamer code to Banshee HEAD that has been “in the works” for quite some time (close to 4 months :-(). HEAD now features support for using GStreamer for tag reading, instead of Entagged. I think I am going to fall back on Entagged for certain potential failures in the GStreamer layer (no demuxer for a certain format, like WMA), but for now I’ve disabled Entagged to focus on testing the new GStreamer code.

This means that in libbanshee there is a new GstTagger API that is dedicated to synchronously parsing metadata. I’ve never done any synchronous operations in GStreamer 0.10, so I may not be doing something correctly, but I think it’s mostly right. Nevertheless, this code needs a lot of testing (hint, hint).

I also landed what is currently a static GStreamer element, mbtrm. This is a simple element that uses the MusicBrainz TRM API to calculate an acoustic fingerprint of a raw audio stream. Currently I am using the element in the CD ripping pipeline, like this:

cdparanoiasrc ! mbtrm ! ...

When a fingerprint has been calculated (currently only after EOS, which needs fixing), the element raises the ‘have-trm-id’ signal. I would like to make this happen before EOS so the TRM ID could possibly be set as a tag on the stream, but I’m mostly happy for now. At least the fingerprinting is decent. Currently the TRM ID is just printed to stdout, but eventually it will be used to query MusicBrainz for more track metadata, making it possible to fetch metadata for partial or mixed audio CDs. This too needs more testing.

Of course the element can be adapted in any pipeline to perform fingerprinting on any raw audio stream, not just for CD ripping (wouldn’t it be cool to have a “first pass” kind of option to fingerprint an entire imported library that may suffer from mangled or missing metadata ;-)).

And on the managed front…

I also moved HEAD to use gmcs! This means I will slowly introduce generics into newly written code where it makes sense. The keyword is slowly. Banshee has been running against the .NET 2.0 class libraries for over a week for me and I’ve yet to run into any unexpected problems. The GstTagger managed wrapper features the first snippet of generics, and it’s working great.

I’m really looking forward to introducing code that uses generics, nullable types, etc. where it makes sense and, over much, much, much time possibly migrating existing code to use generics.

Hopefully Banshee against gmcs/.NET 2.0 will re-enforce all of the great development and stability that has been going on in that front in Mono, and maybe we’ll start to see other applications migrate as well.