Introducing PodSleuth

PodSleuth is a new project I started last May that aims to probe, identify, and expose properties and metadata bound to iPods. It obsoletes libipoddevice, which had the same goals, but due to many reasons ended up being a mess of spaghetti code and hacks due to effectively being obsoleted overnight about a year ago when Apple made a particular firmware release.

PodSleuth takes the many lessons learned from libipoddevice and evolutions of the iPod itself and implements hardware support in a more future proof, yet backwards compatible and easy to maintain way.

The Technical

The core of its design is platform independent, although PodSleuth in its current and complete form is most usable in the context of HAL. A HAL layer is provided that is used now in Banshee, and we would like to see other certain iPod libraries at least optionally support PodSleuth. This layer is implemented as a HAL callout. Because the callout runs as the same user as the HAL daemon (typically root), it is allowed to cleanly perform operations that were implemented as hacks in libipoddevice (like reading from SCSI code pages).

The metadata that PodSleuth collects is then merged into the HAL device as properties. Once the PodSleuth callout ends, HAL finishes probing the device and exposes the iPod on the Global Device List (GDL) where the mount daemon (i.e. gnome-volume-manager) will then mount the iPod in a manner accessible to the user session. The key here is that by the time the iPod is exposed on the GDL, all PodSleuth properties will have been merged into the device. This makes application and user session library support for PodSleuth properties very easy and straight forward. One need only check for org.podsleuth.version on the device to check if PodSleuth ran against the device. If that property exists, other properties will be available that expose iPod metadata.

if (device.PropertyExists ("org.podsleuth.version")) {
    // We have a PodSleuthed iPod, it's ok to expect 
    // at least the minimum set of properties

    Console.WriteLine (String.Join (", ", device.GetPropertyStringList 
        ("org.podsleuth.ipod.capabilities")));
}

One of the most important changes that affects the end user directly is that the iPod will always be supported based on metadata extracted from the plist file located in the SCSI code pages. This means that even if the iPod is not found in the model table, no functional limitations will be imposed. That is, the model table is now only purely cosmetic (for presenting things like shell color and the right icon). This is in contrast to earlier versions of libipoddevice.

Additionally, PodSleuth sets the info.icon property in HAL which means that once we get new icons for the new iPods and update the existing ones to follow the new required naming specification, the proper iPod icon will finally show up in Nautilus and elsewhere (provided you are running a recent enough GNOME). A cute thing indeed.

The Files

PodSleuth will see its first official release within the next week, alongside a new ipod-sharp release, and Banshee 0.13.2. All of these releases spell out support for new iPods (Classics and Nanos), so stay tuned. PodSleuth is developed in GNOME subversion under the podsleuth module.

The Unfortunately Political

The fun part is that PodSleuth is written in C# of course, a detail that I like to think doesn’t matter, but it certainly and most unfortunately offends many for various reasons. There are two lovely facts that I like to point out when opposition to my language of choice for this project arises:

  • Because PodSleuth is not used as a library, any application written in any language with HAL support can benefit from PodSleuth. I could have written PodSleuth in shell, and it would not affect applications wishing to support it.

    Furthermore, PodSleuth requires only the ECMA approved portions of Mono (which is a 100% open source project and has a strict policy to keep patented code out) and Mono.Posix (which is an open source library wrapping POSIX system calls). Therefore there are no patent concerns (although I know the facts don’t matter to the noisy zealots).

    What matters on the technical side of this point is that most modern distributions ship at least this core subset of Mono these days, so I feel the dependency argument falls apart here as well.

  • My favorite argument against PodSleuth because it is Mono-based comes from the crowd that hates Mono for unfounded political reasons, yet require free software support for their iPod.

    Let’s see, the iPod is closed hardware, proprietary software, contains DRM, supports proprietary formats (doesn’t even have any open format support), and just recently added a hashing mechanism against the primary track database to lock out third party clients. I’m not sure if I’ve ever come across a more proprietary device, yet this zealot crowd has the nerve to scream about Mono!

    This theme is unfortunately all too common in our community (“Oh, it’s OK to run a Mac! Just don’t use Windows! The Mac is shiny too!”).

    In response to this very argument a few days ago in the #banshee IRC channel and mocking what could be seen as a double standard, Jorge Castro quipped,

    “I value freedom so I refuse to use Mono, oh look, a new iPod!”

    So true, so sad. On a somewhat related topic, I recently looked at the APSL next to the Ms-PL. Now that’s funny.

The Bottom Line

There is no reason an application or library focused around iPods shouldn’t support PodSleuth. Regardless of political alignment, PodSleuth is designed in such a way to make it very easy to optionally support. Supporting it technically places zero dependencies on a library or application. If a distribution happens to provide PodSleuth, then why not take advantage of the properties that would be readily available to you? They’re just strings in HAL! If the properties exist, use them. If not, fall back to what you’re doing today.

Think of your users! There’s a very small majority of users out there who will actually care about what a PodSleuth is, let alone what it’s written in. They just want their iPod to work, and PodSleuth provides a very solid hardware metadata layer in a clean language and platform agnostic way. Of course, these users should already be using or are already using Banshee, which has provided this experience from the get-go.

People who care about the political issue and oppose the platform shouldn’t be using iPods anyway, lest they be hypocrites.

Dogfooding Banshee’s new iPod support

I would like to hold an impromptu bug day, with very short notice, focusing on iPod support on Wednesday, October 10th, starting at around 1PM Eastern time in #banshee on Gimpnet. I have tossed together some quick instructions on building all the necessary components.

A lot has changed regarding iPod support in Banshee.

  • Using a new hardware layer called PodSleuth that integrates into HAL.
  • ipod-sharp has gone through some extensive device level refactoring and has been updated to support the new iPod Nanos and iPod Classics that have caused so much unfortunate commotion in the past few weeks.
  • Banshee has been updated to use the new ipod-sharp API and there are a handful of general iPod fixes at this level as well.

Anyone willing to test out this new code with an iPod of any kind (except the iPhone or iPod Touch) will be much appreciated. I plan on releasing Banshee 0.13.2 late this week or early next week with this stuff rolled in. Please follow the instructions above for building a fresh Banshee/iPod stack for the testing so everyone involved will be on the same slate.

I have created a component in Banshee’s bugzilla called iPodBugDay. Any bugs filed as a result of tomorrow’s testing should be filed under this component.

We need you! (and apple sucks – part 2)

Snorp and I absolutely hate Apple. With almost every release of a new iPod device, firmware, or even iTunes updates, there’s a chance that they have royally screwed us into not being compatible with them. Most often there are only minor changes to the iTunesDB database format, and they’re easy to cope with. Sometimes they hit hard.

A couple of days ago, they hit hard. We are now hard at work trying to figure out what to do next, and have almost solved it, but we could use your help. If you have not used iTunes 7, upgraded your firmware since the late-June release, or do not have one of those new-fangled evil iPods, we could use a file from your iPod to help us out.

It appears this last round of updates from Apple removes the iPod_Control/Device/SysInfo file or zeros it out, for good. Removing it (if zeroed) and rebooting does not restore it (what used to happen on older firmware). This file was critical in detecting all sorts of useful things about your iPod, and ultimately provided a key (model number) into our capability table in libipoddevice for the device (does the device support photos? videos? what resolutions does it prefer? what’s its generation? what icon should we display?).

To anyone with an iPod that hasn’t been touched by anything new from Apple in the last week, we are requesting your iPod_Control/Device/SysInfo. This file contains model information and your serial number. We are still able to find the serial number in other places on the device, and it just so happens that the serial number actually has encoded in it model and manufacturing information. My point here is that if you send us your SysInfo, we need the serial number along with everything else, so don’t mask it out – it would make your data donation useless.

There’s only one really unique part of the serial number, if you’re worried about sending that unique part in you can mask characters 6-8, which apparently are a 3 digit base 34 number identifying the production order of your iPod for that week of production at some factory. In other words, if your serial number is OOOOOOOOOOO, and you’re worried about privacy, security, whatever, send us OOOOOXXXOOO – O being characters we need, X are UID characters we don’t care about.

Email your SysInfo files to: aaron abock org — your help is much appreciated.

I also feel obligated to mention that this post is also on the heels of Crispin’s post about his new 8GB iPod Nano:

I was hoping for the ‘just work’ integration I have come to expect from Ubuntu, and sadly it doesn’t ‘just work’, lets take the apps I have tried one at a time:

  • Banshee – This doesn’t find my iPod at all
  • ipod – this is a little command line tool to test the libraries – it claims Not a Valid iPod!

(edited for brevity and relativity)

Well, Banshee can’t detect this new device for the very reasons explained above. The ‘ipod’ tool is a tool that comes with libipoddevice. It’s used for diagnostics, and doesn’t manage your iPod. It shows nothing for the same reason that Banshee shows nothing – libipoddevice is the common link, and Apple broke it.

This does lead me to want to add a fallback “unknown iPod” setup in libipoddevice. We can tell by just the USB vendor and product IDs if a device is an iPod… we just can’t detect the bells and whistles without the model number. Nevertheless it should still display something. Maybe even present a dialog asking for the model number manually, have it automatically collect data, and offer to submit it back to us somehow.