Media players and the reader interface...

This is quite a long post, so I'm going to put the final line at the top too in case you don't read that far... ;-)

Your thoughts on media players would be most welcome!

The trouble, um, I mean, beauty of digital collections is that they redefine what a "manuscript" is. This is nothing new. Once upon a time someone somewhere probably upset the apple cart when they arrived at the hallowed doors with a basket full of photographs. Now we have video, audio and images, all of which can be encoded in any number of "standard" ways. (Not to mention a zillion different binary formats for just about any purpose you can imagine from sheet music to the latest car designs, which may well require more than just document-like presentation too - 3D models for example). These new manuscripts bring challenges for preservation, of course, but they also present challenges for presentation.

To address this, I've been learning more about media players in browsers with a view to picking one for the reader interface. I'm no expert in this field, so here is my layman's consideration of what I've found out and if you want to read more then this is great!

The traditional method to render audio/video in browsers, which pre-dates their ability to handle video themselves, is to use a browser plug-in, either directly (for example VLC plugin) or (more commonly) to build on top of Flash (eg. Flowplayer) or Java (eg. Cortado). The exact mark-up required to use these players varies. Some will simply use the "embed" tag and others have JavaScript libraries to simplify their usage and allow for graceful degradation in the event that the browser does not have the correct plug-in and/or understand/run JavaScript. (This may be an issue when we deploy the interface into a reading room with machines we do not control the configuration of).

But the times, they are a-changin'. Just as old browsers knew what to do when presented with an "img" tag, most modern browsers are beginning to support HTML5's "video" and "audio" tags, allowing the browser itself to handle the playback rather than farming this out to a plug-in. (For more on HTML5 generally see this presentation - the video tag is mentioned at about 58 minutes in). As an added bonus of bringing video into the browser in this way is it has inspired folks to build media players that manipulate the Web page to add the correct mark-up, be it a video tag, an embed, or whatever to play the media. This is currently being used to generate some nice media players that'll use the browser, the Flash-plugin, or whatever is available (see OpenVideoPlayer and OSMPlayer).

So now we get to the crux of it. What should we do for the reader interface? Go old-school (and annoy Steve Jobs) and use a Flash-based player? Adopt the new ways of HTML5? Insist on an Open Source player? Buy something in?

To work out the answer I did a bit of investigating and have installed most of the players mentioned thus far in this post - Flowplayer, OSMPlayer, video-tag only, VLC and Cortado, as well as JWPlayer.

Flowplayer uses the Flash-plugin to play Flash video (and, with an additional plug-in, MP3 audio) - it does not support Ogg. It is very simple to use and very slick to look at. It is open source, released under GPL3 with an additional (and reasonable) "attribution clause" which basically means the Flowplayer logo must appear on the player unless you pay extra.

JWPlayer works much like Flowplayer (though there is also a beta HTML5 video player in the making) and seems pretty good. While the source code is available, it is not clear if this is an open source product or otherwise - the source files do not include a LICENSE.txt or any boilerplate. Probably I'm just missing something there though, and JWPlayer seems a good choice if you don't mind Flash.

OSMPlayer is also open source and has numerous options for installation including a Drupal module (untested), a PHP library and a "stand-alone" configuration. In theory it supports lots of different audio and video formats and uses several divs to create a nice browser based player. Unfortunately, following the guidelines for both PHP and stand-alone configurations, I could not get it to work on my test server.

Video-tag only works pretty well with Firefox 3.6 on Ubuntu 10.04 and is very easy to include in a Web-page. Unfortunately it isn't nearly as slick at playback as Flowplayer - there is a delay in starting the video and it is unclear what is going on.

The VLC plug-in is also open source and seems to work pretty well and should be able to handle many different formats, but it isn't nearly as refined as other players and the provided example code fails to stop the video or make it full-screen. The VLC desktop player is wonderful, but I'm not convinced by the Firefox plug-in.

Cortado is a Java-applet provided to play Ogg Theora among other things. Usage is very simple - you just add an applet tag to the page - but playback is jerky, slow and lacked sound. I do not know if my machine is to blame for this or if it is the player itself so will have to investigate further.

Were I sat on and forced to make a choice I think I'd struggle. Flowplayer is slick to use and easy to implement, but requires we convert everything to Flash video or MP3 (mind you, most media will arrive in suitable formats I imagine). JWPlayer is very similar in this regard. I'd like to adopt the video-tag as this supports a wide range of formats, including open ones, but currently the experience is not very smooth and refinements in this area provided by things like OSMPlayer are still in their early stages of development. JWPlayer's HTML5 offering is still beta for example.

I guess my feeling for now is to either go with Flowplayer (and swallow the conversions required - actually pretty easy with ffmpeg) or spend a bit of time with OpenVideoPlayer's HTML5 work and the video tag. At this stage I think we probably need both working in the interface and see where the better user experience is...

I should throw one more thing into the pot - the problem of formats. Video and audio files are complicated beasts consisting of containers and tracks and such - a bit like cassettes! The contents of these containers are encoded in a variety of ways, each requiring different software to decode and render their content. We have the same problem with documents and we solve that by converting all the text-based materials we get into PDFs (for presentation before anyone starts worrying about the preservation implications of PDF!) and use a PDF plug-in to display them.

Can we do the same with our audio/video material and if we can, what format (I'm using "format" as a general term to mean "container/encoding"!) do we use? (Victoria has already done some work along these lines, creating WAVs for storage and MP3s for presentation, from audio CDs). Is there any additional concerns given that most born-digital video/audio is likely to arrive at our doors in a compressed format? Should we uncompress it? Is such a thing even possible? Should we (and do we have the processing power to) convert all audio/video materials to open formats for both preservation and presentation purposes?

We're going to raise this final question at our next Library developer meeting and see what folks think. In theory we can delay the decision because most browsers and their plug-ins handle multiple formats, but perhaps we should have a standard delivery format much like we currently have PDF?

Oh dear. I started writing this post with the hope of finding all the answers! I have found out a lot about media players at least, which can only be a good thing, and I've also found out that that state of the art is not quite as far along as the proponents of HTML5 killing Flash would like us to believe - though there is good work going on here and this is the future. I'm also unclear just how much my experience of these things is hindered by using Ubuntu - I often wrestle with the playback of media files under Linux! :-)

Still, I think we're further along, nearer an answer and at least in a place to know where to start testing...

Your thoughts on media players would be most welcome! :-)