I’ve been hard at work over the last few weeks to iron out some of the main blockers to the Linux port of Voxeliens. One of the major hurdles I’ve been coming up against is the the dangerous area that is ABI compatibility on Linux.

ABI compatibility

For those that don’t know, an ABI (application binary interface) is what allows you to use existing libraries of code in your application, whether they’re things you’re in control of (like our PolyVox library) or system libraries (like glibc or the C++ standard library). For the purposes of the discussion here, an ABI is defined by the symbols that a library contains where symbols are things like functions. As a library evolves over the years they will add functions and each time a release of the library is made with new symbols the library ABI version will increase. Most library developers (and this is particularly true of system libraries like glibc) will promise to never remove any symbols from their libraries which means that an application built against an old version of glibc will still work when run on a new system as all those symbols will still be present (this is called forward compatibility).

The opposite cannot be said to be true. If you build an application on the latest greatest version of your favourite Linux distribution it will freely try to use newer symbols in its dependent libraries. If you then copy that executable to an older distribution and try to run it, it will balk and complain that it can’t find the required symbols. There are tricks one can play to trick the compiler into only using symbols from an old ABI version but the simplest way to achieve the goal is to install an old distribution in a virtual machine and compile all your code there.

This is what I’ve been doing for the past few weeks. I created a VM with a 32 bit installation of openSUSE 11.1 (released December 2008) and have recompiled Voxeliens and all its dependencies in the most minimal way I can. The reason I am doing this in a 32 bit VM rather than a 64 bit one is that 32 bit binaries (technically i686) are compatible with 64 bit operating systems (technically x86_64) whereas the inverse is not true. Ideally these binaries built in the 32 bit VM will work on any Linux OS released in the last 3 or 4 years regardless of whether it’s 32 or 64 bit. This will be one of the main things we’ll need testers for when the time comes.

Bundling libraries

For low level system libraries there’s little choice apart from building against an old version. However, for higher level libraries we have more options. For some libraries with liberal open source licenses we can simply statically link them which essentially involves bundling the library’s code directly into the final executable – thereby avoiding and ABI problems. Any library under a license like the GPL or LGPL, however, cannot legally be statically linked and so the simplest solution here is to include the library file along with the game and tell the executable to use the bundled version rather than any provided by the system.

Shaders are complete

David’s done some excellent work to port all the shaders over to GLSL. In the last update you saw that there was nothing there but flat colours but now everything, including shadows and lighting is working.

What’s left to do?

So now that the building of a portable version of Voxeliens is complete, there’s a few areas that need finishing before it can be released:

• Sound. The state of sound output on Linux is a little confusing and not as simple as on Windows. The main problem we’re encountering is that Phonon (the sound API from KDE/Qt), while generally excellent does not provide a sound mixer and so only allows one sound to be played at once. In Voxeliens we often have multiple explosions, laser shots and the music all playing and so we’re investigating using SDL_mixer to mix our sound for us.
• There’s still some checks to be done to ensure that the X11 keyboard handling is working as it should be.
• Finally I need to analyse the distribution package, remove debug symbols, streamline running the application etc.

Once these are sorted, it will be ready for testing among a wider audience and we’ll put out a request for beta testers nearer the time.

Following on from David’s work to port the shaders to GLSL, I’ve been working over the last week or so to get the code compiling for the Linux port. Fortunately Voxeliens is built on top of a number of fantastic open-source projects such as OGRE and Qt which work perfectly on Linux already. As far as our own code, Voxeliens make extensive use of our PolyVox library which I have all along been building and testing on Linux.

Since all our dependencies were working fine, it simply became a case of getting the Voxeliens code itself compiling. This was largely a trivial job, with the bulk of the work involving fixing incorrect case in C++ includes and media files. The largest single change that had to be made was to the keyboard handling. Qt doesn’t provide access to the low level keyboard APIs on Windows or Linux so I had to write some native X11 code to complement the windows.h code we already had.

We still have some work to do on the GLSL shaders to get shadows working and to ensure compatibility with OpenGL 2.1 (since that seems the best target for Linux) and then it’s mostly the work to get it packaged up for distribution across all the different flavours and versions of Linux. Sorting out shared libraries and dependencies is going to be the most amount of work. We’ll be putting out a call for testers once it’s nearer to release to hammer out any problems in this area.

I’ll write up a more technical, in-depth post after the release about the porting process for those who are interested.

This weekend I’ve started the process of getting Voxeliens to run under OpenGL as well as Direct3D. A large part of this transition is handled automatically as we use Ogre as our rendering system and it already has support for both APIs. However, we do need to convert all our HLSL shaders into GLSL versions.

So far this transition is going fairly smoothly, and the screenshot below is taken from the new OpenGL version. As you can see, the voxel colours are coming through but the lighting and shadows are missing. Shadows are probably going to be the trickiest part but hopefully it will come together over the next week of so.

Although I’m still working on Windows, the motivation behind the OpenGL version is to allow the port to Linux. Stay tuned for further updates!