VolDat: A standard format for exchanging volume data

One of the challenges which we face when developing a voxel engine is finding appropriate datasets to test with. For researchers working with mesh data there are a number of standard test models (Sponza, Dragon, etc) which have been used in hundreds of papers over the years, but our options for volume data are a lot more limited. There are some archives containing CT/MRI data but this is not appropriate for all usage scenarios.

Another challenge is the lack of a standard format for volume data, as this impedes our ability to pass data between different voxel applications as well as making the creation of the above mentioned volume data archive more difficult. The only real standard is DICOM, but this is very complex and heavily geared to the medical imaging industry rather than more general scenarios.

Therefore we are taking the initiative to create ‘VolDat’ as a standard format which we hope will be adopted by the various applications which work with volume data, and which will form the basis of our own volume data archive. This is not meant to replace an application’s own format, but instead to be an additional import/export option to promote interoperability.

The format itself is very simple, and is independent of the kind of data your voxels represent (you can use it for coloured cubes, Minecraft-style terrain, density fields for smooth terrain, etc). We basically store each volume as a series of images in a standard format, along with a text file describing some of the volume’s properties. This makes it very easy to understand the data even without reading the specification. For example, the image below shows one of the maps from Voxeliens in VolDat format being viewed in Windows Explorer:

When rendered in 3D it looks as follows (you should all know this having played Voxeliens!):

We’d really appreciate any feedback on this initiative or on the specification itself. The specifiaction is currently hosted on Google Docs which means anyone can add comments, and we can give edit permission if people want to contribute:

You can also provide feedback in the blog comments below or in this forum thread. We’ve also prepared a couple of (small) sample volumes which you can take a look at.

We will try and contact potentially interested parties regarding this initiative, and please do pass this on to anyone who could benefit from it. Thanks!

Share

Introducing: Cubiquity

Regular followers of our blog and forums will know that over the last few months we have done a lot of work integrating PolyVox with the Gameplay3D engine. We’ve also mentioned Unity3D and talked about creating a higher level framework to integrate PolyVox with such external game engines. Today we are pleased to officially introduce this technology as ‘Cubiquity’, and provide a very early demonstration of where we are going with it.
Cubiquity is a C++ library which is currently dependant only on PolyVox and which provides higher-level features such as a scene graph, level-of-detail, mesh management, and tools for manipulating volume data. Additionally we have developed ‘Cubiquity for Gameplay3D’ which basically connects Cubiquity to a Gameplay3D scene node so that volumes can be treated as first-class citizens in Gameplay3D. We also have Lua bindings so that Gameplay3D scripts can create and manipulate volume data.

The video below shows some of the features which are present in the tech demo. Cubiquity supports both cubic (hence the name!) and smooth voxel terrain but only the cubic terrain is being shown at the moment. The terrain in the video has a resolution of 512x512x256 voxels which are displayed as coloured cubes. A LOD system allows these cubes to be grouped together into larger cubes as they move into the distance. The terrain is fully dynamic and can be edited in real time (but note that lighting is baked in to this data set).

As mentioned, we have bindings to Lua which have allowed the whole tech demo to be driven by a simple script. This handles input, drives the rendering loop, and also exposes the entire Gameplay3D API meaning that (in theory) you can take this tech demo and build a game on it. In practice you might want to wait for a more mature version 😉

You can download the tech demo at the link below. Please remember this is a very early version and, amoung other problems, whole system is currently running on a single thread. This causes pauses/hiccups when the terrain is being modified. The initial loading of the terrain is also slow and may take a minute of so.

Download: http://www.volumesoffun.com/downloads/Cubiquity/CubiquityTechDemo_ver_0_1.zip

We do not currently expect Cubiquity to be free, and will probably be targeting the professional indie market with it. Hopefully this will provide a way to support the development of PolyVox (which is also directly benefiting from the development of Cubiquity). That said, it is likely that we will continue to provide free tech demos which can be scripted to make your own games. We’re really still working out the details here.

I think there will probably be a lot of questions about all this, so ask away in the comments. Eventually we’ll get some proper information available on the website.

Share

Level-of-detail for cubic-style terrain with PolyVox

I just thought I’d post an update about some tests I did which worked out a lot better than I once expected. The idea here is to perform LOD for cubic-style meshes simply by downsampling the volume, such that in the distance a group of eight cubes is replaced by a single cube.

In the past I’ve said this wouldn’t work well as the transition would be too visually jarring, but recently I’ve seen a couple of screenshots from voxel engines which do indeed take this approach. I also talked about it with Matt over Christmas, and hacked together a test implementation last night.

The screenshots below start at full resolution and then gradually bring the LOD cut-off distance forward until all parts of the terrain are at the lower LOD. The transition is much less noticeable than I thought, and it practice it would also be much further from the camera.

This has all been implemented within our new framework built on PolyVox and Gameplay3D. Hopefully we’ll be able to provide a demo of this framework in the near future.

Share

PolyVox 0.2.1 released

As a late Christmas present to you all we’ve just released version 0.2.1 of PolyVox. This is a bug fix release to PolyVox 0.2.0.

This release includes the following bug fixes:

  • A compilation error on VS2008 (commit)
  • One of the sampler peek functions would check in the wrong direction before moving (commit)
  • SimpleVolume and LargeVolume had a problem with negative positions (commit, commit)
  • The LowPassFilter test was using the wrong kernel size (commit)

Apart from those bug fixes there are no new features and so should be completely safe to upgrade to.

You can get the details for downloading the source from the PolyVox download page or download directly from these links:

If you’re following us on Git, the 0.2.1 is now what is represented on the master branch.

Share

New home for PolyVox git repo

The source code for PolyVox is now located at BitBucket rather than its previous home on Gitorious. There are a few reasons we wanted to move hosting provider. Primarily we wanted an easier way of keeping track of planned features without messy email threads between David and myself and better bug tracking than our current forum-based method.

BitBucket offered everything we needed and since we were already using it for its unlimited private repos, it made sense to move PolyVox there too.

While we will be using the issue tracker to keep track of features and bugs, we will keep using the forum for most discussions. Feel free so submit any simple/small bugs to the tracker directly e.g. for compile errors etc.

To switch your local clone to follow the new repo on the command line just do the following:

# First check where you're currently pointing
$ git remote -v
origin  git://gitorious.org/polyvox/polyvox.git (fetch)

# Change the 'origin' remote's URL
$ git remote set-url origin https://bitbucket.org/volumesoffun/polyvox.git

# Now, it should look like
$ git remote -v
origin  https://bitbucket.org/volumesoffun/polyvox.git (fetch)

Otherwise, you can just use your GUI tool to change the repo URL to be https://bitbucket.org/volumesoffun/polyvox.git. You should now be able to carry on doing git pull as if nothing had changed.

Share

PolyVox version 0.2 released

We’re pleased to announce that a new version of PolyVox is released today. This release brings a number of improvements to both the library itself and also to our development process. Looking at the library first, the main changes include:

  • Enhanced control over the surface extractors, which allows them to execute on more voxel types (including primitive types)  and also provides more control over the way in which triangles are generated during the extraction process.
  • Much improved documentation, including an expanded manual and API docs. These are still work-in-progress and will be developed further in future releases.
  • Some restructuring of code, particularly the unclassing of raycasting and ambient occlusion functionality. Again this is something we expect to see more of in the future.
  • Lots of warnings and fixes to the unit tests. We use CDash for monitoring the quality of our software and it’s all green for the first time that I can remember.

Some refactoring has made the surface extractors much more flexible.

Additionally, there have been a number of improvements to the build system and development process which were largely spearheaded by Matt. These include:

  • A switch to Git flow for our development process, in order to keep the Git repository in a more stable state and to facilitate collaboration. As a result this is our first release with a proper version number.
  • Improvements to the CMake build system to better detect which optional dependencies are available and to configure the build appropriately.
  • A bash script to automate the process of performing a release.
  • The use of CMake folders to improve navigation in the Visual Studio solution.

Instructions and links for downloading the latest release can be found on the download page. You should also checkout the changelog for more details on the new features, and do come by the forums if you have any questions or problems with this latest release.

Enjoy!

Share

Level of detail for smooth voxel terrain

Over the last couple of weeks I’ve been continuing my integration efforts between PolyVox and gameplay3d. This is proving to be extremely useful as a way of identifying the problems and limitations within PolyVox, especially as I haven’t worked with smooth voxel terrain for quite a while (since before Voxeliens). In particular I have been focusing on level of detail support, and as you can see below I have had some success:

The MeshDecimator class has recently been deprecated, and I’ve been saying for a while that down-scaling the volume data and running the Marching Cubes algorithm on this smaller data is probably a better way to achieve LOD for smooth terrain. There is even an example in PolyVox which demonstrates this on a sphere. However, doing it on a real world dataset has proved to be a bit more complex, and I can see now that some changes need to be made to the way PolyVox handles sampling outside a volume and to the way the VolumeResampler works. But in principle it does seem like a sound approach.

That said, there’s still an issue with cracks between the mesh patches where the LODs don’t line up exactly (you can’t really see this in the screenshot because it’s wireframe). Some tests have shown that simply overlapping the meshes slightly seems to be a viable solution, but again I need to make some changes to PolyVox to support this more naturally. Of course, the Transvoxel Algorithm is also a more robust solution which may make an appearance in the future.

Next up I’m expecting to look at physics integration for both the smooth and cubic styles of terrain.

Share

Smooth voxel terrain editor in gameplay3d

Over the last couple of weeks I’ve spent some more time on our PolyVox+gameplay3d integration, and have added some basic editor functionality. Here’s how it looks at the moment:

You can paint on the terrain with the various materials, and there is also functionality to edit the shape of the terrain. This can even run on my mobile phone at a slow but useable 10-15fps. I realise the labels say ‘Brush Size’ twice, the second one should actually say ‘Brush Intensity’.

The main reason that I’m doing this is as research into the best way to handle multiple materials within a smooth voxel terrain. These have always presented a problem with  modelling and also with making nice material transitions. I think I’ve made some significant breakthroughs here so I’m hoping to write a research paper on the coming months… so if you want details on how it’s done you’ll have to wait for the that 😉

Share

Where’s the next PolyVox release?

PolyVox users are no doubt aware that it’s been a while (10 months actually) since the last release so I thought it would be worth taking a moment to provide an update to what’s going on. Obviously the main distraction has been Voxeliens (which filled the first 6-8 months and is still in the background), but we’ve also been working on gameplay integration and having a busy couple of months at work. Still, there has been a steady stream of commits to PolyVox so progress is being made.

In a previous post we identified the main areas we wanted to work on before doing another release. I’ve listed these below and colour coded them according to how complete the work is. Green is finished, red is not done yet, and yellow is somewhere in between:

  • Finish voxel refactoring
  • Surface extraction callbacks
  • Unclassing work
  • SWIG bindings
  • Fix A* test
  • Fix warnings
  • Remove redundant code
  • Documentation
  • Release script
  • Switch to Git flow
  • Improved CMake consistency

Note that’s we’ve decided not to pursue the SWIG bindings for the next release. They might still get some improvements but they won’t be in a finished state. Future versions of PolyVox are likely to bring more changes (particularly more unclassing) which might break any bindings we write now but which should also make them easier to implement later.

So when is the next release coming? Well so far we’ve been pretty bad at predicting dates so I’ll just say I hope it’s done by the the end of the year. The plan is still to do smaller and more frequent updates in the future, now that we have the Git flow model and release script.

Share

Smooth terrain in gameplay3d

Over the last couple of weeks I’ve been doing some work to integrate PolyVox’s smooth terrain into RIM’s gameplay3d engine. This follows on from our previous work on cubic terrain. The smooth terrain is no more difficult than cubic terrain, and the main challenge has been designing the code in such a way that the integration elegantly supports both (as internally they have different voxel and vertex types).

The images below are again based on one of the data sets from Voxeliens, but this time we’ve run the data through the LowPassFilter to perform smoothing and basic triplanar texturing is also applied:

These screenshots are taken on PC but the test application does run on my mobile phone as well. It’s a bit slow though (20-30fps) which is probably because there’s no LOD present at the moment. It’s nice to be back to tech development are spending so long working on Voxeliens and we have some big ideas for the next engine 🙂

Share