Python bindings for PolyVox preview

Posted on April 26, 2013 by Matt Williams

As part of the next release of PolyVox, we’re going to have the first Python bindings available and supported as well as preliminary support for C♯ bindings. This will allow easy integration of voxels into the technologies you’re already using.

History

For a long time now (at least 4 years) we’ve had some form of Python bindings available for PolyVox, created via the SWIG tool. If you were paying attention in those days, you would have seen that the bindings were often broken or even disabled from building completely. This was largely due to the fact that PolyVox was a fast-developing piece of software without a stable API. We didn’t want to slow down the development of new features by having the impedance of also having to update the bindings.

However, since we starting making proper numbered releases (beginning with 0.2.0 last year) the API has settled down and we’re making promises about how much (or little) it will change. This makes it a much more suitable target for building bindings to other languages.

Current status of Python bindings

My main target for the next release of PolyVox (0.3 when it’s out) has been improving and polishing the bindings for Python. In addition to the Python bindings, I have also enabled the building of C♯ bindings for those who would like to try those out too. The Python bindings are better supported however (we have nightly tests run on them, documentation and a demonstration example) and are less liable to change.

If you’re interested in taking a look, then I’ve uploaded a first draft of a section documenting the Python bindings which introduces the bindings and goes through a short code example. That short example is expanded as a full example in the usual example location, PythonExample.py, which renders a blocky sphere to the screen using Python, PyOpenGL and PyGame without a single line of C++ needing to be written.

A voxel sphere rendered though PyOpenGL

For example, creating a volume is as simple as

import PolyVoxCore as pv

r = pv.Region(pv.Vector3Dint32_t(0,0,0), pv.Vector3Dint32_t(63,63,63))
vol = pv.SimpleVolumeuint8(r)

You can set the value of any voxel with a single call like:

vol.setVoxelAt(x, y, z, 10)

and you can extract a polygon mesh to be passed to your 3D engine of choice with

mesh = pv.SurfaceMeshPositionMaterialNormal()
extractor = pv.CubicSurfaceExtractorWithNormalsSimpleVolumeuint8(vol, r, mesh)
extractor.execute()

The unusually long names for the classes is an artefact of SWIG being able to represent all the varieties of C++ template instances. You’ll also notice that the API is not very Pythonic at present. I’m trying to get all the functionality working before adding another layer of Pythonicness.

All of the above is already in the develop branch of Git already so feel free to try it out. If you have any question please do ask in the comments below or in the forums.

In a future post I’ll detail some of the technical challenges I’ve faced to get all of PolyVox’s functionality working in both Python and C♯ via SWIG.

A quick update and some new screenshots

Posted on April 2, 2013 by David Williams

I thought I’d write a quick update on where we stand with our various projects as I feel like we’re juggling a dozen things at a time here I’ve thrown in some pictures to spice it up a bit, but if you want to see these when they are fresh then be sure to follow us on Twitter.

Cubiquity

Since the first tech demo we have implemented threading in Cubiquity so that surface extraction can be performed in the background. This improves loading times and responsiveness but currently it only works with PolyVox’s SimpleVolume, which limits the amount of volume data we can load at a time. I’ve also done some work overhauling the shaders so that normal mapping is now supported, and hopefully this will get improved further in the future.

Normal mapping the voxels in Cubiquity

PolyVox

The most significant addition to PolyVox is that the LargeVolume class now supports plugable compression code. We’ve also provided an implementation based on the miniz library. This works well for smooth terrain while the existing RLE compressor can be used for cubic ‘Minecraft style’ terrain.

Next up we need to improve the thread safety of the LargeVolume as this has been requested many times, and we’re starting to need it for Cubiquity (as mentioned above).

VolDat

We’ve received some feedback about the format and will do a proper post about this soon. The main issues raised are paging for very large volumes (probably beyond the scope of what we are aiming for here) and also an existing similar format (but it doesn’t make the slice data easy to visualise). We’ve also bee working on some test data sets which we will make available as part of an online archive. A couple of examples are below:

A ‘Build and Shoot’ map loaded into Cubiquity. Source data is here: http://www.buildandshoot.com/viewtopic.php?f=8&t=913

I wrote a program to create a Mandelbulb as these can be very large and still have fine detail. This one is only 512x512x512 though.

Unity3D

I downloaded Unity and had a play with it last weekend. It’s a really cool system, and I can see why it’s got such a big following as it makes development really easy. Integrating Cubiquity is a bit more difficult than I hoped as it turns out that plugin .dlls need to provide a ‘C’ interface (not C++) so we’re having to create a wrapper for Cubiquity. This takes a bit of time but shouldn’t be a big problem over all.

That will do for now – see you next time

Introducing: Cubiquity

Posted on February 9, 2013 by David Williams

Regular followers of our blog and forums will know that over the last few months we have done a lot of work integrating PolyVox with the Gameplay3D engine. We’ve also mentioned Unity3D and talked about creating a higher level framework to integrate PolyVox with such external game engines. Today we are pleased to officially introduce this technology as ‘Cubiquity’, and provide a very early demonstration of where we are going with it.
Cubiquity is a C++ library which is currently dependant only on PolyVox and which provides higher-level features such as a scene graph, level-of-detail, mesh management, and tools for manipulating volume data. Additionally we have developed ‘Cubiquity for Gameplay3D’ which basically connects Cubiquity to a Gameplay3D scene node so that volumes can be treated as first-class citizens in Gameplay3D. We also have Lua bindings so that Gameplay3D scripts can create and manipulate volume data.

The video below shows some of the features which are present in the tech demo. Cubiquity supports both cubic (hence the name!) and smooth voxel terrain but only the cubic terrain is being shown at the moment. The terrain in the video has a resolution of 512x512x256 voxels which are displayed as coloured cubes. A LOD system allows these cubes to be grouped together into larger cubes as they move into the distance. The terrain is fully dynamic and can be edited in real time (but note that lighting is baked in to this data set).

As mentioned, we have bindings to Lua which have allowed the whole tech demo to be driven by a simple script. This handles input, drives the rendering loop, and also exposes the entire Gameplay3D API meaning that (in theory) you can take this tech demo and build a game on it. In practice you might want to wait for a more mature version

You can download the tech demo at the link below. Please remember this is a very early version and, amoung other problems, whole system is currently running on a single thread. This causes pauses/hiccups when the terrain is being modified. The initial loading of the terrain is also slow and may take a minute of so.

Download: http://www.volumesoffun.com/downloads/Cubiquity/CubiquityTechDemo_ver_0_1.zip

We do not currently expect Cubiquity to be free, and will probably be targeting the professional indie market with it. Hopefully this will provide a way to support the development of PolyVox (which is also directly benefiting from the development of Cubiquity). That said, it is likely that we will continue to provide free tech demos which can be scripted to make your own games. We’re really still working out the details here.

I think there will probably be a lot of questions about all this, so ask away in the comments. Eventually we’ll get some proper information available on the website.

Level-of-detail for cubic-style terrain with PolyVox

Posted on January 5, 2013 by David Williams

I just thought I’d post an update about some tests I did which worked out a lot better than I once expected. The idea here is to perform LOD for cubic-style meshes simply by downsampling the volume, such that in the distance a group of eight cubes is replaced by a single cube.

In the past I’ve said this wouldn’t work well as the transition would be too visually jarring, but recently I’ve seen a couple of screenshots from voxel engines which do indeed take this approach. I also talked about it with Matt over Christmas, and hacked together a test implementation last night.

The screenshots below start at full resolution and then gradually bring the LOD cut-off distance forward until all parts of the terrain are at the lower LOD. The transition is much less noticeable than I thought, and it practice it would also be much further from the camera.

This has all been implemented within our new framework built on PolyVox and Gameplay3D. Hopefully we’ll be able to provide a demo of this framework in the near future.

PolyVox 0.2.1 released

Posted on December 28, 2012 by Matt Williams

As a late Christmas present to you all we’ve just released version 0.2.1 of PolyVox. This is a bug fix release to PolyVox 0.2.0.

This release includes the following bug fixes:

A compilation error on VS2008 (commit)
One of the sampler peek functions would check in the wrong direction before moving (commit)
SimpleVolume and LargeVolume had a problem with negative positions (commit, commit)
The LowPassFilter test was using the wrong kernel size (commit)

Apart from those bug fixes there are no new features and so should be completely safe to upgrade to.

You can get the details for downloading the source from the PolyVox download page or download directly from these links:

If you’re following us on Git, the 0.2.1 is now what is represented on the master branch.

New home for PolyVox git repo

Posted on December 3, 2012 by Matt Williams

The source code for PolyVox is now located at BitBucket rather than its previous home on Gitorious. There are a few reasons we wanted to move hosting provider. Primarily we wanted an easier way of keeping track of planned features without messy email threads between David and myself and better bug tracking than our current forum-based method.

BitBucket offered everything we needed and since we were already using it for its unlimited private repos, it made sense to move PolyVox there too.

While we will be using the issue tracker to keep track of features and bugs, we will keep using the forum for most discussions. Feel free so submit any simple/small bugs to the tracker directly e.g. for compile errors etc.

To switch your local clone to follow the new repo on the command line just do the following:

# First check where you're currently pointing
$ git remote -v
origin  git://gitorious.org/polyvox/polyvox.git (fetch)

# Change the 'origin' remote's URL
$ git remote set-url origin https://bitbucket.org/volumesoffun/polyvox.git

# Now, it should look like
$ git remote -v
origin  https://bitbucket.org/volumesoffun/polyvox.git (fetch)

Otherwise, you can just use your GUI tool to change the repo URL to be https://bitbucket.org/volumesoffun/polyvox.git. You should now be able to carry on doing git pull as if nothing had changed.

PolyVox version 0.2 released

Posted on November 19, 2012 by David Williams

We’re pleased to announce that a new version of PolyVox is released today. This release brings a number of improvements to both the library itself and also to our development process. Looking at the library first, the main changes include:

Enhanced control over the surface extractors, which allows them to execute on more voxel types (including primitive types) and also provides more control over the way in which triangles are generated during the extraction process.
Much improved documentation, including an expanded manual and API docs. These are still work-in-progress and will be developed further in future releases.
Some restructuring of code, particularly the unclassing of raycasting and ambient occlusion functionality. Again this is something we expect to see more of in the future.
Lots of warnings and fixes to the unit tests. We use CDash for monitoring the quality of our software and it’s all green for the first time that I can remember.

Some refactoring has made the surface extractors much more flexible.

Additionally, there have been a number of improvements to the build system and development process which were largely spearheaded by Matt. These include:

A switch to Git flow for our development process, in order to keep the Git repository in a more stable state and to facilitate collaboration. As a result this is our first release with a proper version number.
Improvements to the CMake build system to better detect which optional dependencies are available and to configure the build appropriately.
A bash script to automate the process of performing a release.
The use of CMake folders to improve navigation in the Visual Studio solution.

Instructions and links for downloading the latest release can be found on the download page. You should also checkout the changelog for more details on the new features, and do come by the forums if you have any questions or problems with this latest release.

Enjoy!

Level of detail for smooth voxel terrain

Posted on November 12, 2012 by David Williams

Over the last couple of weeks I’ve been continuing my integration efforts between PolyVox and gameplay3d. This is proving to be extremely useful as a way of identifying the problems and limitations within PolyVox, especially as I haven’t worked with smooth voxel terrain for quite a while (since before Voxeliens). In particular I have been focusing on level of detail support, and as you can see below I have had some success:

The MeshDecimator class has recently been deprecated, and I’ve been saying for a while that down-scaling the volume data and running the Marching Cubes algorithm on this smaller data is probably a better way to achieve LOD for smooth terrain. There is even an example in PolyVox which demonstrates this on a sphere. However, doing it on a real world dataset has proved to be a bit more complex, and I can see now that some changes need to be made to the way PolyVox handles sampling outside a volume and to the way the VolumeResampler works. But in principle it does seem like a sound approach.

That said, there’s still an issue with cracks between the mesh patches where the LODs don’t line up exactly (you can’t really see this in the screenshot because it’s wireframe). Some tests have shown that simply overlapping the meshes slightly seems to be a viable solution, but again I need to make some changes to PolyVox to support this more naturally. Of course, the Transvoxel Algorithm is also a more robust solution which may make an appearance in the future.

Next up I’m expecting to look at physics integration for both the smooth and cubic styles of terrain.

Smooth voxel terrain editor in gameplay3d

Posted on October 27, 2012 by David Williams

Over the last couple of weeks I’ve spent some more time on our PolyVox+gameplay3d integration, and have added some basic editor functionality. Here’s how it looks at the moment:

You can paint on the terrain with the various materials, and there is also functionality to edit the shape of the terrain. This can even run on my mobile phone at a slow but useable 10-15fps. I realise the labels say ‘Brush Size’ twice, the second one should actually say ‘Brush Intensity’.

The main reason that I’m doing this is as research into the best way to handle multiple materials within a smooth voxel terrain. These have always presented a problem with modelling and also with making nice material transitions. I think I’ve made some significant breakthroughs here so I’m hoping to write a research paper on the coming months… so if you want details on how it’s done you’ll have to wait for the that

Where’s the next PolyVox release?

Posted on October 19, 2012 by David Williams

PolyVox users are no doubt aware that it’s been a while (10 months actually) since the last release so I thought it would be worth taking a moment to provide an update to what’s going on. Obviously the main distraction has been Voxeliens (which filled the first 6-8 months and is still in the background), but we’ve also been working on gameplay integration and having a busy couple of months at work. Still, there has been a steady stream of commits to PolyVox so progress is being made.

In a previous post we identified the main areas we wanted to work on before doing another release. I’ve listed these below and colour coded them according to how complete the work is. Green is finished, red is not done yet, and yellow is somewhere in between:

Finish voxel refactoring
Surface extraction callbacks
Unclassing work
~~SWIG bindings~~
Fix A* test
Fix warnings
Remove redundant code
Documentation
Release script
Switch to Git flow
Improved CMake consistency

Note that’s we’ve decided not to pursue the SWIG bindings for the next release. They might still get some improvements but they won’t be in a finished state. Future versions of PolyVox are likely to bring more changes (particularly more unclassing) which might break any bindings we write now but which should also make them easier to implement later.

So when is the next release coming? Well so far we’ve been pretty bad at predicting dates so I’ll just say I hope it’s done by the the end of the year. The plan is still to do smaller and more frequent updates in the future, now that we have the Git flow model and release script.

Volumes Of Fun

Voxel-based games and technology

Tag Archives: PolyVox