Reflections on Cubiquity and finding the path forward

When we began working on Cubiquity the main idea was to expose our PolyVox library through a higher-level interface and tools. PolyVox is powerful and flexible, but requires a strong knowledge of C++, templates and computer graphics to use effectively. By wrapping this in Cubiquity and integrating it with Unity/Unreal we were hoping to bring (some of) it’s capabilities to a larger audience.

The project has been a reasonable success, with a lot of interest from the Unity and Unreal communities and even a game launched on Steam using the technology. However, it is also clear that the original vision is simply too large to finish as a personal side project, especially with diminishing amounts of free time and a desire to adjust my work/life balance. This led to use recently releasing the system for free, but further steps need to be taken to actually reduce the scope.

In this blog post I want to outline the areas in which I/we will be cutting back on development, and explain how it will affect existing users. I also want to define a direction for a future ‘Cubiquity 2.0’ such that it is both smaller and also more technically interesting to work on. This means shifting the focus to research rather than delivering a drop-in system, open-sourcing the complete technology stack, and reducing (but not eliminating) the level of integration provided for Unity and Unreal.

Open sourcing the core Cubiquity library

The proposed change of direction is likely to cause some concerns for users who are already working with Cubiquity, and so I hope to partially alleviate these by announcing that we will release the core Cubiquity library as open source. This is also expected to reduce our workload in the following ways:

  • End users will become responsible for building and testing the library on the platforms which they wish to support. On the plus side, they may also be able to port the library to new platforms (Android, iOS) which we did not have time for.
  • We may remove Cubiquity from the Unity Asset Store and handle distribution only through BitBucket and/or Github. A more developer-focused Cubiquity may not be a good fit for the Asset Store where users expect drop-in solutions.

This applies to both the current version of Cubiquity (which may not see much further development) and also to a future ‘Cubiquity 2.0’ code base (which will likely be a complete rewrite).

Shifting the focus back to research

When we first released PolyVox back in 2006 the idea of using voxels to represent worlds in games was fairly novel. There was some interesting work by the likes of Ken Silverman and Sven Forstman but voxel techniques were not widely used [1]. The arrival of Minecraft changed all of that, and overnight voxels became the new ‘hot thing’.

Fast-forward ten years and it now feels like everyone is developing a voxel engine, and the techniques used within PolyVox and Cubiquity are no longer particularly novel. But there is a lot of interesting research to be done as shown by projects like Atomontage and VoxelQuest, and I’ve actually been working on an interesting prototype of a new (to my knowledge) voxel-rendering algorithm.

I’ll talk more about my ideas in the a future post, but the important point for now is that the difference between implementing new algorithms vs. really making them production ready is at least an order of magnitude. I therefore expect future versions of Cubiquity to be a little more experimental, a little more developer-focused (rather than artist-friendly), and less like the drop-in solution we currently provide on the Unity asset store.

Reducing the scope

I also have some concrete plans to cut a couple of significant features from Cubiquity. However, note that these changes will not directly affect the current codebase. Instead, Cubiquity 2.0 will most likely be a complete rewrite and will be designed without them in place.

No more Marching Cubes terrain

Cubiquity currently provides both colored-cubes style voxels and smooth (Marching Cubes) terrain. This was made possible because PolyVox supports user defined voxel types and custom surface extractors, but the two voxel systems still require different shaders and editing tools. Developing both systems in parallel is proving to be a rather large task.

We will therefore drop the Marching Cubes terrain and focus entirely on simple colored voxels, with the intention of making them as small as possible. Atomontage has shown that sufficiently small voxels can produce attractive terrains while also modeling buildings and objects, so I hope that a dedicated terrain renderer will not be needed. It is the generality of such a system which makes it interesting, as well as the ability to convert any existing dataset to this format.

Moving editing tools from Unity/Unreal to a separate application

Cubiquity for Unity allows the users to sculpt and paint terrains inside the Unity editor, and there is also basic support for editing the colored cubes volumes. Cubiquity for Unreal offers some similar features though they are not currently as complete. Overall this requires a lot of duplication of code and effort.

The plan for future versions of Cubiquity is to instead have a separate application which can be used for creating, importing and (possibly) editing volumes, and to turn the Unity and Unreal integration layers into much something much thinner. If users wanted to create in-game editors this could still be achieved via the Cubiquity API, as is the case at the moment.

Reduced focus on the Unity integration

We began working on Cubiquity around 2012, and at the time Unity was taking the indie game development world by storm. It was providing a cheap yet powerful platform and was being used by a huge number of indie games. As we were aiming to sell Cubiquity it made sense to provide an integration layer with Unity. However, from a technical standpoint it was quite a poor fit for a few (related) reasons:

C# .vs C++: Cubiquity is written in C++ but Unity scripts are written in C# (or sometimes a variant of JavaScript). This means we had to write a C interface around Cubiquity and then call that via C#’s P/Invoke magic from Unity.

No plugin support on Unity Free: Plugin support was a Pro-only feature on Unity 4, so we had to resort to some tricks and hacks to get Cubiquity loading properly. This feature is now free in Unity 5 but we haven’t had time to update our scripts to take advantage of it.

Need to compile libraries: Because Cubiquity was provided as a native-code library we have to compile it for every platform we wish to support. So far we have only supported Windows, Linux and OS X, but with both 32 and 64 bit versions that’s already six libraries to build and test.

Platform support: For the reasons outlined above, it has not been practical for us to support all platforms. Mobile has so far been too much work, and webplayer is simply not possible due to the use of native code. This causes some frustration for users who build their game and then find they cannot export to the platform of their choice.

Some of these problems disappear with Unity 5 and our move towards open source, but we are stuck with the C# vs. C++ issue and the need to compile separately for each platform (though we now defer that problem to the user). By contrast, integration with Unreal should be much more straight-forward, and has mostly been held up by lack of time/resources (see below). In this case we would simply be able to drop our Cubiquity C++ files into an Unreal project.

I’m honestly not sure where I’m going with this. Unity is still a massively popular game development platform, and from what I’ve seen it is probably easier to get into than Unreal. I do expect we will still provide some degree of support for it, but with higher technical requirements on the user and less in the way of Unity-specific examples, documentation, builds, etc. But really we’ll just have to see how the next version of Cubiquity develops, and how well the algorithms match with the architecture of Unity.

Increased focus on the Unreal integration (long term)

Our recent work on ‘The Peasants are Revolting‘ was my first exposure to the new Unreal engine, and I was very impressed. Actually my interest in the engine goes back to the original Unreal which really piqued my interest in game development. However, it was only the arrival of UE4 that the engine became accessible to indies, and by this time I had a good couple of years invested into the Unity integration.

The Cubiquity for Unreal integration was therefore handled exclusively by Matt, who has done an amazing job considering the constraints he was under. But (like me) he has a number of real-life commitments, and the task was made more difficult by Cubiquity being closed source. Furthermore, the motivation for integrating Cubiquity with Unreal is not as clear as for the Unity version, because users have the option of just using PolyVox directly (which offers more flexibility and full source access).

Things look brighter in the future though, as Cubiquity will become open source and so the relevant files can simply be dropped into the project. Also, the core technology behind Cubiquity will (probably) be quite different from PolyVox and so they won’t be competing over the same space. I hope to spend some time in the future either assisting Matt or taking over some aspects myself (depending what he wants), but this won’t happen until the core technology is developed.

Conclusion

This has been a difficult post to write because it so drastically changes the direction of a project which has had a lot of time sunk into it. However, it does seem like the right decision to ensure we stay motivated and work towards something achievable, but exactly what that is remains to be seen.

[1] 90’s-style ‘voxel’ terrain as used in Commanche, Delta Force, etc doesn’t count here as it was not really volumetric (no caves, overhangs, etc). It was just raycasting a heightfield, and as far as I can tell the term ‘voxel’ was just used as a marketing gimmick. If anyone has evidence to the contrary then do let me know!

Share
This entry was posted in Uncategorized by David Williams. Bookmark the permalink.

About David Williams

David is a post-doctorate researcher at the University of Groningen, The Netherlands, where he is investigating GPU computing. Prior to this he worked in the games industry and wrote graphics/engine code for a number of PC/PS3/XBox titles. As well as making games he occasionally enjoys playing them, and also sometimes gets outside to do some photography.

15 thoughts on “Reflections on Cubiquity and finding the path forward

  1. I’m happy to hear this! I just discovered Cubiquity because I’m tired of rewriting everything from scratch (https://github.com/bfops/playform). I’d love to be able to put some love into v2.

    > No more Marching Cubes terrain

    I’m definitely curious to see how this works out. Atomontage has always been eyebrow-raising to me. It vaguely claims that it’s only able to reach its level of performance by having some AI.

    I’ve been following http://procworld.blogspot.com/ for a little while, and I’ve been impressed with how nicely its dual contouring approach works. It’s pretty straightforward to import existing meshes, and since each voxel donates the same number of polys to the final triangle mesh (regardless of size), even the naive isosurface extraction does a decent job of not producing a mesh with too many polys. It does have a mesh simplification step, but the constraints are looser when the mesh isn’t that big to begin with. I’m not familiar with the approaches to mesh generation and simplification when there are this many tiny cubes involves, and I’m interested to see how it performs, and how the amount of work compares.

    I’ve also been incredibly impressed with procworld’s ability to apply textures to its voxels. In general, it just looks gorgeous, and I’m curious to see how much work it is (both for devs and users) to make things look a certain way in the everything-is-tiny-cubes approach.

    • Yes, ProcWorld is very cool. But I’m really keen to try and keep the next version of Cubiquity as simple as possible, and having a single rendering primitive should be a nice step in that direction. I’m hoping it will also make tools simpler because adding/removing cubes/points should be much easier than modifying ProcWorld’s Hermite data (and stuff like Magica Voxel already exists).

      But really I’m just keen to see how far we can push it – I don’t know if we’ll come close to Atomontage but we’ll give it a shot!

  2. I just picked up Cubiquity and started playing with it. Must say I have been having fun, well done 🙂

    I am wondering, what’s your plan with the open source stuff? I have seen some different licenses in different repos ranging from no-commercial to unity asset store free license. If you want this to take off and start a productive life as open source then maybe an MIT-license or similar might make sense? I have great respect for your work though and want to stress that you are, of course, free to do what you wish.

    My main hope for open source would actually be that someone with more knowledge of c++ than I would compile for more platforms.

    Even if noone compiles for more platforms or the liecnse becomes more open I think this project still has a great value. I can do some prototyping using this and if/when I need to release something “for real” I can make an implementation for another voxel engine if need be. So thanks a lot for releasing for free!

    • Hi Mike, the licensing situation is indeed a bit confusing for legacy reasons. Basically:

      PolyVox: Currently zlib (Master branch) and will become MIT (already done on develop branch).
      Cubiquity (C++, builds on top of PolyVox): Closed source but will become MIT.
      Cubiquity for Unity3D (C#, build on top of Cubiquity): Previously non-commercial, now asset-store free license, will be MIT in the future.

      I’m actively working towards getting the whole thing MIT but takes some time. I can tell you that one user has access to the Cubiquity C++ and has apparently got it working on the iPad, but I won’t be supporting this myself.

      Glad you like Cubiquity anyway!

      • Well that sounds awesome! This is quite an awesome asset for the whole community and I’m really interested in seeing where the community will take it in the future!

        Sounds interesting that someone got it working on iPad, shouldn’t be too hard geting it to work on other platforms then I guess. But my knowledge of c++ is very limited so I could be wrong 🙂

        I am working on a package for the asset store, a 2.5D shooter and while waiting for approval I have been working on integration of cubiquity. An early result is displayed here: https://youtu.be/R6u0DqGg8rg

        If the whole thing goes MIT that would be great news as I would not have to create a specific integration package to tie the two assets together but could include cubiquity in my package. 🙂

  3. Pingback: Cubiquity is now fully open source under the MIT license - Volumes Of Fun

  4. Pingback: Wrapping up PolyVox development to focus on Cubiquity 2 - Volumes Of Fun

  5. Pingback: Voxeliens is now open source - Volumes Of Fun

  6. Unity/Cubiquity:
    I am having fun playing with Cubiquity, thank you for providing it to the community.

    I am using PickFirstSolidVoxel() to get a voxel that intercepts a “beam” (line renderer) extending out from my “player”. The voxel is then “destroyed” [using SetVoxel() with QunatizedColor(0,0,0,0)].

    Question: I am trying to position a particle system on the surface of the voxel where my “beam” intercepts the voxel, but I don’t know how to do so. Is there anything in Cubiquity that can help with this? I’m hoping so.

    Thanks.
    — Paul

  7. Pingback: Removing Cubiquity from the Unity Asset Store - Volumes Of Fun

  8. Just don’t drop Marching Cubes! This was the feature that made me come here to PolyVox Community after all 🙂 Also, I never tried something that is running Marching Cubes well and we all know what Euclideon ended being (screen space renderer) at the end.

    If I am not that lazy, I may release TileVox someday – it’s a C# system for Unity written in C# entirely which support 3D Tilepams and Marching Cubes with heightblending and up to 4 materials per Marching Cubes Volume (for now)

    • I currently have no intention of implementing Marching Cubes in the new engine, as the alternative algorithms seem promising so far. I’m going to carry on researching in that direction. When I get around to it I will do some blog posts on the key technical differences between PolyVox and Cubiquity 2.

  9. Pingback: Animated voxel water with MagicaVoxel and VoxBox – David Williams' Homepage

Leave a Reply

Your email address will not be published.