I am also attempting paging across multiple volumes with multiple threads.

However I don't really want to lock the entire system every time I need to get/set a voxel, since different threads may be accessing voxels a large distance apart.

So I am just using smaller volumes 1024^3 which I have wrapped in a class containing a mutex. That way I only have to lock the volume containing the voxel a particular thread is accessing at the time.

This is where built in thread support in Polyvox would concern me - I don't want to switch to a super large volume only to have it lock the entire thing each time it is accessed.

It sounds like you are already accessing the volumes in a thread safe way (by having a mutex per volume) and so you should have no problem with switching to the new system but keeping the multiple volumes. You should then get a significant amount of compression on each 1024^3 volume so hopefully you'll see some memory savings as well.



Indeed, but if I do make the Volume class thread safe it will not be done by locking the whole volume each time a voxel is accessed. Instead PolyVox would just lock the affected block, and even then it would probably only need to lock it if it needs to be compressed/uncompressed.

So I think you can safely switch to the new Volume class, but you will need to keep several of them as you are doing at the moment.

I may try something like that eventually. My plan is to instead of using my task-based thread system to generate chunks, chunks that may map to the same volume, I will instead just divide my world into several large volumes, then make it so only one thread can work on it at a time. The trick is to still use the task-based thread system, just now when a task is done, it will mark that volume as free. This way basically still generate manageble chunks at a time, still use the task based threads, still use have interactivity, have larger chunks with compression, and without using locks. I'm just not sure whether this will work the way I hope it will work, especially WRT to performance.

Anyway, I did some quick timing of my code with various settings of shared block size and cache size and also polyvox without compression. I'm not sure whether the tests are valid or not, it's just quick test. Also, I don't think I will worry too much about this for now. I don't have any game play. I still don't have a clear picture of how my final game will work. Without knowing that it's hard to say how (and how big) I should make my world. I need to finish doing that first.

Here are some results (timing are done with Win32 QueryPerformanceCounter; time are in seconds.):



Without compression (polyvox trunk)

One chunk One volume (32 by 32 by 256 sized volumes)



Multipe pages per volume. Each volume is 1024x1024 in this case.

Yep, I think that's sensible. I'm mostly concerned with making sure I won't completly break somebodys project by switching to the new Volume class, and it seems that in general we're going to be ok. From your figures it looks like the new class is a little slower, but that's hardly suprising with the compression that is being done. I'm sure there's some scope for optimisation here anyway.

That's good to hear. Locking individual blocks should work out better than locking the entire volume in a wrapper class.

Wouldn't you still need to lock for get/set on a cached block to prevent access violation? Maybe a R/W mutex....

I'm not sure exactly, but I optimistic a solution can be found. Maybe there needs to be a (smaller) block cache per thread, or maybe the volume allows several threads to be reading at a time but only one to be writing. I'm not going to jump to any conclusions about this until I give it some proper thought in the future.

But I'll only implement something in PolyVox if it is better than just locking the whole volume, because otherwise it's best left in the hands of the user.

I think a good idea would be to allow streaming of regions.

Allow to create a whatever size volume but only have it in RAM per request. That's what I did in my height map used for the GPU Clip Maps.

In mine it worked by setting a chunk size (you can use it as constant region size), so for example the world could have been the size of 50,000 * 50,000 and each chunk was 1,000*1,000. When walking around the world I'd always have the 9 squares at the center of my position handled by the renderer and the parts surrounding those ready in RAM so if I move away from a chunk's bounding region they are ready to replace the ones no longer needed. Then I start loading the new required chunks and unload the ones that are no longer in my second layer surrounding after some time.

Of course if you do so, don't bother with the threats etc, just make a function to load region (or region index if constant size) from stream and to unload and let the programmer handle it in his own threads so it is easier for you to create. This is pretty fast and flexible system to allow for supposedly "infinite" terrains. It should also handle the problem of holes and mis-matching geometry quite nicely and allow for very large worlds (I think morrowind or oblivion were 20 kilometers in size, you can easily get to be larger with this kind of system)

In general I agree, it would be nice to support streaming. If implemented it will probably provide some kind of callback funtion to handle missing regions, so that the user can choose to load it from disk, generate it procedually, etc. Such infinite streaming volumes are pretty low priority for me though.

Maybe I will look into adding that functionality to the source code myself when I have some time and it becomes more urgent for me, if it will turn good enough you could add it to the svn if you like. But might take me some time to get free time for this.

Sure, that'd be great. On a related note I'm going to hold off merging this into the trunk until next weekend, because I have a busy week and may need time to deal with any problems which occur.