Yes, but I've already made the change in the latest Git and snapshot (which has just been released).

I was reading some of the documentation and I think I may have misunderstood some of the previous posts. Is the uint32 limitation the maximum total volume, or just the maximum number of voxels that can be in memory at one time?

In other words is the large volume capable of supporting 'infinite' volume?

Secondly how do I add to an existing volume? My question might be worded badly, so let me explain. I am creating a procedural terrain generation engine for a senior project. I want to be able to generate and add terrain when the camera moves towards that point. This will hopefully give the impression of a fluid and infinite world.

I assume the volume would start with only the information about the starting area and add blocks as camera moves towards the borders. My question would then be how to add new blocks of terrain as needed. I am having a hard time finding information relevant to this question in the links provided. The documentation was nice though.

_________________
--Real Programmers use a magnetized needle and a steady hand.
--xkcd--

The maxBlocksInMemory etc only relates to how much data can be loaded at one time. LargeVolume is specifically designed to allow infinite paging volumes.

How you page data in/out is up to you. If you use the dataRequiredHandler and dataOverflowHandler, these will be called when a block is allocated/de-allocated, so you can use these to load or generate data, or save to disk etc. Loading the blocks in that case should be as easy as just accessing a voxel in an area at the edge of the visible zone as you want to load them in (eg during mesh extraction). Saving would happen automatically as blocks exceed the blocks in memory limit, or you can call flush.

Alternatively, you can handle paging yourself by just generating/loading/saving blocks as they come in/out of your load range.

Since serialization is broken you'll just need to use getVoxel setVoxel to retrieve/set the voxel data.

Thank you for the reply its good to finally understand what the limit applies to.
However in the second part of your response I wasn't referring to paging existing voxels, I meant new data. For example in minecraft, when a player approaches the bounds of the existing world, the program generates new terrain in the block the player is approaching. Previously it was empty, and on demand it terrain was created and added to the volume.

I believe I understand how to overload those functions to provide paging, but how do I add new data? I'm sure I'm not expected to provide all voxel information to the volume when initialized. I'm pretty sure there is a way to expand the volume as new information is added. Otherwise why make an 'infinite' volume to begin with. I just don't understand how I do this.

Perhaps you meant that the overload for paging could handle generating new blocks as well. If that is the case, how does polyvox know it is approaching the boundary of its loaded area?

I hope this clarifies my question. Thanks for the help.

_________________
--Real Programmers use a magnetized needle and a steady hand.
--xkcd--

I believe you'll have to handle the cases of loading from harddisk or generating new voxels yourself and this depends how you define your data and it's correspondence with PolyVox::Regions. For generating new data using a noise function, the simplest thing you can do is to map PolyVox::Region directly into the noise function. So for example if you have a region <320, 320>, <320 + 32, 320 + 32>, your noise function would generate data for this 32x32 block given the PolyVox::Region. It really depends on how you map PolyVox::Region to your own data representation.

You say what does PolyVox::Region have anything to do with this. The data handler / delegate you provide to BigVolume work with PolyVox::Regions.

What happens when the dataRequiredHandler overloaded function is called is entirely up to you. As a general case you would check to see if data for that block already exists saved to disk, if it does you load it, if it doesn't then you generate data and add it into the volume with setVoxel.

PolyVox doesn't know and doesn't care how and when data is generated. All it handles is data that is currently in memory at any given time.

As far as knowing when the 'edge' has been reached, just use your normal paging / distance based mesh extraction procedure, and dataRequiredHandler will automatically be called when needed.

Generating data 1 block at a time may not be the best approach of course. You could just as easily pre-generate data for a large region, save it to disk, then those blocks will be loaded normally with your dataRequiredHandler function.

This is just based on what blocks are currently in memory. For LargeVolume, blocks start off not loaded into memory at all. When you call getVoxel, setVoxel etc, this will allocate the block and call dataRequiredHandler.

Since the number of blocks in memory is limited, an old compressed block will be de-allocated when the limit is reached and a new block needs to be added. Before the block is de-allocated, dataOverflowHandler is called.

So, all you need to do is extract meshes for areas you want to render, and stop extracting meshes for areas you don't want to render, and PolyVox + dataRequiredHandler and dataOverflowHandler will handle allocations/de-allocations/paging.

Just make sure dataRequiredHandler can load or generate voxel data, and dataOverflowHandler can save it, and that should be all you need.

If any of that is incorrect someone please correct me, as I do paging externally so I haven't actually tested the overloaded functions.

everything has basically already been said.

if you do not want heavy loading to start happening during the extraction phase, you can choose not to create any dataRequiredHandler, and instead make sure any regions you extract have enough data.
and use dateOverflowHandler to only copy out the data that has been unloaded to some other place in memory where another thread will take care of saving it.
this way rendering will end up taking the least time possible.

also be sure to look at prefetch and flush (member functions of the volume class). these are there for forcing block unloads and loads. you could also (when taking care) create a system that uses them to load your data at another time than rendering.

I have to agree with ker that doing the generating/saving/loading is better done in a separate thread to extraction, otherwise your extraction thread will block up waiting for disk access.

In my case the paging thread allocates polyvox block, generates/loads data, then flags the page active for mesh extraction in my mesh manager, which is then handled by the mesh extraction thread.

It's also worth pointing out that ther is a 'PagingExampe' available in PolyVox. It doesn't actually let you fly around but it does demonstrate the basic usage of the LargeVolume.



It doesn't. PolyVox doesn't know where your camera is or what it can see. However, you know where you camera is, so you can decide what regions you need to extract a surface for. If you try to extract a region and there is no data for that region then PolyVox will try to load the data via the dataRequiredHandler() which you have provided.

In other words, you can create a volume without any data and start extracting. PolyVox will the request data from you as it needs it.

Alternatively, you can use the setVoxelAt() function to put data into part of the volume before you perform the extraction. If you do this, PolyVox will see the data is already present in memory and won't call the callback to request it. As you insert data you may cause some other data to be pushed out of memory, in which case PolyVox will call the dataOverflowHandler() to give you a chance to save that data.

You can think of it as a pyramid. At the top we have a small amount data which is stored uncompressed and can be accessed very quickly. In the middle we have a larger amount of data, stored in a compressed form. And at the bottom we have a huge amount of data in some user defined form (files, database, network, etc). When PolyVox uses data (for surface extraction, etc) it gets pulled to the top of the pyramid. If it is untouched for periods of time it gets pushed back down, first to a compressed form and then to the user defined form.

Data is moved between the top level and the middle level via compress() and uncompress() functions, and it is moved between the middle level and bottom level via the dataRequiredHandler() and dataOverflowHandler().