I don't think there are actually any downsides to allowing the backwards compatibility - it should all be internal to the Volume class. All it has to do is make sure the internal buffer is big enough for the requested volume size and then no paging should occur.

And actually I'm going to be one of those who benefits, because believe it or not my current game prototype uses a single volume of size 128x128x32. I might yet double each side length but I'm hardly in need of the streaming

I think backward compatibility is great. Not everyone/everything wants/needs streaming. Even in my engine I can imagine cases I would not want streaming.

Also, some people who do not want streaming will not want to be complicated with the new system.

I am still a bit puzzled by how it works from what was said here. Correct me if I am wrong.

Edit:

Ok, I just remembered there was an example and went back to see it


Here are my questions then:

1. When will createPerlinTerrainChunk actually be called? Is it when surfaceExtractor.execute(); is called? Is it any time before?

2. What will be the region size requested by the callback? 4096? or is 4096 the maximum size? is it 4096^3 or 4096 bytes or 4096 * sizeof(UINT32) or what? What does volData.setBlockCacheSize(4096); exactly mean?

3. What region size can I expect the createPerlinTerrainChunk will receive if I call any specific voxel? How do I calculate that to work on prediction?

4. How does saving of a volume works now? (I never really used it back then either btw) does it have compression when saving? Do I need to save it myself (which means for now no compression...?) How is saving handled with the streaming of voxels?

5. If there is saving integrated in PolyVox (as far as I understand there is?) then considering the new streaming system, how do I navigate through it to any specific voxel or "region" so I can integrate it with my own callback function?

6. Is streaming currently in the svn with its latest version? Can I start trying it? What problems should I expect with the current versions and what should I avoid? What can I try to help you with finding new problems?

Edit again:
Still some very important pieces in the puzzle:
How can we force to unload a region? Is there any callback which tells us when such event happens?

okidoki, backwards comaptibility it is... a new diff is coming up soon since I'm basically just reverting some functions back to their original state.
yea, my project got a little rewind due to large changes in the networking, so I wasn't able to test out the volume to the full extend that I want to use it.

anyway, to the questions:

1. yes during execute when blocks are accessed that are not loaded yet. but you can "preload" the blocks by accessing voxels that are inside of it (ok that is a little stupid, but we could add a preload function that gets a region...
2. the region that is requested will be 32^3 since that is the default block side length for the volume class
volData.setBlockCacheSize(4096) means that once 4096 blocks are loaded, blocks will start to unload
3. 32^3, it is always that size, but if you wish to preload your volume, just don't create a callback and fill it with setVoxel as you've done it before streaming.
4. saving works like loading. you add a callback to unload and you get a region. you use getVoxel to obtain the voxel values and then you store them however you want
no compression there obviously, but you can do that yourself then
5. I think that kinda merges with 4? or did I misunderstand you there?
6. I didn't check the svn, might be in there already
oh you will have problems... everything having to do with volume size has been broken just wait until I get the compatibility mode back in there

7. good point, I was thinking about that, too... a function to force an unload would be practical... even thou I'm not sure how to handle it when someone chooses a region that is not exactly on block borders... (imagine you have a 64x64x64 volume made up of 8 32x32x32 blocks... and you choose coordinates from 0x0x0 to 16x16x16... you have not selected a full block to be unloaded, should I unload the whole block from 0x0x0 to 32x32x32 or should I do nothing then? (ok this could be kindof fixed by creating a boolean argument roundUp/roundDown)
and while we're at it a function to force a load would be good, too

also the setBlockCacheSize function could be changed to get a vector as a parameter which defines the x,y,z dimension that should be loaded at all times. (like if you want a region around your player to be loaded at all times, and if the player moves, and you try to load new blocks, old regions start getting unloaded)

ok... or that way, it's probably better. I'll not change polyvox then until I get your cleaned up version.
soo many posts there seems to be real interest in this.

how do you intend to make borders values work if you have the volume with a fixed size?

Ok, so always 32x32x32? Can I change that? any function to allow me changing it?

Now, say I do getVoxel(5,5,5), then I can expect to get a call to load a region of (0,0,0) to (31,31,31)?

as I said, 32 is the default block side length (formerly parameter 4 of the volume constructor), just use the appropriate function to change it

jup, that's how it works, be sure to use <= and not < when iterating through the coordinates since the region defines which voxels you're supposed to use, not what the boundaries are. (the perlin noise function in the basic example shows that)

ty! sounds simple.

Now.........! You say there is a callback for unload, right? How do I know if the voxel was modified or not and if I need to even bother saving the data?

When does it set it was modified? with a call to SetVoxel? How can I know if it happened from in-game or from the callback load?

Where can I get the newest streaming version? sounds simple enough I want to start working on it in my engine.

svn + the diff file I posted here, but you might want to wait for david.

uh... good point... you don't know if it was modified.
since there is yet another level of caching involved now... I could add a modified bit/bool to every region that then gets passed to the unload function...
I currently don't need this at all, since I just store everything that gets unloaded... loading is faster than generating... ok it takes up disk space... if you pack it, it shouldn't take much thou...

Not quite sure how packing works really. Maybe you can share some code for this?

that's not part of streaming, if you want to pack data, look at zlib (messy code, but good results) or google RLE (easy, and usually very good results on regular maps) or just google after any type of array/data packing