I think overall you will be best off using voxels rather than trying to implement direct modification of the mesh. But if you do want to implement your own mesh-based CSG approach then you shouldn't do it directly on the vertex/index buffers. Instead you will want more information about the connectivity in the mesh so that (for example) you can rebuild the surrounding faces after you delete a vertex.
You should probably look into mesh structures like these:
http://en.wikipedia.org/wiki/Polygon_mesh (Scroll down to 'representaions').