Whilst I cannot help out with the proper way to disable the blocks from rendering inside this area, and substituting your own animation, I can quite confidently state that it will not be as performance intensive as you think, if you keep OpenGL calls to a minimum.
I've made some menger-sponge inspired compressed blocks (20^1, 20^2.... 20^6 block compression) and whilst I was dealing with pre-calculated models and bakedQuad-caching, T3 compression(20^3) or 8,000 miniature blocks rendered inside a single blockspace did cause some stutter, but not game-breaking. (Though it wasn't animated)
Though 11^3 isn't a small number, that still only equates to 1,331 blocks. Yes, scaling all of this will have a higher performance drain than just basic rendering, but it shouldn't be impossible if you focus on performance.