Vertexbuffer.com

Some people have figured out that disabling the new operator, via overriding it, prevents you from using dynamic memory. This can help you prevent cache misses, but most people’s understanding of why this reduces cache misses is flawed. Is dynamic memory really the issue? No. Using dynamic memory poorly is the issue. Removing “new” just ensure that you don’t use it incorrectly.

So I’d bet no one ever told you that you can use “new” incorrectly. Using “new” to create small pieces of memory and not deleting them may cause you to allocate data structures in a non-sequential manor. Allocating memory sequentially, is essential to proper cache performance.

Spreading memory across system memory is known as fragmentation. When the memory controller pulls data from system memory, it does it in pages. Pages are blocks of memory. Your goal when moving though your data structures is to gather as much data related to your current work as possible. If your data structure fragments memory, it will require the memory system to fetch more pages. This is, obviously, less efficient.

But there’s another piece. Did you know that your CPU can make a prediction about what it thinks you are going to load into memory? It does. And if you don’t understand the rules of how it makes that prediction, chances are it won’t guess correctly.