Vertexbuffer.com

Automatic Hardware Prefetching

Your memory system contains an automatic prefetching mechanism that attempts to fill cache with data before you ask for it. One caveat, the prediction about your movement is not very sophisticated. If you move though memory in a sequential manor, it is likely that your data will be sitting in cache waiting for a load into the register. Automatic hardware prefetching will attempt to load two cache lines ahead of the processors memory access.
So what do I mean by sequential? Automatic hardware prefetch will fetch data ahead of you if you move forward or backwards. It will also predict your movement in strides; however, the length of the stride is can predict is different from processor.

Software Prefetch

A alternative to the automatic hardware prefetch is the software prefetch. Unlike the automatic prefetch, the software prefetch allow the user to give a “hint” to the cache system about what memory we are going to load next. This hint, which executes as a line of code, does have an overhead associated with it. Abusing software prefetching can cause more performance problems than it solves.

Software prefetching is a worthwhile option when you are accessing memory in ways that are not optimal for the automatic hardware prefetching. One example of an opportunity to use software prefetching is a memory lookup on a texture.

It is generally a better, more efficient practice, for a game programmer to organize data for hardware prefetch then to add in code to use a software prefetch.