Vertexbuffer.com

Optimization – A Right Brain Process

There are several characteristics that draw people towards computer science. Math and logic dominates our trade, and sometimes our lives. You could probably describe a group of computer scientists’ arguments by diagramming their statements in pseudo code and UML. Writing code sharpens the attributes associated with left-brained characteristics. The left side of our brain dominates our ability to think logically and absolutely. Because of these characteristics, many computer scientists are left-brained individuals.

Optimization relies heavily on left-brained abilities, but the right-brained activities, characterized by creative and abstract thinking, distinguishes it from many skills that we develop as computer scientists. Design, another aspect of computer science that relies on right-brained abilities, is an important aspect of optimization. So is detection, which is the process we use to locate the hotspots and bottlenecks that will yield the largest return for our efforts.

A well designed code base is the foundation for optimization. Without a flexible design, it will be difficult to adapt your code to different methods for achieving optimal performance. When we write code that is inflexible, we trap ourselves in one design that will be difficult to change in a reasonable amount of time. Inflexible code and inflexible schedules usually prohibit us from making all our desired optimizations.

The process of detection is where we discover what optimizations to perform next. During the detection process, our job resembles a detective as he gathers evidence. As programmers, we use our tools to gain insight about the way our code performs. When examining an executable, our tools show us different views of our program and each view is a clue that leads us to a conclusion about what to optimize next. Optimization “clues” are similar to the clues a detective uses to determine suspicion. Some clues, such are DNA are very conclusive, other clues are circumstantial and require analytical decisions and a hypothesis. Similar situations occur when gathering the data required to point us to the next, highest yielding optimization.

We optimize our code using educated guesses supported by correctly gathered data. It is very difficult to guess where optimization opportunities lie. Sometimes, it’s just difficult to correctly gather data. Graphics systems are very complex and contain many hidden hotspots and bottlenecks that create problems under different circumstances- it is arrogant to assume you know what code, or art resource, is causing the largest hotspot or bottleneck without first using data to back your conclusion.

Once determining where to optimize, a programmer will likely need to use a creative solution to increase performance. First, we should consider solutions that take full advantage of our hardware. Finding ways to move work from over utilized resources to under utilized resources may require unconventional means. Another tactic is to find better ways to do less work. Instead of calling a function five thousand times, maybe a better algorithm can return the same result with only five hundred calls. A third type of optimization focuses on line by line optimizations. Sometimes, this means using hand coded assembly or trade-offs that increase memory or reduce accuracy to achieve better performance. All of these methods are viable, challenging, and often require creative solutions.

Optimization requires us to think abstractly, especially when trying to identify a section of code that is slow. As stated above, many of our tools show us performance problems in sections of code. Those sections, depending on the view, are more similar to many Venn diagrams then they are to discrete groups. We choose to create groups in a defined way to facilitate an organized process. Some people like to pay careful attention to line-by-line optimizations through out their whole program while others prefer to optimize one function at a time. Both strategies have their merit, therefore neither are incorrect; however, the correct mix of both strategies at the correct time are the most efficient. Since the process of optimization is nearly indefinite, it is important to consider practices that allow us to maximize efficiency. In order to optimize efficiently, it is helpful to visualize groups of code in a hieratical tree to facilitate abstract thinking.

There are occasions when optimizing, where our tools point to a symptom of a problem, not the problem itself. These red herrings surface quite often when performance problems exist due to incorrect API usage. The tool will suggest that a section of code is running slow inside a module, and although that’s true, the reason for the performance decrease is due to incorrect API usage.

It is very easy to use an API to achieve correct results in a suboptimal way. Without reading the documentation or understanding the assumptions of the API developers, it may be unclear that you have done anything wrong. It can be very difficult to determine if the data you are viewing is the problem, or the symptom of another problem. There are many opportunities to use an API incorrectly; most are due to incorrect submission of the size of data, the frequency of use of a subsystem or function, the incorrect order, or calls that cause inefficient use of parallelism due to poor synchronization.

When performance problems are the result of poor parallelism, the results from our tools can be quite confusing. Sometimes, our tools will report a hotspot that suggests we are overusing an idle process. An idle process runs when we are not utilizing that hardware to its fullest extent. And although we may be spending a lot of time in an idle process, the tool is showing us a potential opportunity for more parallel work, not a hotspot caused by overuse of a useful function. Note that taking advantage of more parallelization does not guarantee a performance increase; there are other factors that may affect our ability to increase performance through parallelization.

Poor parallelization can also skew the results of testing how busy a hardware resource is. Tight loops of idle work, sometimes used for synchronization, can cause our tools to report a busy hardware resource. Busy hardware performing useful processing is a goal when optimizing; busy hardware performing idle operations is a waist of expensive hardware.

Lastly, we have to be very deductive when a hotspot or bottleneck is located in sections of code we did not write. Code reuse through APIs and specialized hardware increase our productivity since programmers don’t need to reinvent the wheel. It is no longer a programmer’s responsibility to write a software rasterizer if they would like to develop a game. The downside is that we have very little visibility of the problem and potential solution. It is unlikely that we will be using less APIs in the future. This is especially true when considering the potential for increased numbers of specialized hardware such as physics cards.

Even though these difficulties exist, it is still justifiable to suggest that Video Game Optimization is not as difficult as you may believe. The challenge is to sharpen the skills that are not a part of traditional left-brain programming. A goal of this book is to provide a logical process in order to facilitate thinking that us, as computer programmers, are more accustomed to.