Performance Rant
I've often seen code with performance problems, and watched people struggle to improve it. I've also seen people struggling to achieve optimal performance while writing code. In both cases, I've witnessed lots of folly committed in the name of performance. So here are some of my observations on the subject:
If You Haven't Measured It, You Don't Know What You're Doing
In performance work, everybody has an instinct about what will help and what won't. If you're new at the game, your instincts will be wrong 80% of the time. Once you've got 20 years of experience under your belt, you'll only be wrong 40% of the time. What this really means is that opinions aren't worth the paper they're not printed on -- you've got to measure. Measure before you make a change, measure again afterward, compare. That's the mantra. Otherwise, you can make performance-related changes all day and not get anywhere.
Model It
During the early stages of development of something new, you generally don't have a working system that you can measure. This is where modeling helps.
A model can be anything from very simple to quite complex, and frequently includes the results of small prototypes. Most of my successful models involve simple spreadsheets. For example:
During the development of the WHS Backup product, I had a design point where I said that in order to complete a certain phase of a backup, I'll need to do 10 million random reads from disk. How long will that take? So I wrote a simple prototype, discovered that you can do about 50 random reads per second on one disk, divided, and determined it'll take almost 3 days. So it was time to come up with an alternate design.
With the next design, I concluded I could accomplish the same task by reading a gigabyte sequentially, writing a gigabyte sequentially, and sorting 10 million 24-byte records, all at the same time. How long would this take? More simple prototypes determined that I could sequentially read 50 megabytes per second, but could initially write only about 5 megabytes per second. Researching that, I eventually found out how to get the write rate up to 50 megabytes too. I also discovered that there weren't any pre-canned sort routines that could sort more data than fit in memory, and offer same-day service. With more prototyping and modeling, I eventually discovered how to write one of these that worked well, too.
I was still a long way from having a working system, but at least I had rejected a design approach that wasn't going to work, and gained confidence in a different approach.
Prototype It
Often during development, you'll want to answer questions like "how fast can I reticulate splines?" This is where very simple prototypes come into play. You code up a small program that does nothing but reticulate splines, and measure it.
Other times, you'll have a working system, and be thinking about performance changes that are a lot of work. "I think it might help to make the framistats non-persistent. But to do that I'll need to write 1500 lines of code to cope with the case where the Jeffries tubes boil over." You can either:
- Spend 6 months writing and debugging the code before you find out if it helps, or
- Just change your code to use non-persistent framistats, even though the code is no longer shippable.
If you do the second, and discover that it didn't really help performance, you've just saved yourself 6 months of work. (Of course, if it did really help, you're not absolutely guaranteed that the new 1500 lines of code won't obliterate the gains, but you've at least validated that your intuition is on the right path.)
The point here is that a prototype doesn't need to be a stand-alone thing. It can be your current system, plus a few tweaks.
By modeling and prototyping, you can save yourself from going down a path that will never work well. You can find it out at design time, long before you've spent the effort to code and debug things. Models and prototypes will seldom be good enough to prove that a design will work, but even rudimentary models and prototypes can often weed out designs that won't work.
Micro-Optimization Doesn't Help
Fred is chasing a performance problem. "Oh, look here," he says. "I'm repeatedly computing the size of my window from its bounds. I should compute it once and save it." He makes the change, and goes home feeling satisfied. The next day, he wonders why life didn't get any better.
|
The big issue here is that, in almost everything you'll ever write, the time isn't spent in your code. If you're writing a window-based CRUD application (database Create, Read, Update, Delete), substantially all of the time is going to either the window manager or the database manager. So making the details of your code more efficient just isn't going to help. Instead, you need to change the manner and frequency with which you call other people's code. So you have to learn which of the calls you're doing are expensive, and redesign to do fewer of them.
|
|
"We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil." (Donald Knuth)
|
The same principle applies even if, for example, you're building the database manager where all of the time is being spent. Even then, most of the CPU time isn't being spent in your component. You still need to design such that you minimize the number of expensive calls you make to components below you, and minimize the number of expensive calls that people above you need to make in order to accomplish their jobs.
This means that for most common micro-optimizations ("Should I hoist a+b out of that loop?") the answer is "it doesn't matter for performance. Which way expresses your intent most clearly?"
Fast Enough Is Fast Enough
At least 90% of the code you'll ever write will be fast enough, even before you think about performance. If you're writing something that will be used only once, it's not worth spending a day of your time to slice a minute off the execution time. If you're writing something interactive, it doesn't help to improve things even by a factor of 10, if you're just reducing the response time from 10 milliseconds to 1 millisecond.
So, where do you spend your time? Concentrate on those programs where:
- You have real users telling you it takes too long.
- You have models or prototypes that say that performance isn't even in the right ballpark.
It just doesn't matter how much you improve the performance of your solitare game; it was probably fast enough when you started.