Given typical defect densities, any non-trivial design or code is going to contain some errors. Even seemingly trivial maintenance fixes are likely to be defective.

It’s your job as a reviewer to find as many of these defects as possible. If you’re not finding defects, you’re wasting your time on reviews.

That “one simple step”? Remind yourself that there are almost certainly defects in the work product you’re reviewing, and then find them. It’s all about attitude.

Related posts:

1. Who Else Wants Better Short Term Memory? In “Talent is Overrated”, Geoff...
2. 3 Easy Ways to Stick to a Coding Standard When you’re writing python, you...
3. 9 “Must-Have” Tools for Software Teams The items below are useful...

His simple example is the 13-letter word “lexicographer”. To you and I (assuming you speak English and have a decent vocabulary), it is easy to remember. We don’t have to remember 13 letters, we just remember the whole chunk. But when presented with “trgdpxhdewfwm” for 3 seconds, you probably can’t remember more than half a dozen letters.

Another example is that chess masters can recall board positions after being shown a chess board with pieces on it for just a few seconds. They do this by chunking sets of pieces together — almost like “words” — whereas novices will try to remember individual pieces (“letters”).

It struck me that programmers do the same thing when reading and writing code.

The coding standard helps us, by telling us where the chunks are and how to draw the boundaries between chunks.

If you have a coding standard.

And apply it consistently.

When you choose names at random, you destroy your short term memory. You become a crappy programmer, and you drag everyone around you down too.

Quick example in C. Assume you’re writing a small module for checking environmental status.

BOOL isOverTemp();
BOOL isUnderTemp();

int GetCurTemperature();

BOOL env_isOverVoltage();
BOOL env_isUnderVoltage();

Anyone writing this pile of gibberish should be fired excommunicated.

Why?

For starters, I just wrote it, and I can’t now remember which was camel case and which was underscored. Also: which one was abbreviated?

Before you object that this is a contrived example, two points: (1) yes, it is contrived, that’s the point of an example, and (2) I have worked with far worse on an almost daily basis — the example above is fairly tame.

On the other hand, if you know the coding standard has some simple rules — and they are followed uniformly — you can easily remember the function names.

1. Initial 2-3 letter module prefix (“env” for this example).
2. All functions are lowercase with underscores.
3. No abbreviations.

Then we have:

BOOL env_is_over_temperature();
BOOL env_is_under_temperature();

int env_get_cur_temperature();

BOOL env_is_over_voltage();
BOOL env_is_under_voltage();

Now, when you’re writing, reviewing, or maintaining code, you don’t need to constantly refer to the header or documentation to get it right. A simple three-line coding standard just boosted your memory capacity by over 643%.

(In the interest of full disclosure: I made that number up.)

I realize that implementing even this simple three line standard is controversial because all the camel case folks are pulling out big sticks and the underscore people are grabbing rocks. To which I say: just flip a coin and enjoy the extra brain power. Adopt a three-line standard, follow it, and save the curly-brace debate for the next major coding standard revision.

Related posts:

1. 3 Easy Ways to Stick to a Coding Standard When you’re writing python, you...
2. One Simple Step for Avoiding Shallow Reviews It's your job as a reviewer...
3. Open an SSH Tunnel in Four Seconds or Less As I mentioned in a previous...

Jason Cohen pointed to an article of his in which he said “List no items that can be automated.” (His emphasis, but I second the motion.)

In the context of the SANS paper, let’s look at five items that should be automated so that no human has to find the defects:

1. CWE-665: Improper Initialization. As the write-up for CWE-665 suggests, you could use a programming language that forces you to explicitly initialize all variables before use. Or, if you’re stuck with something like C, make sure you turn on the warnings in your compiler, or use a static analysis tool (e.g. lint) to verify that all variables are initialized before being used. Tools like Coverity can be very sophisticated in their static analysis.
2. CWE-119: Failure to Constrain Operations within the Bounds of a Memory Buffer. Same goes for this one. First, use a programming language that doesn’t require attention to every tiny little detail of string handling. Failing that, apply static analysis. It’s also worth performing runtime analysis (e.g. electric fence, Purify) with appropriate test cases to verify that you’ve avoided buffer overflows.
3. CWE-404: Improper Resource Shutdown or Release. Even garbage collected languages can have problems with resource release. First, some GC systems have problems with circular references. Second, GC systems are typically worried about releasing memory. You also have to worry about database handles, file handles, and sockets. Configure your static analysis tool to track resources like these so that you don’t have to do it manually.
4. The write-up for CWE-404 also mentions in passing that you should “wash your garbage before you dispose of it”. This is useful for two reasons: first, an attacker won’t have access to the contents of previously used memory, and second, it often makes debugging memory problems easier if you write a known value into freed memory. (I modified malloc() at my last job to write 0xcacacaca into freed memory so that we would know right away when someone stepped on a turd, um, I mean used freed memory.) Configure your libraries to shred objects before you free them and you won’t have to worry about it on a line-by-line basis in the code you review.
5. CWE-362: Race Condition. I’m typically worried less about security than the nightmare of isolating and debugging race conditions. I haven’t seen any tools that detect race conditions well, but Coverity does a decent job of telling you when you’ve mishandled a race condition in a couple of cases: first, it warns you when you fail to release a lock, and second, it warns you when you access a resource that is statistically accessed from within a lock. So you still have to beware of race conditions (at the design level is the bset place to find these), but there is an option for finding tedious errors in the mechanics of dealing with races.

And now I’m going to throw out everything above.  Remember that the tools are built to detect common errors. If it is really important that you find a certain class of bug, put it on your checklist. In fact, if it’s super critical, make it a checklist of length one. For example, I’ve gone over codebases with the single-minded focus of finding race conditions. When you perform a review like this, it’s amazing how many defects you might find in areas that you never suspected.

Related posts:

1. One Simple Step for Avoiding Shallow Reviews It's your job as a reviewer...
2. Who Else Wants Better Short Term Memory? In “Talent is Overrated”, Geoff...
3. An Interesting pid File Race ISC’s dhcpd uses this code...

Why A Review Checklist is Essential

Most (all?) experts recommend the use of a checklist for performing design and code reviews. It’s more effective to ask a set of questions about the item being reviewed than to simply stare at a bunch of code looking for problems. The checklist focuses your attention on aspects that are most important.

If you want to perform a broader review, develop two or three checklists with different foci and give the lists to separate reviewers. The defect lists from each reviewer will have less overlap than if you gave everyone the same list.

What Your Review Checklist Should Look Like

The following advice is partly based on advice from “Software Inspection” by Gilb and Graham. A good checklist is a list of questions. For practical purposes, I’m strongly in favor of a checklist that does not exceed one side of a printed page — at normal font size. For me this means a checklist of about 15 items, 20 as an absolute maximum. Remember, we’re focusing, which means looking harder for just a few types of items. If you’re using online checklists, I’d extend this to mean that it should fit on the screen without scrolling. Either way, I don’t think you should have more than 20 questions or so at the max. Fewer is probably better, more is too overwhelming.

Checklist items are yes/no questions. They should all be phrased such that a “no” answer means something is broken.

When reviewing, proceed through the entire item under review (the entire code listing or design) with each question. Don’t work linearly, this is a horrible way to review code. I prefer to review on paper, simply because I can make little notes and marks all over the paper as I go through it. For the first question, you might work mostly front-to-back. Then the second question might send you backwards through the code. After a couple of passes over the code you have some familiarity with it, so the third question might send you straight to a location that has potential problems.

How To Choose Questions for the Review Checklist

Since the whole point of the checklist (and reviews in general) is to find defects, the questions should focus on finding the highest value defects. Why waste valuable checklist real estate on a question like “Does the code conform to formatting conventions?” Duh. If your team has a formatting convention, and you look at a piece of code that doesn’t conform, it’s going to jump off the page and bite you in the nose. You don’t need a checklist question for this!

Back to the specific problem at hand: security errors. SANS has done a great job of sifting through tons of security errors and boiling them down to the top 25. We could probably develop a 100 question review checklist from their Top 25 list, but it would be too big to be usable. So let’s focus on the most important for our application.

For this exercise, our application is a blog like Wordpress but much simpler: no plugins, no comments, single author. This is not a hosted application like Blogger, so we are less concerned about vulnerabilities like cross-site scripting, SQL injection, and OS command injection. We do need to make sure that only the author is allowed to post, so let’s look at the “Porous Defenses” category.

• CWE-285: Improper Access Control (Authorization)
• CWE-327: Use of a Broken or Risky Cryptographic Algorithm
• CWE-259: Hard-Coded Password
• CWE-732: Insecure Permission Assignment for Critical Resource
• CWE-330: Use of Insufficiently Random Values
• CWE-250: Execution with Unnecessary Privileges
• CWE-602: Client-Side Enforcement of Server-Side Security

The entry for CWE-285: Improper Access Control lists the following for prevention and mitigation:

For web applications, make sure that the access control mechanism is enforced correctly at the server side on every page. Users should not be able to access any information that they are not authorized for by simply requesting direct access to that page. One way to do this is to ensure that all pages containing sensitive information are not cached, and that all such pages restrict access to requests that are accompanied by an active and authenticated session token associated with a user who has the required permissions to access that page.

Here are some straightforward checklist questions we can extract from that description:

1. Does every administrative page enforce access control on the server side?
2. Are unauthenticated (anonymous) users prevented from accessing administrative pages?
3. Is caching disabled for administrative pages?
4. Does every administrative page validate the session token properly?

Now you can go walk through your design and/or code to verify that you can say “yes” to each of these questions.

If you found this post helpful, subscribe to The Daily Build to get updates as they are posted. Thanks!

Related posts:

1. One Simple Step for Avoiding Shallow Reviews It's your job as a reviewer...
2. 9 “Must-Have” Tools for Software Teams The items below are useful...
3. Using Python’s ctypes to Call Into C Libraries The ctypes module makes loading and...

Two answers:

1. Because it feels like an extra step. If you want to change a behavior like time tracking, you have to make it automatic. Developers and other project members are already too busy, you can’t just ask them to add another step to their daily workflow unless the benefits are immediate and obvious. The benefits of time tracking are not immediate and obvious — it takes some time (granted a small amount of time) to reap the benefits of time tracking.
2. Second, they perceive a risk. Even if the software is free (they are providing a 30 day free download), there’s the risk that you spend an hour or two (or more) downloading and fiddling with the software only to discover that it won’t work in your environment for one reason or another.

To use concrete examples from my recent experience:

1. I’m developing in Python. I’ve used Pylint in the past and it has been helpful in pointing out bugs before I start testing. I want to use Pylint on my current project, but I keep skipping it because it feels like an extra step and I’m trying to get work done, not fiddle with extra tools. The solution is for me to integrate this tool into my workflow so that it is automatic and not an extra step. This is what I’ve done for the current iteration and I expect that it will stick.
2. I’ve recently been using more social media sites like twitter and I read about a toolbar for Firefox from Minggl. Sounded interesting and after I saw a few mentions of it I decided to give it a whirl. There should be a warning: “roadblocks ahead”.
   1. They make you register to download.
   2. The registration form does not allow periods or spaces in your last name, so it took a couple of tries to register. (This smacks of ethnocentrism! I am being oppressed because I have a period and a space in my last name!)
   3. It doesn’t work on linux… bummer.

   Contrast this with the experience of signing up for something like twitter or Mahalo Answers. You sign up (albeit with the gotcha of only allowing alphabetics in the name) and you start using it immediately.

Related posts:

1. 3 Easy Ways to Stick to a Coding Standard When you’re writing python, you...
2. One Simple Step for Avoiding Shallow Reviews It's your job as a reviewer...
3. How to Tell SSH Who You Are Do you log in to several...