Makefiles are Software Too

This post was inspired by recent experience with some horrible build scripts from the open source world — but I've seen enough in-house badness over the years that I wanted to establish some basic parameters for build scripts.

This is a really broad way to divide the world, but I see two important categories of build scripts. (By "build script" I mean Makefiles, SConsfiles, Ant files, and their ilk).

  1. Build scripts intended for in-house use, never seen by outsiders.
  2. Build scripts that you distribute to external users.

Both sets of build scripts are important. Let's consider the build script (or Makefile system, or whatever you're using) as a software system independent of the software that is being built. Its primary requirement is to reliably convert your source code into an executable. Any decent build script has numerous other requirements, most of which are probably implied. I have never seen a written requirement for the numerous build scripts I've written. I have only rarely heard them discussed — and never actually posed as "requirements".

I have always taken the following meta-requirements for granted. Based on my experience with some fairly awful build scripts, I guess these aren't universally acknowledged.

  1. The build script should build all the "normal" executables with a single command.
  2. The build script must encapsulate all environment variables within the script. (This is a corollary of the above.)
  3. The build script should not unnecessarily rebuild source code in a directory tree that has previously been built. (This is make's raison d'etre.)
  4. The build script should be documented. It doesn't have to be elaborate, but a five-line comment at the top of the script describing the available command-line variables would be nice.
  5. make clean or its equivalent must work reliably.
  6. It should rarely be necessary to run make clean.
  7. Bonus: the build should be parallelizable, to be able to take advantage of multicore machines and/or distributed builds.

Back to those two categories. If you screw up an in-house build script, it's primarily your team that's going to suffer. I'm not aware of any bugs from SQA or Customers on projects I've been on that were traced back to a build script, but I can see where it could happen. Mainly what I've seen is developers habitually wasting time rebuilding source that should not need to be rebuilt. The worst offender in this respect was a build script that forced a "make clean" at the beginning of every build. In the immortal words of Dave Barry, I'm not making this up. Really.

(Actually, I think it was even worse when I was dealing with a build script that required a "make clean" every time around but didn't have it coded into the script!)

I can't say I have a completely clean conscience in this area: I wrote a big pile of Makefiles for a past employer in which I unknowingly used features (bugs?) of a specific version of GNU Make that were "fixed" in a subsequent version. As time went on it became (a) a much bigger task to fix the Makefile code that used the "feature" and (b) harder to keep the specific version of GNU Make that worked with the Makefiles! I imagine they're still stuck with this situation... sorry guys.

In the second category, when you toss a slapped-together build script to your unsuspecting users you do all of human society a disservice. Now they are not only wasting their time fighting with a bad build script, but they often have the disadvantage of being unable to contact the author directly for support or commiserate with other experienced users of the system.

So do us all a favor: if you don't know what you're doing when you start to write your next build script, please ask for help. Otherwise we'll have to resort to some sort of professional licensing scheme for build script authors.

Posted on 2009-04-13 by brian in engineering .
Comments on this post are closed. If you have something to share, please send me email.