Falsehoods programmers believe about build systems

[pozorvlak]

Inspired by Falsehoods Programmers Believe About Names, Falsehoods Programmers Believe About Time, and far, far too much time spent fighting autotools. Thanks to Aaron Crane, totherme and zeecat for their comments on earlier versions.

It is accepted by all decent people that Make sucks and needs to die, and that autotools needs to be shot, decapitated, staked through the heart and finally buried at a crossroads at midnight in a coffin full of millet. Hence, there are approximately a million and seven tools that aim to replace Make and/or autotools. Unfortunately, all of the Make-replacements I am aware of copy one or more of Make's mistakes, and many of them make new and exciting mistakes of their own.

I want to see an end to Make in my lifetime. As a service to the Make-replacement community, therefore, I present the following list of tempting but incorrect assumptions various build tools make about building software.

All of the following are wrong:

• Build graphs are trees.
• Build graphs are acyclic.
• Every build step updates at most one file.
• Every build step updates at least one file.
• Compilers will always modify the timestamps on every file they are expected to output.
• It's possible to tell the compiler which file to write its output to.
• It's possible to tell the compiler which directory to write its output to.
• It's possible to predict in advance which files the compiler will update.
• It's possible to narrow down the set of possibly-updated files to a small hand-enumerated set.
• It's possible to determine the dependencies of a target without building it.
• Targets do not depend on the rules used to build them.
• Targets depend on every rule in the whole build system.
• Detecting changes via file hashes is always the right thing.
• Detecting changes via file hashes is never the right thing.
• Nobody will ever want to rebuild a subset of the available dirty targets.
• People will only want to build software on Linux.
• People will only want to build software on a Unix derivative.
• Nobody will want to build software on Windows.
• People will only want to build software on Windows.
  (Thanks to David MacIver for spotting this omission.)
• Nobody will want to build on a system without strace or some equivalent.
• stat is slow on modern filesystems.
• Non-experts can reliably write portable shell script.
• Your build tool is a great opportunity to invent a whole new language.
• Said language does not need to be a full-featured programming language.
• In particular, said language does not need a module system more sophisticated than #include.
• Said language should be based on textual expansion.
• Adding an Nth layer of textual expansion will fix the problems of the preceding N-1 layers.
• Single-character magic variables are a good idea in a language that most programmers will rarely use.
• System libraries and globally-installed tools never change.
• Version numbers of system libraries and globally-installed tools only ever increase.
• It's totally OK to spend over four hours calculating how much of a 25-minute build you should do.
• All the code you will ever need to compile is written in precisely one language.
• Everything lives in a single repository.
• Files only ever get updated with timestamps by a single machine.
• Version control systems will always update the timestamp on a file.
• Version control systems will never update the timestamp on a file.
• Version control systems will never change the time to one earlier than the previous timestamp.
• Programmers don't want a system for writing build scripts; they want a system for writing systems that write build scripts.

[Exercise for the reader: which build tools make which assumptions, and which compilers violate them?]

Comments

[ungratefulninja.livejournal.com]

- All build systems look like make.

[ungratefulninja.livejournal.com]

And on a related, $DAYJOB-aggravating note:

- Tools always exit with non-zero status on failure.
- Tools never exit with non-zero status on success.

[pozorvlak.livejournal.com]

Aaaaaargh. I feel your pain.

[david r. maciver (from livejournal.com)]

Though I'm mostly irritated by those tools rather than the build systems which get that "wrong"...

[gareth-rees.livejournal.com]

• A build always runs on a single computer.
  
• There's always a human available to interact with the build system.
  

[pozorvlak.livejournal.com]

Good ones!

[dr-strych9.livejournal.com]

• Build tools are used by writing scripts in one or more languages.

[dr-strych9.livejournal.com]

• Tools should not require users to write build scripts in a language of any kind.

[pozorvlak.livejournal.com]

I'm inclined to say that the less scripting your build manager has to do the better - though a tool that doesn't require scripting in any situation sounds like an impossibility (and a tool that doesn't allow scripting sounds like it would eventually become painful).

[senji.livejournal.com]

• Build steps are idempotent
• It can be known in advance how many times a particular build step should be executed
• Build steps do not modify the repository
• It is possible to determine whether the build will succeed
• It is possible to determine whether the build will even halt

[pozorvlak.livejournal.com]

*smacks forehead* - all good ones.

[(Anonymous)]

What is the point of this?

[(Anonymous)]

disabusement

[pozorvlak.livejournal.com]

Sorry that wasn't clear - I've added an explanation to the top of the post.

[(Anonymous)]

maven sux.

svn sux.

python sux.

lets agree on this first

[jens timmerman (from livejournal.com)]

* Users will only will only use one specific compiler, library and flags, so you can hardcode them in your build scripts.
* Users will always agree with the location you want to install stuff to.

Shameless plug:
At my current job (HPC system administator at Ghent University) we have been building a lot of software which had these assumptions.
So to automate all of them we created a framework EasyBuild (http://hpcugent.github.com/easybuild/) which is a layer on top off all these build systems that tries to correct their mistakes (be the human to answer questions during the installation, patching makefiles/code to work with different compilers, install under a prefix and generate module files...) and automate the process of building the software.
This is not usefull for programmers, but for end users who want to install the software.

[(Anonymous)]

build systems are a symptom of software languages that are not designed to build software systems

[dr-strych9.livejournal.com]

I hate make(1). I only slightly dislike OMake (http://omake.metaprl.org), which I'm sure would piss you off for a variety of reasons, but I bet it would piss you off less than make(1) too.