GNU Make is a popular and commonly used program for building C language software. It is used when building the Linux kernel and other frequently used GNU/Linux programs and software libraries.
Most embedded software developers will work with GNU Make at some point in their career, either using it to compile small libraries or building an entire project. Though there are many, many alternatives to Make, it’s still commonly chosen as the build system for new software given its feature set and wide support.
This article explains general concepts and features of GNU Make and includes recommendations for getting the most out of a Make build! Consider it a brief guided tour through some of my favorite/most used Make concepts and features 🤗.
If you feel like you already know Make pretty well, feel free to skip the tutorial portion and jump to my personal recommendations.
Table of Contents
What is GNU Make?
GNU Make is a program that automates the running of shell commands and helps with repetitive tasks. It is typically used to transform files into some other form, e.g. compiling source code files into programs or libraries.
It does this by tracking prerequisites and executing a hierarchy of commands to produce targets.
Although the GNU Make manual is lengthy, I suggest giving it a read as it is the best reference I’ve found: https://www.gnu.org/software/make/manual/html_node/index.html
Let’s dive in!
When to choose Make
Make is suitable for building small C/C++ projects or libraries that would be included in another project’s build system. Most build systems will have a way to integrate Make-based sub-projects.
For larger projects, you will find a more modern build system easier to work with.
I would suggest a build system other than Make in the following situations:
- When the number of targets (or files) being built is (or will eventually be) in the hundreds.
- A “configure” step is desired, which sets up and persists variables, target definitions, and environment configurations.
- The project is going to remain internal or private and will not need to be built by end users.
- You find debugging a frustrating exercise.
- You need the build to be cross platform that can build on macOS, Linux, and Windows.
In these situations, you might find using CMake, Bazel, Meson, or another modern build system a more pleasurable experience.
Invoking Make
Running make
will load a file named Makefile
from the current directory and attempt to update the default goal (more on goals later).
Make will search for files named
GNUmakefile
,makefile
, andMakefile
, in that order
You can specify a particular makefile with the -f/--file
argument:
You can specify any number of goals by listing them as positional arguments:
You can pass Make a directory with the -C
argument, and this will run Make as if it first cd
‘d into that directory.
Fun fact:
git
also can be run with-C
for the same effect!
Parallel Invocation
Make can run jobs in parallel if you provide the -j
or -l
options. A guideline I’ve been told is to set the job limit to 1.5 times the number of processor cores you have:
Anecdotally, I’ve seen slightly better CPU utilization with the -l
“load limit” option, vs. the -j
“jobs” option. YMMV though!
There are a few ways to programmatically find the CPU count for the current machine. One easy option is to use the python multiprocessing.cpu_count()
function to get the number of threads supported by the system (note on a system with hyper-threading, this will use up a lot of your machine’s resources, but is probably preferable to letting Make spawn an unlimited number of jobs).
Output During Parallel Invocation
If you have a lot of output from the commands Make is executing in parallel, you might see output interleaved on stdout
. To handle this, Make has the option --ouput-sync
.
I recommend using --output-sync=recurse
, which will print the entire output of each target’s recipe when it completes, without interspersing other recipe output.
It also will output an entire recursive Make’s output together if your recipe is using recursive make.
Anatomy of a Makefile
A Makefile contains rules used to produce targets. Some basic components of a Makefile are shown below:
Let’s take a look at each part of the example above.
Variables
Variables are used with the syntax $(FOO)
, where FOO
is the variable name.
Variables contain purely strings as Make does not have other data types. Appending to a variable will add a space and the new content:
Variable Assignment
In GNU Make syntax, variables are assigned with two “flavors”:
- recursive expansion:
variable = expression
The expression on the right hand side is assigned verbatim to the variable- this behaves much like a macro in C/C++, where the expression is evaluated when the variable is used: - simple expansion:
variable := expression
This assigns the result of an expression to a variable; the expression is expanded at the time of assignment:
Note: the
$(info ...)
function is being used above to print expressions and can be handy when debugging makefiles!*`
Variables which are not explicitly, implicitly, nor automatically set will evaluate to an empty string.
Environment Variables
Environment variables are carried into the Make execution environment. Consider the following makefile for example:
If we set the variable YOLO
in the shell command when running make
, we’ll set the value:
Note: Make prints the “No targets” error because our makefile had no targets listed!
If you use the ?=
assignment syntax, Make will only assign that value if the variable doesn’t already have a value:
Makefile:
We can then override $(CC)
in that makefile:
Another common pattern is to allow inserting additional flags. In the makefile, we would append to the variable instead of directly assigning to it.
This permits passing extra flags in from the environment:
This can be very useful!
Overriding Variables
A special category of variable usage is called overriding variables. Using this command-line option will override the value set ANYWHERE ELSE in the environment or Makefile!
Makefile:
Command:
Overriding variables can be confusing, and should be used with caution!
Target-Specific Variables
These variables are only available in the recipe context. They also apply to any prerequisite recipe!
Implicit Variables
These are pre-defined by Make (unless overridden with any other variable type of the same name). Some common examples:
$(CC)
- the C compiler (gcc
)$(AR)
- archive program (ar
)$(CFLAGS)
- flags for the C compiler
Full list here:
https://www.gnu.org/software/make/manual/html_node/Implicit-Variables.html
Automatic Variables
These are special variables always set by Make and available in recipe context. They can be useful to prevent duplicated names (Don’t Repeat Yourself).
A few common automatic variables:
See more at: https://www.gnu.org/software/make/manual/html_node/Automatic-Variables.html
Targets (Goals)
Targets are the left hand side in the rule syntax:
Targets almost always name files. This is because Make uses last-modified time to track if a target is newer or older than its prerequisites and whether it needs to be rebuilt!
When invoking Make, you can specify which target(s) you want to build as the goal
s by specifying it as a positional argument:
If you don’t specify a goal in the command, Make uses the first target specified in the makefile, called the “default goal” (you can also override the default goal if you need to).
Phony Targets
Sometimes it’s useful to have meta-targets like all
, clean
, test
, etc. In these cases, you don’t want Make to check for a file named all
/clean
etc.
Make provides the .PHONY
target syntax to mark a target as not pointing to a file:
If you have multiple phony targets, a good pattern might be to append each to .PHONY
where it’s defined:
NOTE!!!
.PHONY
targets are ALWAYS considered out-of-date, so Make will ALWAYS run the recipe for those targets (and therfore any target that has a.PHONY
prerequisite!). Use with caution!!
Implicit Rules
Implicit rules are provided by Make. I find using them to be confusing since there’s so much behavior happening behind the scenes. You will occasionally encounter them in the wild, so be aware.
Here’s a quick example:
Full list of implicit rules here:
https://www.gnu.org/software/make/manual/html_node/Catalogue-of-Rules.html
Pattern Rules
Pattern rules let you write a generic rule that applies to multiple targets via pattern-matching:
The rule will then be used to make any target matching the pattern, which above would be any file matching %.o
, e.g. foo.o
, bar.o
.
If you use those .o
files mentioned above to build a program:
Prerequisites
As seen above, these are targets that Make will check before running a rule. They can be files or other targets.
If any prerequisite is newer (modified-time) than the target, Make will run the target rule.
In C projects, you might have a rule that converts a C file to an object file, and you want the object file to rebuild if the C file changes:
Automatic Prerequisites
A very important consideration for C language projects is to trigger recompilation if an #include
header files change for a C file. This is done with the -M
compiler flag for gcc/clang, which will output a .d
file you will then import with the Make include
directive.
The .d
file will contain the necessary prerequisites for the .c
file so any header change causes a rebuild. See more details here:
https://www.gnu.org/software/make/manual/html_node/Automatic-Prerequisites.html http://make.mad-scientist.net/papers/advanced-auto-dependency-generation/
The basic form might be:
Order-Only Prerequisites
These prerequisites will only be built if they don’t exist; if they are newer than the target, they will not trigger a target re-build.
A typical use is to create a directory for output files; emitting files to a directory will update its mtime
attribute, but we don’t want that to trigger a rebuild.
Recipe
The “recipe” is the list of shell commands to be executed to create the target. They are passed into a sub-shell (/bin/sh
by default). The rule is considered successful if the target is updated after the recipe runs (but is not an error if this doesn’t happen).
If any line of the recipe returns a non-zero exit code, Make will terminate and print an error message. You can tell Make to ignore non-zero exit codes by prefixing with the -
character:
Prefixing a recipe line with @
will disable echoing that line before executing:
Make will expand variable/function expressions in the recipe context before running them, but will otherwise not process it. If you want to access shell variables, escape them with $
:
Advanced Topics
These features are less frequently encountered, but provide some powerful functionality that can enable sophisticated behavior in your build.
Functions
Make functions are called with the syntax:
where arguments
is a comma-delimited list of arguments.
Built-in Functions
There are several functions provided by Make. The most common ones I use are for text manipulation: https://www.gnu.org/software/make/manual/html_node/Text-Functions.html https://www.gnu.org/software/make/manual/html_node/File-Name-Functions.html
For example:
User-Defined Functions
You can define your own functions as well:
A more complicated but quite useful example:
Shell Function
You can have Make call a shell expression and capture the result:
I’m cautious when using this feature, though; it adds a dependency on whatever programs you use, so if you’re calling more exotic programs, make sure your build environment is controlled (e.g. in a container or with Conda).
Conditionals
Make has syntax for conditional expressions:
The “complex conditional” syntax is just the if-elseif-else
combination:
include
Directive
You can import other Makefile contents using the include
directive:
sources.mk
:
Makefile
:
Sub-make
Invoking Make from a Makefile should be done with the $(MAKE)
variable:
This is often used when building external libraries. It’s also used heavily in Kconfig builds (e.g. when building the Linux kernel).
Note that this approach has some pitfalls:
- Recursive invocation can result in slow builds.
- Tracking prerequisites can be tricky; often you will see
.PHONY
used.
More details on the disadvantages here:
http://aegis.sourceforge.net/auug97.pdf
Metaprogramming with eval
Make
’s eval
directive allows us to generate Make syntax at runtime:
Note that approaches using this feature of Make can be quite confusing, adding helpful comments explaining what the intent is can be useful for your future self!
VPATH
VPATH
is a special Make variable that contains a list of directories Make should search when looking for prerequisites and targets.
It can be used to emit object files or other derived files into a ./build
directory, instead of cluttering up the src
directory:
I recommend avoiding use of VPATH
. It’s usually simpler to achieve the same out-of-tree behavior by outputting the generated files in a build directory without needing VPATH
.
touch
You may see the touch
command used to track rules that seem difficult to otherwise track; for example, when unpacking a toolchain:
I recommend avoiding the use of touch
. However there are some cases where it might be unavoidable.
Debugging Makefiles
I typically use the Make equivalent of printf
, the $(info/warning/error)
functions, for small problems, for example when checking conditional paths that aren’t working:
For debugging why a rule is running when it shouldn’t (or vice versa), you can use the --debug
options: https://www.gnu.org/software/make/manual/html_node/Options-Summary.html
I recommend redirecting stdout to a file when using this option, it can produce a lot of output.
Profiling
For profiling a make invocation (e.g. for attempting to improve compilation times), this tool can be useful:
https://github.com/rocky/remake
Check out the tips here for compilation-related performance improvements:
https://interrupt.memfault.com/blog/improving-compilation-times-c-cpp-projects
Using a Verbose Flag
If your project includes a lot of compiler flags (search paths, lots of warning flags, etc.), then you may want to simplify the output of Make rules. It can be useful to have a toggle to easily see the full output, for example:
To enable printing out the full compilation commands, set the V
environment variable like so:
Full Example
Here’s an annotated example of a complete build process for an example C project. You can see this example and the source tree here.
Recommendations
A list of recommendations for getting the most of Make:
- Targets should usually be real files.
- Always use
$(MAKE)
when issuing sub-make commands. - Try to avoid using
.PHONY
targets. If the rule generates any file artifact, consider using that as the target instead of a phony name! - Try to avoid using implicit rules.
- For C files, make sure to use
.d
automatic include tracking! - Use metaprogramming with caution.
- Use automatic variables in rules. Always try to use
$@
for a recipe output path, so your rule and Make have the exact same path. - Use comments liberally in Makefiles, especially if there is complicated behavior or subtle syntax used. Your co-workers (and future self) will thank you.
- Use the
-j
or-l
options to run Make in parallel! - Try to avoid using the
touch
command to track rule completion
Outro
I hope this article has provided a few useful pointers around GNU Make!
Make remains common in C language projects, most likely due to its usage in the Linux kernel. Many recently developed statically compiled programming languages, such as Rust or Go, provide their own build infrastructure. However, when integrating Make-based software into those languages, for example when building a C library to be called from Rust, it can be surprisingly helpful to understand some Make concepts!
You may also encounter automake in open source projects (look for a ./configure
script). This is a related tool that generates Makefiles, and is worth a look (especially if you are writing C software that needs to be very widely portable).
There are many competitors to GNU Make available today, I encourage everyone to look into them. Some examples:
- CMake is pretty popular (the Zephyr project uses this) and worth a look. It makes out-of-tree builds pretty easy
- Bazel uses a declarative syntax (vs. Make’s imperative approach)
- Meson is a meta-builder like
cmake
, but by default uses Ninja as the backend, and can be very fast
References
Good detailed dive into less common topics (shout out on
remake
):
https://blog.jgc.org/2013/02/updated-list-of-my-gnu-make-articles.htmlMix of very exotic and simpler material:
https://tech.davis-hansson.com/p/make/Useful tutorial:
http://maemo.org/maemo_training_material/…Nice pictures:
https://www.jfranken.de/homepages/johannes/vortraege/make.en.htmlVery nice summary:
https://www.alexeyshmalko.com/2014/7-things-you-should-know-about-make/