Michael G

Briefly, an interesting negative result: consider benchmarks b1, b2,
b3 and so on, with associated .c and .h files. Consider libraries
p and q, with their .c and .h files. You want to run each
benchmark against each library.

P and Q implement the same API, but they have different ABI: you need to
separately compile each benchmark for each library. You also need to
separate compile each library for each benchmark, because p.c also
uses an abstract API implemented by b1.h, b2.h, and so on.

The problem: can you implement a short GNU Makefile that produces
executables b1.p, b1.q, b2.p, b2.q, and so on?

The answer would appear to be “no”.

You might think that with call and all the other functions available
to you, that surely this could be done, and indeed it’s easy to take the
cross product of two lists. But what we need are new rules, not just
new text or variables, and you can’t programmatically create rules. So
we have to look at rules to see what facilities are available.

Consider the rules for one target:

b1.p.lib.o: p.c
	$(CC) -o $@ -include b1.h $<
b1.p.bench.o: b1.c
	$(CC) -o $@ -include p.h $<
b1.p: b1.p.lib.o b1.p.bench.o
    $(CC) -o $@ $<

With pattern
rules,
you can easily modify these rules to parameterize either over
benchmark or over library, but not both. What you want is something
like:

*.%.lib.o: %.c
	$(CC) -o $@ -include $(call extract_bench,$@) $<
%.*.bench.o: %.c
	$(CC) -o $@ -include $(call extract_lib,$@) $<
%: %.lib.o %.bench.o
    $(CC) -o $@ $<

But that doesn’t work: you can’t have a wildcard (*) in the pattern
rule. (Really you would like to be able to match multiple patterns, but
the above is the closest thing I can think of to what make has.)

Static pattern
rules
don’t help: they are like pattern rules, but more precise as they apply
only to a specific set of targets.

You might think that you could use $* or other special variables on
the right-hand side of a pattern rule, but that’s not the case.

You might think that secondary
expansion
might help you, but then you open the door to an annoying set of
problems: sure, you can mix variable uses that are intended to be
expanded once with those to be expanded twice, but the former set better
be idempotent upon second expansion, or things will go weird!

Perhaps the best chance for a make-only solution would be to recurse
on generated makefiles, but that seems to be quite beyond the pale.

To be concrete, I run into this case when benchmarking
Whippet: there are some number of
benchmarks, and some number of collector configurations. Benchmark code
will inline code from collectors, from their header files; and
collectors will inline code from benchmarks, to implement the
trace-all-the-edges functionality.

So, with Whippet I am left with the strange conclusion that the only
reasonable thing is to generate the
Makefile
with a little custom generator, or at least generate the part of it to
do this benchmark-library cross product. It’s hard to be certain about
negative results with make; perhaps there is a trick. If so, do let
me know!