New gcc warning catches plenty of bugs

A new warning message added to gcc catches hundreds of potential memory corruptions in the kernel.

One of the features of gcc14 is a new warning for improper flexible arrays, -Wflex-array-member-not-at-end, contributed by Oracle developer Qing Zhao. This enhancement was highlighted in an excellent talk by Google security researcher Gustavo A. R. Silva at last week’s Kernel Recipes conference, and identified a large number of (potential) bugs in the kernel.

An interesting (ab)use of the C standard allows for variable length objects to be defined as “fake flexible arrays” in C. This use is fairly common in the Linux kernel as it allows array length to be defined at run time, but it was only codified in the C99 standard and (until now) not explicitly supported by the compiler. While useful, this construct is also particularly ‘fragile’, as the flexible array members must be always at the end of a struct – or risk undefined behavior when any trailing struct members read or write on top of the flexible array. This results in very subtle bugs where adding a member to a struct can create a crash or security hole.

An example of how subtle this bug can be was highlighted in the bug introduced in this commit where undefined behavior was prevented simply by placing the new struct member before the flexible array, rather than after.

Can you spot the bug introduced by this code change? Click here for the answer.

--- a/drivers/net/cxgb3/l2t.h
+++ b/drivers/net/cxgb3/l2t.h
@@ -76,6 +76,7 @@ struct l2t_data {
    atomic_t nfree;     /* number of free entries */
    rwlock_t lock;
    struct l2t_entry l2tab[0];
+   struct rcu_head rcu_head;   /* to handle rcu cleanup */
 };

Also see this article on LWN that pre-dates the compiler warning.

Thanks to Qing, Gustavo and the Kernel Self Protection Project for helping to keep Linux secure!

UPDATE: New LWN article goes into detail on Qing's changes. Check it out! Not a subscriber? Please consider supporting LWN with a subscription, it's a fantastic resource for the Linux community. Also corrected the link to point to the standalone talk instead of the livestream.