Introduction
In this blog we continue describing our recent security related GCC additions, the counted_by attribute and the __builtin_counted_by_ref builtin function. We will focus on extending them to another class of dynamically-sized arrays, i.e, pointers that point to arrays. Readers will learn the motivation for these new features and how to use them in their applications.
As we have learned in the previous blog on the topic of code security, (“Improvements in GCC Buffer Overflow Detection for C Flexible Arrays Members“), with the help of the new attribute counted_by and the new builtin function __builtin_counted_by_ref, most of the flexible array members in applications can now be annotated with size information in the source code. As a result, the current buffer overflow detection tools in GCC are able to protect most FAMs.
But what about the last class of the dynamically-sized array, pointers that point to arrays? It is natural to extend the counted_by attribute and __builtin_counted_by_ref to pointers that point to arrays. This extension has been completed and committed to GCC16 which will be released soon. In the meantime, the Linux kernel has been prepared to adopt these new extensions in order to provide more buffer overflow protection for pointers. The adoption status of these new features in Linux kernel and other applications is provided at the end of this blog.
Pointers to dynamically-sized arrays
Dynamically-sized arrays include variable-length arrays (VLAs), flexible array members (FAMs) in structures, and pointers (for example pointers into memory allocated via memory allocator calls such as malloc()).
The size of a VLA is carried in the variable declaration, for example, a[n] is a VLA whose number of elements is n.
The size of a FAM can be annotated with the new attribute counted_by, such as:
struct A {
unsigned count;
int buf[] __attribute__ ((counted_by (count)));
};
In the above, the size of the FAM buf[] is specified by the argument count of the attached attribute counted_by.
But, what about the size of a dynamically-sized array pointed to by a pointer?
The following is a simple example including a pointer to a dynamically-sized array when the pointer is a field of a structure (one of the most common situations with pointer arrays):
$ cat t1.c
struct pointer_array {
int *buf; /* a pointer to an array with integer elements. */
unsigned count; /* the number of elements of this dynamic array. */
};
static struct pointer_array __attribute__((__noinline__)) *setup (unsigned count)
{
struct pointer_array *p_array
= (struct pointer_array *) __builtin_malloc (sizeof (struct pointer_array));
p_array->buf = (int *) __builtin_malloc (sizeof (int) * count);
p_array->count = count;
return p_array;
}
int main ()
{
struct pointer_array *p = setup (10);
p->buf[12] = 10; /* out-of-bound access */
return 0;
}
In the above, the structure pointer_array includes a pointer field buf pointing to a dynamically allocated array, whose number of elements is not specified explicitly in the source code. Each object of this structure is allocated through the routine setup(), whose only parameter is the number of elements for the dynamic array that is pointed by the pointer field buf. Inside the routine setup(), memory for the array is allocated and the number of elements of the dynamic array buf is recorded into the other field count of the structure pointer_array.
Inside the function main(), an object of the structure pointer_array, p, with 10 elements in the array pointed by the pointer buf, is allocated first, followed by an out-of-bound access to this array.
With GCC15 and the bounds sanitizer (-fsanitize=bounds), the out-of-bound access to this dynamic array cannot be detected because the bound of the array is not known.
$ gcc -O -fsanitize=bounds t1.c; ./a.out
A similar test case which uses the builtin function __builtin_dynamic_object_size on pointers to dynamically-sized array would result in undetected buffer overflow too.
For example:
$ cat t2.c
#include
struct pointer_array {
char *buf; /* a pointer to an array with char elements. */
unsigned count;/* the number of elements of this dynamic array. */
};
#undef memset
#define memset(dest, src, n) \
__builtin___memset_chk (dest, src, n, __builtin_dynamic_object_size (dest, 1))
void __attribute__ ((noinline)) my_memset (struct pointer_array *p, int n)
{
memset(p->buf, 'b', n);
return;
}
static struct pointer_array __attribute__ ((noinline)) *setup (unsigned count)
{
struct pointer_array *p_array
= (struct pointer_array *) __builtin_malloc (sizeof (struct pointer_array));
p_array->buf = (char *) __builtin_malloc (sizeof (char) * count);
p_array->count = count;
return p_array;
}
int main ()
{
struct pointer_array *p = setup (8);
my_memset(p, 10);
__builtin_printf("Pass \n");
return 0;
}
In the above t2.c, the call to my_memset results in a buffer overflow at runtime as the destination buffer p->buf is smaller than the memset size.
But the overflow is not detected even by the __builtin___memset_chk, because the size of the array which is computed by the builtin __builtin_dynamic_object_size is not known to the compiler.
$ gcc -O t2.c; ./a.out Pass
Ideally, if the compiler could connect the value of the p->count to the size of the resulting buffer p->buf, the overflow could be detected.
Extend the attribute counted_by to pointers inside structures
For a well-defined structure with a pointer field, there is usually another field in the same structure that holds the number of elements for the array pointed by the pointer field, similarly to FAMs. It’s natural to extend the counted_by attribute to associate this count field to the pointer field to tell the compiler about the size of the array pointed by the pointer.
Starting from the upcoming GCC16, the counted_by attribute in GNU C can be attached to a pointer field inside a structure to inform the compiler about the size of the array pointed by this pointer.
With this new feature, the buffer overflow detection tools in GNU are able to detect more overflow issues.
Let’s slightly update the above t1.c with the counted_by attribute as follows:
$ cat t1_new.c
struct pointer_array {
int *buf __attribute__ ((counted_by (count))); /* a pointer to an array with integer elements. */
unsigned count; /* the number of elements of this dynamic array. */
};
static struct pointer_array __attribute__((__noinline__)) *setup (unsigned count)
{
struct pointer_array *p_array
= (struct pointer_array *) __builtin_malloc (sizeof (struct pointer_array));
p_array->buf = (int *) __builtin_malloc (sizeof (int) * count);
p_array->count = count;
return p_array;
}
int main ()
{
struct pointer_array *p = setup (10);
p->buf[12] = 10; /* out-of-bound access */
return 0;
}
Let’s compile and run the program. We observe the following:
$ gcc -O -fsanitize=bounds t1_new.c; ./a.out t2.c:18:9: runtime error: index 12 out of bounds for type 'int [*]'
We can see that the out-of-bound access is successfully caught by the sanitizer with the help from the attribute counted_by!
At the same time, the __builtin_dynamic_object_size can also provide accurate size information to the dynamically-sized array pointed by the pointer.
If we ONLY add a counted_by attribute to the above test case t2.c:
$ cat t2_new.c
#include
struct pointer_array {
char *buf __attribute__ ((counted_by (count))); /* a pointer to an array with char elements. */
unsigned count;/* the number of elements of this dynamic array. */
};
#undef memset
#define memset(dest, src, n) \
__builtin___memset_chk (dest, src, n, __builtin_dynamic_object_size (dest, 1))
void __attribute__ ((noinline)) my_memset (struct pointer_array *p, int n)
{
memset(p->buf, 'b', n);
return;
}
static struct pointer_array __attribute__ ((noinline)) *setup (unsigned count)
{
struct pointer_array *p_array
= (struct pointer_array *) __builtin_malloc (sizeof (struct pointer_array));
p_array->buf = (char *) __builtin_malloc (sizeof (char) * count);
p_array->count = count;
return p_array;
}
int main ()
{
struct pointer_array *p = setup (8);
my_memset(p, 10);
__builtin_printf("Pass \n");
return 0;
}
With the latest trunk GCC, we get:
$ gcc -O t2_new.c; ./a.out *** buffer overflow detected ***: ./a.out terminated Aborted (core dumped)
The buffer overflow can be detected successfully!
Special features of the counted_by attribute on pointers
In addition to the general requirements of the counted_by attribute as described in our previous blog, “Improvements in GCC Buffer Overflow Detection for C Flexible Arrays Members“, the counted_by attribute on pointers has some special features.
First, the counted_by attribute is not allowed for a pointer to function, or a pointer to a structure or union that includes a flexible array member.
For example, in the following two examples, struct pointer_func_array and struct pointer_struct_fam_array, the counted_by attribute is not allowed for the corresponding pointer field. The compiler issues errors for such usage during compilation time.
struct pointer_func_array {
int count;
int (*fpr)(int,int) __attribute__ ((counted_by (count))); /* error: "attribute is not allowed for a pointer to function" */
};
struct item {
int a;
float b[];
};
struct pointer_struct_fam_array {
int count;
/* structure with flexible array member is not allowed. */
struct item *array __attribute__ ((counted_by (count))); /* error: "attribute is not allowed for a pointer to structure or union with flexible array member" */
};
Second, even though the counted_by attribute is not allowed for the above two cases, it is allowed for a pointer to non-void incomplete structure or union types, as long as the type can be completed before the first reference to the pointer.
For instance, in the following test case “pointer-to-incomplete.c”, the structure struct pointer_array_incomplete includes four pointer fields: array_1 and array_3 point to the incomplete structures Item1 and Item3 and array_2 and array_4 point to the incomplete unions Item2 and Item4. GCC will not report any error when processing the counted_by attributes on these four pointer fields.
However, at the pointer reference points inside the routine foo(), GCC will check whether the pointer’s types are complete. If the types are still incomplete or the structure or union contains flexible array members at that time, GCC will report errors.
$ cat pointer-to-incomplete.c
typedef struct item1 Item1;
typedef union item2 Item2;
typedef struct item3 Item3;
typedef union item4 Item4;
/* Incomplete structure and union are allowed. */
struct pointer_array_incomplete {
int count1;
int count2;
Item1 *array_1 __attribute__ ((counted_by (count1)));
Item2 *array_2 __attribute__ ((counted_by (count2)));
int count3;
int count4;
Item3 *array_3 __attribute__ ((counted_by (count3)));
Item4 *array_4 __attribute__ ((counted_by (count4)));
};
struct item1 {
int a;
float b;
};
union item2 {
int c;
float d;
};
struct item3 {
int a;
float b[];
};
union item4 {
int c;
float d[];
};
void foo ()
{
struct pointer_array_incomplete *pointer_data
= (struct pointer_array_incomplete *) __builtin_malloc (sizeof (struct pointer_array_incomplete));
pointer_data->array_1
= (Item1 *) __builtin_malloc (sizeof (Item1) * 3);
pointer_data->count1 = 3;
pointer_data->array_2
= (Item2 *) __builtin_malloc (sizeof (Item2) * 6);
pointer_data->count2 = 6;
pointer_data->array_3 /* error: "attribute is not allowed for a pointer to structure or union with flexible array member */
= (Item3 *) __builtin_malloc (sizeof (Item3) + 3 * sizeof (float));
pointer_data->array_4 /* error: "attribute is not allowed for a pointer to structure or union with flexible array member */
= (Item4 *) __builtin_malloc (sizeof (Item4) + 3 * sizeof (float));
return;
}
$ gcc pointer-to-incomplete.c
pointer-to-incomplete.c: In function ‘foo’:
pointer-to-incomplete.c:48:15: error: ‘counted_by’ attribute is not allowed for a pointer to structure or union with flexible array member
48 | pointer_data->array_3 /* { dg-error "attribute is not allowed for a pointer to structure or union with flexible array member" } */
| ^~
pointer-to-incomplete.c:50:15: error: ‘counted_by’ attribute is not allowed for a pointer to structure or union with flexible array member
50 | pointer_data->array_4 /* { dg-error "attribute is not allowed for a pointer to structure or union with flexible array member" } */
| ^~
Third, the attribute counted_by is allowed for a pointer to void. However, warnings will be issued for such cases when -Wpointer-arith is specified. When this attribute is applied on a pointer to void, the size of each element of this pointer array is treated as 1.
For example, in the following structure pointer_array_void, the pointer field array is a pointer to void type. The counted_by attribute is allowed for such pointer. A warning will be issued when it is compiled with the option -Wpointer-arith.
$ cat pointer-to-void.c
struct pointer_array_void {
int count;
void *array __attribute__ ((counted_by (count)));
};
$ gcc -Wpointer-arith pointer-to-void.c
pointer-to-void.c:7:9: warning: ‘counted_by’ attribute is used for a pointer to void [-Wpointer-arith]
7 | void *array __attribute__ ((counted_by (count)));
| ^~~~~
The builtin __builtin_counted_by_ref is extended to pointers inside structures
The new builtin __builtin_counted_by_ref was added to help the adoption of counted_by attribute for flexible array members in large applications. It’s a natural step to extend this builtin to pointers to dynamically-sized array.
This work has already been committed into the upcoming GCC16. As a result, the allocation macro for a pointer inside a structure could be simply updated with the new __builtin_counted_by_ref in order to use the counted_by attribute for pointers in large applications.
Take the pointer_array in the previous t1.c as an example:
struct pointer_array {
int *buf; /* a pointer to an array with integer elements. */
unsigned count; /* the number of elements of this dynamic array. */
} *p1, *p2, *p3;
Multiple objects of the same structure in the application might need to be initialized, such as the objects pointed by the above pointers p1, p2 and p3.
Introducing an allocation and initialization macro like the following is a common practice to avoid repeating the same steps at each initialization:
/* The allocation and initialization macro for a pointer "P", which points to
a structure, a pointer field "PA", which is a field of the structure,
pointing to a dynamically-allocated array, and an integer "COUNT", which
is the number of element of this dynamically-allocated array. */
#define MY_POINTER_ARRAY_ALLOC(P, PA, COUNT) ({ \
__auto_type __p = &(P); \
__auto_type __c = (COUNT); \
size_t __size_1 = (sizeof (*(*__p))); \
size_t __size_2 = (sizeof (*((*__p)->PA)) * __c); \
if ((*__p = __builtin_malloc (__size_1))) { \
(*__p)->PA = __builtin_malloc (__size_2); \
} \
})
Then each initialization of the objects of the structure pointer_array would be done in different places like:
MY_POINTER_ARRAY_ALLOC(p1, buf, count1); MY_POINTER_ARRAY_ALLOC(p2, buf, count2); MY_POINTER_ARRAY_ALLOC(p3, buf, count3);
When the counted_by attribute is added to the pointer buf as:
struct pointer_array {
int *buf __attribute ((counted_by (count))); /* a pointer to an array with integer elements. */
unsigned count; /* the number of elements of this dynamic array. */
} *p1, *p2, *p3;
Even with the above macro MY_POINTER_ARRAY_ALLOC, in order to correctly set the counted_by field for each object, an initialization of it needs to be added after each allocation:
MY_POINTER_ARRAY_ALLOC(p1, buf, count1); p1->count = count1; MY_POINTER_ARRAY_ALLOC(p2, buf, count2); p2->count = count2; MY_POINTER_ARRAY_ALLOC(p3, buf, count3); p3->count = count3;
This is a very time-consuming, tedious and error-prone work for large applications, such as the Linux kernel. However, with the help of the builtin __builtin_counted_by_ref for pointers, the previous allocation macro MY_POINTER_ARRAY_ALLOC could be updated as:
#define MY_POINTER_ARRAY_ALLOC(P, PA, COUNT) ({ \
__auto_type __p = &(P); \
__auto_type __c = (COUNT); \
size_t __size_1 = (sizeof (*(*__p))); \
size_t __size_2 = (sizeof (*((*__p)->PA)) * __c); \
if ((*__p = __builtin_malloc (__size_1))) { \
(*__p)->PA = __builtin_malloc (__size_2); \
__auto_type ret = __builtin_counted_by_ref((*__p)->PA); \
*_Generic(ret, void *: &(size_t){0}, default: ret) = __c; \
} \
})
With the above updated macro, the additional counter initialization for each allocation is not needed anymore since the counted-by field is accessible within the macro itself through the new builtin __builtin_counted_by_ref.
A detailed explanation of the above macro is similar to what was explained in the previous blog, “Improvements in GCC Buffer Overflow Detection for C Flexible Arrays Members“. Please refer to that blog for more details.
Status of the attribute and builtin on pointers inside structures
The work to extend the counted_by attribute and the builtin __builtin_counted_by_ref to pointers inside structures has been committed to the latest trunk GCC and will be released as GCC16 soon.
At the same time, the Linux kernel has been prepared to use these new extensions. A new __counted_by_ptr macro was recently added to the upcoming Linux v7.0 along with the first use in it. More and more usages of the new __counted_by_ptr macro are expected to be integrated into the Linux kernel or even other applications (see this use of counted_by_ptr in firmware) to protect pointers inside structure from buffer overflow flaws.
Please start to add the counted_by attribute and the builtin __builtin_counted_by_ref in your applications to gain more protection from buffer overflow detection tools.
Next steps
There is still some remaining work to annotate size information for pointers in general.
-
In addition to pointers inside structures, there are other contexts where pointers to dynamically-size array present. The most common cases include:
- Pointers that are returned by function calls;
Such as:void* my_malloc (size_t); void* my_calloc (size_t, size_t);
In the above, both my_malloc and my_calloc return pointers that point to dynamically-allocated memory.
-
Pointers that are passed as function parameters;
Such as:void *my_memcpy (void *dest, const void *src, size_t);
In the above, both the 1st and 2nd parameters of the function my_memcpy are pointers, so annotating their size and passing the size info to buffer overflow detection tools is important too.
In the current GCC, there are some available attributes provided for this purpose. Please see the
access attribute and the alloc_size attribute in the GCC documentation for more information.However, there is a well-know problem with these attributes: the size information gets lost after the function with the attribute is inlined by the compiler. This problem inhibits these attributes from being adopted in real applications.
Please see GCC bug 96503 for more details on this issue.
We should resolve this issue to enable annotation of more pointers with size information.
- Pointers that are returned by function calls;
In order to cover more pointers and flexible array members, we might also need to support more general expressions as argument of the “counted_by” attribute.
Acknowledgments
Many people have provided helpful insight and feedback for this work, without them it could not have been done: Kees Cook, Martin Uecker, Joseph Myers, Bill Wendling, Jakub Jelinek, Yeoul Na, Siddhesh Poyarekar, Richard Biener, and others.