What are Jiffies?

The interval between two system timer interrupt ticks is known as a jiffy in the Linux kernel. Think of them as the heartbeat of the kernel, marking regular intervals at which the kernel performs essential timekeeping tasks.

The jiffies hold the number of ticks that have occurred since the system booted. The variable is initialized by the kernel to zero at boot, and it is increased by one at each timer interrupt. Each increment is called a tick. The frequency of these timer interrupts is denoted by HZ. Thus, just as there are HZ jiffies in a second, there are also HZ timer interrupts in a second. This blog is based on the UEK6U3 kernel v5.4.17-2136.322.6.2, however is also applicable to UEK7.

Representation of Jiffies

The notion of jiffies is integrated into the Linux kernel via the jiffies variable, associated functions, and macros offered by the kernel’s timekeeping subsystem.
The jiffies variable is declared in <linux/jiffies.h> as :

extern unsigned long volatile jiffies;
extern u64 jiffies_64;

Computer architecture has changed considerably over time, the Linux kenel has evolved with these changes, and so have Jiffies. The timer tick increments jiffies and this poses a challenge on a 32-bit systems. Where HZ = 1,000, this will lead to an jiffies overflow in about 49 days, where as on 64-bit architecture, the duration is about 600 million years.

To address this issue, a second variable, jiffies_64, was introduced. Depending on the processor type, only one of these variables is used in the Linux kernel. For x86_64, the u64 data type is used, and for x86, an unsigned long is used. We see this by looking at the arch/x86/kernel/vmlinux.lds.S linker script: jiffies = jiffies_64;

In the case of x86_32, the jiffies will be the lower 32 bits of the jiffies_64 variable. Jiffies are integral to various aspects of kernel operations, including scheduling tasks, maintaining time-related data structures, and implementing time-sensitive functionalities.   ## How jiffies are incremented? Each time the timer interrupt occurs, it triggers the execution of the tick_periodic() function. This function, in turn, invokes do_timer(), which is responsible for updating the jiffies variable and subsequently updating the wall time.

Let’s look at the source code:

static void tick_periodic() {
    /*acquire the required locks*/
    ...
    do_timer(1);
    ...
    /*releases the lock*/
    update_wall_time();
}

void do_timer(unsigned long ticks) {
  jiffies_64 += ticks;
  calc_global_load();
}

do_timer() increments the jiffies_64 by the number of ticks passed which is 1 for each tick as seen above.

Working with Jiffies

Accessing the current time in jiffies is straightforward. Kernel code can simply read the value of the jiffies global variable by its name or with the call to the get_jiffies_64() function. This function returns the full 64-bit value of the jiffies.

Now that we can access the jiffies or the jiffies_64 variable, conversion to human-readable time units becomes feasible. To retrieve time in seconds, the following expression can be used: jiffies / HZ.

Now that we are acquainted with these variables, we can determine a value for any time in the future as needed. For example:

unsigned long time_stamp = jiffies;            /* now */
unsigned long next_tick = jiffies + 1;         /* one tick from now */
unsigned long later = jiffies + HZ;            /* One second from now */
unsigned long later = jiffies + 10*HZ;         /* ten seconds from now */

The kernel also provides additional helper functions like msecs_to_jiffies() and usecs_to_jiffies() to facilitate the conversion of desired milliseconds or microseconds to jiffies when needed and jiffies_to_msecs() and jiffies_to_usecs() to facilitate conversion of jiffies to milliseconds or microseconds when needed.

unsigned long current_time = jiffies;
unsigned long current_time_in_ms = jiffies_to_msecs(jiffies);
unsigned long current_time_in_us = jiffies_to_usecs(jiffies);

unsigned long time_stamp = jiffies;                              /* now */
unsigned long later = jiffies + msecs_to_jiffies(10);            /* ten milliseconds from now */
usnsigned long later = jiffies + usecs_to_jiffies(50);            /* 50 microseconds from now */

Use of jiffies for timekeeping:

The following is an example of timekeeping using jiffies:

#include <linux/jiffies.h>
#include <linux/delay.h>

unsigned long start_time = jiffies;
unsigned long timeout = jiffies+10*HZ;       /* ten seconds from now to finish the task*/

/* Perform some operations */

unsigned long end_time = jiffies;
unsigned long elapsed_time = end_time - start_time;


if(elapsed_time<=timeout){
      /*----we did not time out, good ...----*/
}else{
      /* we timed out, error ... */
}

Can you see the problem with the above code?

  • If jiffies is wrapped back to zero after setting a timeout, it could lead to an unexpected behavior. In such a scenario, the first conditional check might fail because the jiffies value would appear smaller than the timeout, despite logically being larger. Because it overflowed its maximum value, it would now be a very small number, perhaps only a few ticks over zero. Consequently, due to the wraparound, the outcomes of the if statement would be reversed.

So what can be done to avoid wraparound?

  • To handle wraparound in the tick count effectively, the kernel offers four macros for tick count comparison defined in <include/linux/jiffies.h>:
#define time_after(a,b)
#define time_before(a,b)
#define time_after_eq(a,b)
#define time_before_eq(a,b)

a is an unknown parameter and typically represents current jiffies while b is a known parameter, the value we wish to compare against. The time_after(a,b) returns true if the time a is after time b; otherwise, it returns false. Similarly, time_before(a, b) returns true if time a is before time b; otherwise, it returns false. The final two macros function in the same way as the previous two, except they also return true if the parameters are equal.

Here’s the timer-wraparound-safe version of the previous example using the provided macros:

unsigned long start_time = jiffies;
unsigned long timeout = jiffies+10*HZ;       /* ten seconds from now to finish the task*/

/* Perform some operations */

unsigned long end_time = jiffies;
unsigned long elapsed_time = end_time - start_time;


if(time_before_eq(elapsed_time,timeout)){    /*these macro will check whether elapsed_time is before or equal to timeout*/
      /*----we did not time out, good ...----*/
}else{
      /* we timed out, error ... */
}

This functionality is frequently utilized in the Linux kernel. For instance, examining the arch/x86/kernel/smpboot.c source code reveals the do_boot_cpu function. This function initializes all processors except the bootstrap processor. We can find a snippet that waits ten seconds for a response from the application processor:

if (!boot_error) {
    timeout = jiffies + 10*HZ;
    while(true){
    ...
    ...
    if (time_after_eq(jiffies, timeout))
        break;
    }
    ...
}

Clocksource structure for jiffies

Let’s look at the clocksource structure for jiffies as defined in the file kernel/time/jiffies.c. As kernel documentation mentions the “The Jiffies based clocksource is the lowest common denominator clock source which should function on all systems”. Before understanding the meaning of that let’s get acquainted with what is Clocksource and why it is used.

As we know the Linux kernel is designed to run on a wide range of hardware architectures, from desktop computers to embedded devices and servers, and each architecture may have different types of hardware timers available, such as Programmable Interval Timers (PITs), High Precision Event Timers (HPETs), or platform-specific clock devices. When the kernel boots, it needs to choose the most appropriate clocksource for the underlying hardware.

Solving the above problem clocksource abstracts away the hardware-specific details and provides a consistent interface for the kernel to query timing information and choose the most appropriate clocksource.

Now let’s finally look at the clocksource structure for jiffies:

static struct clocksource clocksource_jiffies = {
    .name           = "jiffies",
    .rating         = 1,
    .read           = jiffies_read,
    .mult           = TICK_NSEC << JIFFIES_SHIFT,
    .shift          = JIFFIES_SHIFT,
    ...
    ...
};

static u64 jiffies_read(struct clocksource *cs){
    return (u64) jiffies;
}

static inline s64 clocksource_cyc2ns(u64 cycles, u32 mult, u32 shift){
    return ((u64) cycles * mult) >> shift;
}

Let’s break down each field of this structure:

  • name: the name of the clocksource, which in this case is “jiffies”. The name is used to identify the clocksource within the kernel.

  • rating: The rating indicates the suitability or quality of the clocksource relative to others. A higher rating value typically indicates a more accurate and reliable clocksource and this is how the kernel chooses the most appropriate clocksource at bootup. For example, the rating of the time stamp counter is 300, and the rating of the high precision event timer is 250.

  • read: The read is a pointer to a function responsible for reading the current clocksource’s cycle value. In this case, it just returns the jiffies variable.

  • mult and shift: mult and shift are used to convert the clocksource’s period to nanoseconds, these fields are used by the clocksource_cyc2ns function.

What is Loops Per Jiffy(LPJ)?

Now that we understand what jiffies are and how we can use them in timekeeping. Let’s delve into the concept of LPJ in the Linux kernel. Loops per jiffy quantifies the rate at which a CPU can execute loop iterations relative to the kernel’s timer frequency. It represents the number of tight loop iterations a CPU can complete within the duration of one kernel jiffy. For instance, if a CPU manages 5 million loop iterations in one kernel jiffy, its loops per jiffy value is 5 million. LPJ is essential for tweaking timing functions precisely and maintaining accurate system timing. It is defined as an unsigned long variable:

unsigned long loops_per_jiffy;

Now, let’s look into how the Linux kernel calibrates LPJ to ensure accurate timekeeping.

How is Loops per Jiffy Calibrated

Before diving into calibrating loops_per_jiffy let’s see what the __delay(loops) function does. It takes a parameter, loops, which represents the number of loop iterations to perform. The function uses inline assembly to perform a busy-wait loop. It decrements the loop counter (loops) until it reaches zero, effectively introducing a delay.

Due to variations in hardware and system configurations, the duration of a delay loop may differ across different systems. Calibration ensures that these delays are accurately tuned, enabling consistent timing behavior across different architectures.

The main function responsible for calibrating loops per jiffy in x86 architecture is a function named calibrate_delay_converge() that returns an unsigned long integer which is the loops_per_jiffy value. Let’s understand how this function works with the below snippet from the function:

trials = 0, band = 0, trial_in_band = 0;
lpj = (1<<12);
ticks = jiffies;
while (ticks == jiffies)
    ; /* nothing */
/* Go .. */
ticks = jiffies;
do {
    if (++trial_in_band == (1<<band)) {
        ++band;
        trial_in_band = 0;
    }
    __delay(lpj * band);
    trials += band;
} while (ticks == jiffies);

The void while loop inside is to let go of the current jiffy – that is, to start at the beginning of the next jiffy tick as we don’t know where in the current jiffy window we are, so if we start __delay() now, then our loops_per_jiffy calculation is skewed.

The core of the calibration process lies within this do-while loop where the delay is gradually increased until another clock tick occurs. The delay is incremented by a factor determined by the band variable, which doubles each time trial_in_band reaches 1<<band. The trials variable keeps track of the total number of delay loop iterations.

When we come out of the loop, the following condition holds that lpj*trials causes __delay() to cross a jiffy and lpj*(trial-band) does not. So, the exact LPJ is somewhere between these two values. therefore, starting from here we have to refine LPJ further. Let’s look at the code snippet for the refining:

trials -= band;
loopadd_base = lpj * band;
lpj_base = lpj * trials; 

lpj = lpj_base;
loopadd = loopadd_base;
    
while (loopadd > chop_limit) {
    lpj += loopadd;
    ticks = jiffies;
    while (ticks == jiffies)
        ; /* nothing */
    ticks = jiffies;
    __delay(lpj);
    if (jiffies != ticks)   /* longer than 1 tick */
        lpj -= loopadd;
    loopadd >>= 1;
}

After the initial estimate for LPJ, the function compensates for overshooting the target delay. A binary search algorithm is employed to fine-tune the delay until it approximates one clock tick. We take the lpj value to lpj*(trials-band) and the variable loopadd to be the last incremented value before crossing the previous jiffy that is lpj*band.

Now on each iteration lpj is incremented by loopadd and if the delay is longer than one tick, lpj is reset, and loopadd is halved for finer adjustments until it reaches the limit.

Finally, after all this hard work the lpj value is calibrated and returned.

How udelay and ndelay uses LPJ

The functions udelay and ndelay are used to introduce small delays in code execution. They are primarily utilized for timing purposes, especially in device driver programming or situations where precise timing is required without sleeping (or blocking).

Let’s look at the logic behind obtaining the microseconds or nanoseconds delay:

loops_per_jiffy = x;
loops_per_sec = x * HZ    /*  there are HZ jiffies in a sec  */
loops_per_usec = x * HZ * (1/1000000)
loops_per_n_usec = n * loops_per_usec

Can you see the problem with the above calculation ?

  • When employing integer arithmetic, the value of loops_per_usec becomes zero due to the (1/1000000) component.

What is the solution to the above issue?

  • Let’s make it non-zero, so loops_per_usec is multiplied with 2^32. This operation shifts the entire product left by 32 bits, and then we consider only the most significant 32 bits.

  • For usecs 2^32 / 1000000 rounded up is 0x000010c7 (4295) and for nsecs 2^32 / 1000000000 rounded up is 0x00005, these will come handy for further calculations.

udelay and ndelay are just user-facing functions, they call __udelay() and __ndelay() respectively for introducing delay in execution.

Now let’s look directly at __udelay(usecs). It takes a parameter usecs, which represents the desired delay in microseconds, and makes a call to another function __const_udelay(xloops) passing usecs*0x000010c7 as an argument. Now __const_udelay() will obtain the value of loops_per_jiffy and call __delay() passing loops_per_jiffy*xloops as an argument and we already discussed that __delay() will busy-wait for the number of loops passed.

A similar procedure is followed by __ndelay as well.

Conclusion

This blog discusses jiffies, the clocksource structure for jiffies and how we can use them for timekeeping, and the concept of loops_per_jiffy, how it is calibrated, and how mdelay, udelay, and ndelay uses the loops_per_jiffy value to introduce a delay in code execution.

References