EECE-4029 Operating Systems Fall 2016
Lab 2

processes, mutex, semaphores, memory management, producer-consumer, files, deadlock, more..

Kernel module: kthread and work_queue

Due: Sep 9 (submit instructions: here)

Rationale:
    The kernel kthread and work_queue are needed to offload intensive work so that the kernel can handle interrupts in a timely manner.
 
Lab:
The following kernel module dings the console once a second for 10 seconds. It is implemented with a timer (my_timer) that is hooked by setup_timer to the function my_timer_callback which is invoked when the timer goes off. The time at which the timer goes off is set by mod_timer. The callback function just outputs to the console and reinvokes mod_timer with a new time.

The unit of time in the kernel is the jiffie. The constant HZ is 1 second's worth of jiffies. Use jiffies to determine the current time in jiffies.

For convenience, download timer.c and Makefile, enter the directory in which they wind up (which is usually Downloads), type make at the command prompt, as root type insmod timer.ko to load the module, type rmmod timer to unload it, and type dmesg as user or root (does not matter) to see the results. In all other labs this is the procedure for creating, loading and unloading a module.

#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/timer.h>

static struct timer_list my_timer;
static int count;

void my_timer_callback( unsigned long data ) {
   printk(KERN_EMERG "my_timer_callback called\n");
   if (count++ < 10) mod_timer(&my_timer, jiffies + HZ);
}
		 
int init_module( void ) {
   int ret;
   count = 1;
		 
   printk("Timer module installing\n");
		 
   // my_timer.function, my_timer.data
   setup_timer( &my_timer, my_timer_callback, 0 );
		 
   printk("Starting timer to fire in 1 sec\n");
   ret = mod_timer( &my_timer, jiffies + HZ);
   if (ret) printk("Error in mod_timer\n");
		    
   return 0;
}

void cleanup_module( void ) {
  int ret;

  ret = del_timer( &my_timer );
  if (ret) printk("The timer is a zombie\n");

  printk("Timer module uninstalling\n");

  return;
}
Note that KERN_EMERG results in notification to the console.

Create two kernel modules that perform the same function as the above (i.e., executes printk(KERN_EMERG "Ding") 10 times at most) but using different kernel structures and objects as follows:

  1. Use a single kthread. Support for kthread objects comes via linux/kthread.h. The kernel uses a linked list of runnable tasks to determine which thread should be scheduled to run next. The list contains pointers to objects of type task_struct, each containing information about a particular task. When a kernel thread is created, its information needs to be inserted into the runnable list through a task_struct object. The module creating the thread should also keep a reference to this object and use it to signal the kernel regarding control, for example to stop the thread. Thus, globally define
       struct task_struct *ts;
    
    and create a thread in init_module like this:
       int id = 101;  /* global */
       ...
       int init_module (void) {
          ts = kthread_run(function, (void*)&id, "spawn");
       }
    
    where function is the method the thread is assigned to execute and has the following prototype:
       int function(void *data);
    
    In the above, the address of variable id is passed to data. To use this in a sensible way requires dereferencing, for example, like this:
       int n = *(int*)data;
    
    The function could be looping when the module is unloaded and in that case the thread needs to be terminated. This can can done with the following:
       kthread_should_stop()
    
    which returns 1 when some process invokes kthread_stop(ts) such as what should be done in cleanup_module. However, if the function has already completed, executing kthread_stop(ts) will fail and prevent the module from unloading. To handle that case a flag can be set just before exiting the function and checked in cleanup_module: if set, do not execute kthread_stop, otherwise execute kthread_stop. To elaborate on this, the goal is to make sure the function completes on its own before or when the module is unloaded. There are two cases:
    1. The function reaches return 0, then returns, and then you unload the module.
      In this case DO NOT EXECUTE kthread_stop because there is no thread to stop (it completed) and you will get an error causing a stack trace.
    2. The function does not reach return 0 by the time the module is unloaded.
      In this case kthread_stop is used to signal kthread_should_stop which returns true and the code that tests for this condition should cause return 0 to be reached immediately after that.

    Inside function the following can be used to delay execution for one second:

       set_current_state(TASK_INTERRUPTIBLE);
       schedule_timeout(HZ);
    
    where HZ is the number of "jiffies" in one second (jiffies are the time unit the kernel understands), the role of schedule_timeout is obvious, and the role of set_current_state is to allow schedule_timeout to sleep (that is, interrupt the function).

  2. Use a single work_queue. The following parallels the discussion for the kthread solution. Support for work_queue objects is obtained via linux/workqueue.h. Work queues are lists, as above, except they contain pointers to objects of type delayed_work. Therefore, declare global pointers to a work_queue and a delayed_work object as follows:
       struct workqueue_struct *queue;
       struct delayed_work *dwork;
    
    and create the work queue and delayed work in init_module as follows:
       queue = create_workqueue("queue");
       dwork = (struct delayed_work*)kmalloc(sizeof(struct delayed_work), GFP_KERNEL);
       INIT_DELAYED_WORK((struct delayed_work*)dwork, function);
    
    where kmalloc is used to reserve space for the delayed_work object that dwork points to and INIT_DELAYED_WORK is a macro that initializes the delayed_work object and associates with it some work to do in the form of a method with the following prototype:
       void function(struct work_struct *work);
    
    Use of kmalloc is supported by linux/slab.h.

    When a module is unloaded the work queue and work must be cleaned up carefully - any work still in progress must be stopped and the work queue flushed. The following does this:

       if (dwork && delayed_work_pending(dwork))
          cancel_delayed_work(dwork);
       flush_workqueue(queue);
       destroy_workqueue(queue);
    
    Important: Use of the destroy_workqueue and flush_workqueue functions REQUIRE adding the following to the kernel code:
       MODULE_LICENSE("GPL");
    
    or else you will see an error saying unknown symbols when you load the module!

    Delayed work needs to be enqueued. The following does this:

       queue_delayed_work(queue, dwork, HZ);
    
    where HZ means wait 1 second before executing the work.