Back to libsm overview

# libsm : Memory Allocation

$Id: heap.html,v 1.1.1.1 2002/03/12 18:00:20 zarzycki Exp $

## Introduction

The heap package provides a layer of abstraction on top of `malloc`, `realloc` and `free` that provides optional error checking and memory leak detection, and which optionally raises an exception when an allocation request cannot be satisfied.

## Synopsis

    #include <sm/heap.h>

    /*
    **  Wrappers for malloc, realloc, free
    */
    void *sm_malloc(size_t size);
    void *sm_realloc(void *ptr, size_t size);
    void  sm_free(void *ptr);

    /*
    **  Wrappers for malloc, realloc that raise an exception instead of
    **  returning NULL on heap exhaustion.
    */
    void *sm_malloc_x(size_t size);
    void *sm_realloc_x(void *ptr, size_t size);

    /*
    **  Print a list of currently allocated blocks,
    **  used to diagnose memory leaks.
    */
    void  sm_heap_report(FILE *stream, int verbosity);

    /*
    **  Low level interfaces.
    */
    int sm_heap_group();
    int sm_heap_setgroup(int g);
    int sm_heap_newgroup();
    void *sm_malloc_tagged(size_t size, char *file, int line, int group);
    void *sm_malloc_tagged_x(size_t size, char *file, int line, int group);
    bool  sm_heap_register(void *ptr, size_t size, char *file, int line);

## How to allocate and free memory

`sm_malloc`, `sm_realloc` and `sm_free` are portable plug in replacements for `malloc`, `realloc` and `free` that provide error checking and memory leak detection. `sm_malloc_x` and `sm_realloc_x` are variants of `sm_malloc` and `sm_realloc` that raise an exception on error. To use the package effectively, all calls to `malloc`, `realloc` and `free` should be replaced by calls to the corresponding `sm_`* routines.

`void *sm_malloc(size_t size)`
  This function is a plug-in replacement for `malloc`. It allocates `size` bytes of memory on the heap and returns a pointer to it, or it returns `NULL` on failure.

  The C standard says that `malloc(0)` may return either `NULL` or a non-`NULL` value. To ensure consistent behaviour on all platforms, `sm_malloc(0)` is equivalent to `sm_malloc(1)`.

  In addition, if heap checking is enabled, then `sm_malloc` maintains a hash table describing all currently allocated memory blocks. This table is used for argument validity checking in `sm_realloc` and `sm_free`, and it can be printed using `sm_heap_report` as an aid to finding memory leaks.

`void *sm_malloc_x(size_t size)`
  This function is just like `sm_malloc` except that it raises the `SmHeapOutOfMemory` exception instead of returning `NULL` on error.

`void *sm_realloc(void *ptr, size_t size)`
  This function is a plug-in replacement for `realloc`. If `ptr` is null then this call is equivalent to `sm_malloc(size)`. Otherwise, the size of the object pointed to by `ptr` is changed to `size` bytes, and a pointer to the (possibly moved) object is returned. If the space cannot be allocated, then the object pointed to by `ptr` is unchanged and `NULL` is returned.

  If `size` is 0 then we pretend that `size` is 1. This may be a mistake.

  If ptr is not NULL and heap checking is enabled, then ptr is required to be a value that was previously returned by sm_malloc or sm_realloc, and which has not yet been freed by sm_free. If this condition is not met, then the program is aborted using sm_abort.

`void *sm_realloc_x(void *ptr, size_t size)`
  This function is just like `sm_realloc` except that it raises the SmHeapOutOfMemory exception instead of returning `NULL` on error.

`void sm_free(void *ptr)`
  This function is a plug-in replacement for free. If heap checking is disabled, then this function is equivalent to a call to free. Otherwise, the following additional semantics apply.

  If ptr is NULL, this function has no effect.

  Otherwise, ptr is required to be a value that was previously returned by sm_malloc or sm_realloc, and which has not yet been freed by sm_free. If this condition is not met, then the program is aborted using sm_abort.

  Otherwise, if there is no error, then the block pointed to by ptr will be set to all zeros before free() is called. This is intended to assist in detecting the use of dangling pointers.

## How to control tag information

When heap checking is enabled, the heap package maintains a hash table which associates the following values with each currently allocated block:

`size_t size`
  The size of the block.
`char *tag`
  By default, this is the name of the source file from which the block was allocated, but you can specify an arbitrary string pointer, or `NULL`.
`int num`
  By default, this is the line number from which the block was allocated.
`int group`
  By convention, group==0 indicates that the block is permanently allocated and will never be freed. The meanings of other group numbers are defined by the application developer. Unless you take special action, all blocks allocated by `sm_malloc` and `sm_malloc_x` will be assigned to group 1.

These tag values are printed by `sm_heap_report`, and are used to help analyze memory allocation behaviour and to find memory leaks. The following functions give you precise control over the tag values associated with each allocated block.

`void *sm_malloc_tagged(size_t size, int tag, int num, int group)`
  Just like `sm_malloc`, except you directly specify all of the tag values. If heap checking is disabled at compile time, then a call to `sm_malloc_tagged` is macro expanded to a call to `malloc`.

  Note that the expression `sm_malloc(size)` is macro expanded to

  >     sm_malloc_tagged(size, __FILE__, __LINE__, sm_heap_group())

`void *sm_malloc_tagged_x(size_t size, int tag, int num, int group)`
  A variant of `sm_malloc_tagged` that raises an exception on error. A call to `sm_malloc_x` is macro expanded to a call to `sm_malloc_tagged_x`.

`int sm_heap_group()`
  The heap package maintains a thread-local variable containing the current group number. This is the group that `sm_malloc` and `sm_malloc_x` will assign a newly allocated block to. The initial value of this variable is 1. The current value of this variable is returned by `sm_heap_group()`.

`int sm_heap_setgroup(int g)`
  Set the current group to the specified value.

Here are two examples of how you might use these interfaces.

1. One way to detect memory leaks is to turn on heap checking and call `sm_heap_report(stdout,2)` when the program exits. This prints a list of all allocated blocks that do not belong to group 0. (Blocks in group 0 are assumed to be permanently allocated, and so their existence at program exit does not indicate a leak.) If you want to allocate a block and assign it to group 0, you have two choices:

   >     int g = sm_heap_group();
   >     sm_heap_setgroup(0);
   >     p = sm_malloc_x(size);
   >     sm_heap_setgroup(g);

   or

   >     p = sm_malloc_tagged_x(size, __FILE__, __LINE__, 0);

2. Suppose you have a utility function foo_alloc which allocates and initializes a 'foo' object. When sm_heap_report is called, all unfreed 'foo' objects will be reported to have the same source code file name and line number. That might make it difficult to determine where a memory leak is.

   Here is how you can arrange for more precise reporting for unfreed foo objects:

   >     #include <sm/heap.h>

   >     #if SM_HEAP_CHECK
   >     #  define foo_alloc_x() foo_alloc_tagged_x(__FILE__,__LINE)
   >        FOO *foo_alloc_tagged_x(char *, int);
   >     #else
   >        FOO *foo_alloc_x(void);
   >     #  define foo_alloc_tagged_x(file,line) foo_alloc_x()
   >     #endif

   >     ...

   >     #if SM_HEAP_CHECK
   >     FOO *
   >     foo_alloc_tagged_x(char *file, int line)
   >     #else
   >     FOO *
   >     foo_alloc_x(void)
   >     #endif
   >     {
   >     	FOO *p;

   >     	p = sm_malloc_tagged_x(sizeof(FOO), file, line, sm_heap_group());
   >     	...
   >     	return p;
   >     }

## How to dump the block list

To perform memory leak detection, you need to arrange for your program to call sm_heap_report at appropriate times.

`void sm_heap_report(FILE *stream, int verbosity)`
  If heap checking is disabled, this function does nothing. If verbosity <= 0, this function does nothing.

  If verbosity >= 1, then sm_heap_report prints a single line to stream giving the total number of bytes currently allocated. If you call sm_heap_report each time the program has reached a "ground state", and the reported amount of heap storage is monotonically increasing, that indicates a leak.

  If verbosity >= 2, then sm_heap_report additionally prints one line for each block of memory currently allocated, providing that the group != 0. (Such blocks are assumed to be permanently allocated storage, and are not reported to cut down the level of noise.)

  If verbosity >= 3, then sm_heap_report prints one line for each allocated block, regardless of the group.

## How to enable heap checking

The overhead of using the package can be made as small as you want. You have three options:

1. If you compile your software with -DSM_HEAP_CHECK=0 then sm_malloc, sm_realloc and sm_free will be redefined as macros that call malloc, realloc, and free. In this case, there is zero overhead.
2. If you do not define -DSM_HEAP_CHECK=0, and you do not explicitly turn on heap checking at run time, then your program will run without error checking and memory leak detection, and the additional cost of calling sm_malloc, sm_realloc and sm_free is a function call and test. That overhead is sufficiently low that the checking code can be left compiled in a production environment.
3. If you do not define -DSM_HEAP_CHECK=0, and you explicitly turn on heap checking at run time, then the additional cost of calling sm_malloc, sm_realloc and sm_free is a hash table lookup.

Here's how to modify your application to use the heap package. First, change all calls to malloc, realloc and free to sm_malloc, sm_realloc and sm_free. Make sure that there is a -d command line option that uses the libsm debug package to enable named debug options. Add the following code to your program just before it calls exit, or register an atexit handler function containing the following code:

>     #if SM_HEAP_CHECK
>     	/* dump the heap, if we are checking for memory leaks */
>     	if (sm_debug_active(&SmHeapCheck, 2))
>     		sm_heap_report(stdout, sm_debug_level(&SmHeapCheck) - 1);
>     #endif

To turn on heap checking, use the command line option "-dsm_check_heap.1". This will cause a table of all currently allocated blocks to be maintained. The table is used by sm_realloc and sm_free to perform validity checking on the first argument.

The command line option "-dsm_check_heap.2" will cause your application to invoke sm_heap_report with verbosity=1 just before exit. That will print a single line reporting total storage allocation.

The command line option "-dsm_check_heap.3" will cause your application to invoke sm_heap_report with verbosity=2 just before exit. This will print a list of all leaked blocks.

The command line option "-dsm_check_heap.4" will cause your application to invoke sm_heap_report with verbosity=3 just before exit. This will print a list of all allocated blocks.

## Using sm_heap_register

Suppose you call a library routine foo that allocates a block of storage for you using malloc, and expects you to free the block later using free. Because the storage was not allocated using sm_malloc, you will normally get an abort if you try to pass the pointer to sm_free. The way to fix this problem is to 'register' the pointer returned by foo with the heap package, by calling sm_heap_register:

>     bool sm_heap_register(ptr, size, file, line, group)

The 'ptr' argument is the pointer returned by foo. The 'size' argument can be smaller than the actual size of the allocated block, but it must not be larger. The file and line arguments indicate at which line of source code the block was allocated, and is printed by sm_heap_report. For group, you probably want to pass sm_heap_group().

This function returns `true` on success, or `false` if it failed due to heap exhaustion.