Improving the treatment of external libraries

Background

Sometimes Cactus code needs to call code in libraries external to Cactus. In order to ensure that these libraries are available to Cactus, each is represented by a Cactus thorn. These thorns perform the following functions:

1. Provide configuration variables for the user to use in their option lists to specify the location of the library on the machine. Sometimes this is as simple as <lib>_DIR = /path/to/lib, but sometimes additional options need to be set, for example if the lib and include directories are in nonstandard locations in relation to the base library path. The external library thorn translates the user's settings for these variables into compiler and linker options which are then used by Cactus when compiling and linking.
2. In the case that the user doesn't specify the location of the library, the thorn attempts to locate the library in some standard locations.
3. As a last resort, or if requested by the user, the thorn will build a version of the library and install it in the configuration directory. For this reason, the thorn usually contains a distribution tarball of the library which is extracted and compiled during the Cactus configuration (CST) stage. Sometimes there is also a patch which is applied to the source of the library. Having the libraries built as part of the configuration ensures that all the libraries needed are always available even if they have not been installed on the machine, but adds a significant overhead to compilation time.

Problems with the existing mechanism

At present, the logic for performing all of the above is implemented in configure.sh scripts in each library thorn. This logic can be quite subtle and involved, and it is desirable for this to be implemented in a central location in a careful and correct way, rather than being duplicated for each library thorn.

Requirements

Collecting together the behaviour required by library thorns, such a mechanism needs to make it straightforward to do the following:

• Provide a variable, <lib>_DIR which either:
  • points to the library installation, in which case the thorn checks that the library in this location is usable and uses it if this is the case, otherwise compilation is aborted;
  • is BUILD (caseless), in which case the thorn builds the library and uses that version;
  • is NO_BUILD (causeless), in which case the thorn searches for a version on the system and uses that, unless it cannot be found, in which case the compilation is aborted;
  • is unset, or is the empty string, in which case the library is first searched for, and if not found, is built (we could have a synonym AUTO (caseless) for this, to make it explicit)
• Provide variables which point to the include and library directories, but by default set these variables to <lib>/include and <lib>/lib.
• Search some standard locations for the library.
• Test that a given setting of the configuration variables works correctly; i.e. that a working version of the library is actually found at a given location

Current configuration scripts search through some "standard" directories such as /usr and /usr/local and then look in standard subdirectories such as "lib" and "lib64" for libraries with standard extensions such as ".a", ".so" and ".dylib" (for Mac OS). This search logic is very low-level, and should probably be replaced by something higher level. For example, we shouldn't make assumptions about the possible library extensions (.so etc), but instead should leave that to the linker. Similarly, many standard installations of libraries will be in locations which the linker searches automatically, so in those cases, if the linker can find the library without assistance, we should not try to second-guess it.

Proposal

Building

The thorn can provide a build.sh script which is responsible for building and installing the library in the configuration directory. Separating this into a separate script makes the system more modular, and separates the logic for whether to build from the code that builds.

Testing

The thorn can provide a source file conftest.cc which, if successfully compiled, linked and run, indicates that the options used to compile and link it are sufficient to find the library. When the user sets <lib>_DIR = /path/to/lib, <lib>_INC_DIRS and/or <lib>_LIB_DIRS, or sets <lib>_DIR = NO_BUILD, these are used to compile and link this program in order to test that this is possible. This testing ensures that a missing library is detected early during the configuration stage, rather than when the first source file tries to include the missing header file.

Searching

When the library location has not been specified, the system needs to search for the library. It will first attempt to compile, link and run the test program with no additional options. This may be successful if the library is installed in a location which is on the standard compiler include/linker paths. If this fails, a sequence of include and library directories can be tried, and the first combination which works can then be used. It may turn out that this is unnecessary, because any "standard" paths that the thorn might know about would probably be tried by the system anyway. A possible exception would be a MacPorts installation in /opt/local which is not searched for by default by the compiler. Note that this is related to the problem that if we add, e.g. /opt/local/lib to the linker path for, say, HDF5, then this might cause something else like MPICH to be found from there, even if the user has specified MPI_DIR to be something else. We don't have a solution to this problem.

Factoring out common code

Each library thorn's configure.sh script can call a bash function (or maybe a script) which we provide in the flesh: configure_library:

 configure_library <lib> <dist> <patch>...

• <lib>: A capitalised name of the library, e.g. HDF5, HWLOC, MPI suitable for use in environment variable names.
• <dist>: A path to the source distribution tarball relative to the thorn, e.g. dist/zlib-1.2.7.tar.gz
• <patch>: A path to any patch that needs to be applied

configure_library does the following:

• Enable verbose output if requested
• Look at the variables set by the user and choose which of the following to do, and do it:
  • Build the library
  • Use the library from a specified location, and test that this works, otherwise abort
  • Search for the library, and use it if found, otherwise abort
  • Search for the library, use if found, otherwise build
• Set Cactus include and link variables so that the library will be appropriately found by Cactus

Questions

• How should the user get the behaviour of setting <lib>_DIR to point to a directory, i.e. explicitly using a specific installation, if there is no such directory, and the include and lib dirs need to be specified separately? Maybe setting any of these three variables (<lib>_DIR, <lib>_INC_DIRS and <lib>_LIB_DIRS) should trigger using exactly that version of the library.