This was written in January, 2002

This is a short guide to how and why to make incompatible versions of libraries and development tools install in parallel. Both the problem being solved and the solution are based on practical experience working on GNOME and Red Hat Linux; so hopefully I have more to say than just speculation.

If two packages can be parallel installed, then they have no filenames in common, and people developing against the package always get the version they expected. Also, the packages are designed to coexist nicely if they have any runtime features, such as daemons or configuration files.

The Problem

Let's take a simple example. Say you have a library called Foo, and a collection of software packages in your WayCool Linux Distribution of the Week (WCLDW). Library Foo has two commonly-used incompatible versions 4 and 5. Some of the packages for WCLDW use version 4 upstream, and some use version 5.

Sadly, only one version of Foo can be chosen. So the WCLDW developers face the task of porting half of their packages to the other version. The negative impact of this porting work is:

  • It may be a lot of work for very little user-visible gain.
  • It forks all those packages from upstream, complicating matters for bug reports and patches.
  • If some of the packages depending on Foo are themselves libraries, and they are ported to a new Foo, then the soname on the libraries must be incremented, which diverges the soname from upstream for all eternity (since each upstream soname bump will have to be matched by another distribution soname bump).
  • Upstream divergence results in various distributions being binary-incompatible with one another, defeating the goals of something like the LSB.
  • Porting everything to a new Foo may make WCLDW's new version incompatible with its old versions.

As a result many distributions will adopt ad hoc methods of installing two Foo versions in distribution-specific patches, as a lesser evil than the above list of negatives.

The problem here isn't just about Linux (or other OS) distributions, however. It's also an issue for the maintainers of many single software packages, and for projects made up of many packages (such as GNOME or KDE). Some more issues:

  • As a package maintainer, if I don't know what I'll get as a result of "\#include <foo.h>" or "-lfoo", I get lots of bug reports from confused users as they constantly try to use the wrong Foo.
  • If people manage to compile against the wrong Foo, and I get bug reports, it's a mess if users are testing the bug against a Foo that has different semantics from the Foo I'm using.
  • Distributions tend to create ad-hoc ways to work around the two versions of Foo, which means that Makefiles and configure scripts have to special-case these distributions. It's far better if upstream projects address the problem themselves.
  • For large projects like GNOME, the maintainers of 50+ packages all have to change Foo versions at the same time. Upgrading Foo may require people to uninstall the Foo required by all the packages in their distribution. This results in a lot of reluctance to move from Foo 4 to Foo 5.
  • Users of operating systems or distributions with the wrong Foo installed can't install my software, without uninstalling the OS software that requires the other Foo. So my audience is limited - I have to ask people to give up some of their current software to get my new software.

The Solution

The solution to the problem is essentially to rename Foo, and in most cases the nicest way to do so is to include the version number in the name of every file. This means both versions of Foo can be installed at the same time.

For example, say that Foo traditionally includes these files:

  /usr/include/foo.h
  /usr/lib/libfoo.so
  /usr/share/doc/foo/foo-manual.txt
  /usr/bin/foolizer
You might modify Foo version 4 to include these files instead:
  /usr/include/foo-4/foo.h
  /usr/lib/libfoo-4.so
  /usr/share/doc/foo-4/foo-manual.txt
  /usr/bin/foolizer-4
and Foo version 5 to include:
  /usr/include/foo-5/foo.h
  /usr/lib/libfoo-5.so
  /usr/share/doc/foo-5/foo-manual.txt
  /usr/bin/foolizer-5

Now people simply specify which Foo they want, in the same way they would specify that they want Bar instead of Foo - by using a different library name.

For GNOME, we use pkg-config to make this more convenient for application developers. pkg-config essentially abstracts the library name ("-lfoo-4") and include file location ("-I/usr/include/foo-4"), so app developers don't need to hard code these things into their application. pkg-config is a separate utility with no GNOME dependencies, so anyone can use it and some packages already do.

Note that the version numbers in Foo's filenames are not the full version number of Foo. Presumably Foo 4.0.1, 4.0.2, 4.0.3, etc. will all exist; those will be bugfix releases in the Foo 4 series and all have the same API and ABI. Since they are interchangeable, there is no need to rename anything when they come out. All bugfix-only compatible releases should use the same library name.

Removing barriers to new version adoption

The big benefit of parallel installation is that you remove the reason why people are reluctant to upgrade to a new version of Foo, because upgrading to Foo 5 has no effect on users of Foo 4. This means that the packages in WCLDW can be upgraded by their upstream maintainers, one at a time. It means that GNOME packages can upgrade to Foo 5 one at a time. It means that if I use a text editor dependent on Foo 4, but want my package to use Foo 5, I can do that since I can install both versions of Foo at the same time.

In short, parallel install nukes the chicken and egg problem, and it saves everyone a bunch of time and energy.

(A side effect: suddenly you have far more freedom to break backward compatibility; your new incompatible version is in fact a different library, not the same library. So if you need to clean up a big nest of cruft, no big deal. No one is forced to upgrade until they have time to deal with the breakage.)

Theoretical view

There's a higher-level way to explain why parallel install is the Right Thing. That's simply this: programs that request functionality should get what they intended to get.

Functionality is normally requested by name, whether the name is the name of a function, the name of an executable, or the name of a directory.

Therefore, names should change exactly when the named thing becomes different, that is when it becomes incompatible. If names change more often than that, programs requesting the functionality will fail to get it when they could have. If names change less often, programs requesting the functionality will get the wrong functionality and fail to operate properly.

Details of Implementation

This section covers some common issues that arise when implementing parallel installation.

When to do the rename

You can go ahead and include "4" in the filename of every Foo4 file from the start, or you can wait until Foo5 is released, then release a Foo4 that has the files moved. My opinion is that the former (do Foo4 with versions from the start) is simplest for everyone, but the second approach works if you haven't been planning for parallel install in the past.

What's a "version number" for rename purposes?

The version number that goes in filenames is an "ABI/API" version. It should not be the full version number of your package. For example, GTK+ 1.2.9, 1.2.10, 1.2.11 are all fully API/ABI compatible; so they do not install in parallel. However, GTK+ 1.2.x and 2.0.x are not compatible, so they do install in parallel. Their filenames contain "1.2" and "2.0" respectively, rather than "1.2.10" or "2.0.3".

In short, any "bug fixes only" compatible release will not require moving all the files.

Header files

Header files should always be installed in a versioned subdirectory that requires a -I flag to the C compiler. For example, if my header is foo.h, and applications do this:

  \#include <foo.h>
then I should install these files:
  /usr/include/foo-4/foo.h
  /usr/include/foo-5/foo.h
and applications should pass the flag -I/usr/include/foo-4 or -I/usr/include/foo-5 to the C compiler. Again, this is facilitated by using pkg-config.

Best practice suggests that Foo apps should really be including "foo/foo.h":

  \#include <foo/foo.h>
because this namespaces the \#include, to avoid collisions with other libraries. In that case Foo should install:
  /usr/include/foo-4/foo/foo.h
  /usr/include/foo-5/foo/foo.h

You could also just rename the header itself:

  /usr/include/foo-4.h
But the directory approach is nicer, since app developers only have to change one -I flag rather than all their include statements when they upgrade.

There's some temptation to keep one of the header files outside of any subdirectory:

  /usr/include/foo.h
  /usr/include/foo-5/foo.h
The problem there is that users are always accidentally getting the wrong header, since "-I/usr/include" seems to find its way onto compile command lines with some regularity. If you must do this, at least add a check to the library that detects applications using the wrong header file when the library is initialized.

Libraries

Library object files should have a versioned name. For example:

  /usr/lib/libfoo-4.so
  /usr/lib/libfoo-5.so
This allows applications to get exactly the one they want at compile time, and ensures that Foo 4 and Foo 5 have no files in common.

Why aren't library sonames good enough?

Library sonames only address the problem of runtime linking previously-compiled applications. They don't address the issue of compiling applications that require a previous version, and they don't address anything other than libraries.

Configuration files

From a user standpoint, the best approach to config files is to keep the format both forward and backward compatible (both library versions understand exactly the same config file syntax/semantics).

If you can't do that, the config files should simply be renamed, and users will have to configure each version of the library separately.

gettext translations

If you use gettext for translations in combination with autoconf/automake, normally things are set up to install the translations to /usr/share/locale/<lang>/LC_MESSAGES/<package>. You'll need to change the <package> part. The convention we're using in GNOME is to put this in configure.in:

  GETTEXT_PACKAGE=foo4
  AC_SUBST(GETTEXT_PACKAGE)
  AC_DEFINE_UNQUOTED(GETTEXT_PACKAGE,"\$GETTEXT_PACKAGE")
Then use GETTEXT_PACKAGE as the package name to pass to bindtextdomain(), textdomain(), and/or dgettext().

Other runtime dependencies

If your package involves daemons, IPC, etc., then you may have to come up with more complicated solutions to those issues.

Should this be applied to apps, not just libraries and devel tools?

It's much less important, because upgrading an application does not have implications for upgrading any other packages on the same system. However if your application is used by other applications, as a component perhaps, it may well make sense; since your app is effectively a development tool or API at that point.

Aren't all these versioned files kind of ugly?

Perhaps, but it seems silly to make work for everyone, create compatibility problems between operating systems, and discourage adoption of new versions, purely for aesthetic reasons.

Parallel install is really easy to implement and makes life better for users, for distribution maintainers, for application developers using your library, and even for you as the library maintainer.

It also has a long history as the Right Thing To Do; for example COM programmers will be familiar with the principle that all interfaces must be renamed if they are modified.

Are there any alternative approaches?

The only real alternative approach I know of is to avoid breaking compatibility, ever. This is what the standard C library does, for example.

What about symbol versioning?

Symbol versioning isn't really either/or with the parallel install doctrine I'm advocating here. The doctrine is "if you break backward compatibility, you must rename." Symbol versioning is a way to avoid breaking backward compatibility. Parallel install is a way to minimize the impact of breaking backward compatibility. So if you use symbol versioning you may never need to rename anything.

On a more practical level, symbol versioning isn't a portable solution, which is a showstopper for many projects.

Where can I find some examples?

All of the GNOME 2 platform parallel installs with the GNOME 1 platform. Also, libxml1 and libxml2 are parallel installed in recent versions; libxml is interesting because it's often used on Windows as well. See GNOME and libxml. (If anyone has more examples mail hp@pobox.com)