This is a draft document last updated July 10, 2003. Please send comments to Havoc.
One effort has been started to code a solution to this problem.
An interesting library someone pointed out after I wrote this is discover from Progeny. Other related projects include kudzu and of course Linux hotplug and D-BUS.
This document is an attempt to think through the various elements that go in to presenting a nice, user-friendly interface to the hardware of a typical desktop system. No attempt is made to consider issues that apply to "big iron" servers, because I don't understand those issues.
Let's take a simple example of what should happen from 1,000 feet. In the following, "system" means an appropriate conglomerate of kernel and userspace features, it does not imply kernelspace.
Starting from the top down, what should the implementation look like? On the desktop or application level, one reality is that we will have many separate applications that care about hardware. Some of these applications will be trivial applets, others will be standalone apps such as an office suite, others will be desktop components such as the file manager.
Given these many applications, it's critical to be able to add support for new hardware models, and even new kinds of bus (USB, PCI, etc.) without modifying each indidividual application. However, obviously applications will need to understand the type of device - where "type" means something like "camera," "CD-ROM drive," and so on - the user-visible user-interesting type of the device.
Another requirement on the application level is to be cross-platform. Thus at some point in the stack, there will need to be a virtualization with "backends" for various operating systems.
For some time I've been advocating a "hardware abstraction layer." This layer is simply an interface that makes it possible to add support for new devices and new ways of connecting devices to the computer, without modifying every application that uses the device.
My proposal is that the hardware abstraction layer be a shared library API. The API would work in terms of Device objects. On Linux a Device object often maps to a kernel device, but may also map to some entirely userspace devices (printers are typically all userspace, and may even be network devices). "Device" in this API is closer to what a user would consider a device than to any specific implementation concept. The hardware library would maintain a list of devices that currently exist, and could provide those upon request.
A Device object would maintain a set of properties in a key-value mapping. It would be device-dependent which properties existed. Examples of properties:
Not all properties come from the same place.
In short, we create a coherent view of the devices on a system and their properties by merging a set of data sources, including the physical hardware, drivers, subsystems such as CUPS, a bunch of data that comes with the operating system, any data that hardware vendors ship with their hardware, and finally data that users provide manually.
The exact set of information available to be merged depends on what operating system we're using and what kind of device we're talking about.
In pseudocode, the basic API is quite simple:
void list_devices (Device **devices, size_t *n_devices);
char* device_get_property (Device *device, const char *name);
void device_set_property (Device *device, const char *name, const char *value);
It would also be necessary to have complete change monitoring;
not only as the list of devices changes, but also when a device's properties
change.
Some useful properties to provide, again not all would be available for all devices:
The Device abstraction allows applications to ask questions such as "what printers are there?" and allows the desktop to include code such as:
void
new_device_notification_callback (Device *new_device)
{
if (device_type_is_unknown (new_device))
{
type = ask_user_for_device_type (new_device);
device_set_type (new_device, type);
}
}
However, the next problem is how to provide functionality specific to particular types of device. For any camera, the application needs to be able to get the images from the camera. For any printer, the application needs to be able to check what print jobs it has active, and whether there are errors. For any CD-R, the application needs to be able to burn an ISO image.
It seems foolish to pile all this functionality into one giant shared library; instead, we want to build on projects such as CUPS, gphoto, cdrecord, and so forth. The missing piece, though, is a mapping from the Device object to those projects. That is, given a Device that I know is a printer, how do I locate the CUPS print queue that corresponds to that printer? Given a Device that I know is a camera that can appear as a file system, how do I mount it?
Say nautilus wants to get images off a camera; you need an API that has a function like "camera->get_images()." This API probably wouldn't live in the main hardware library. However, you need a way to get that "camera" object given a Device*. So the camera API need not live in hardware lib, but it does need to see the same set of devices seen by the hardware lib and you need a way to map from Device* to whatever a camera is in the camera API. Same holds for CUPS, sound server, whatever. Given that I can list Device* and find two sound cards, how do I specify that one of them is the output device for the sound server.
So what's needed is a way to identify a Device (remember a device often means a kernel device, with major/minor pair) that can then be passed to arbitrary subsystems such as CUPS.
The most obvious solution is to simply pass the Device* from the base hardware library around between more specialized libraries. However, this requires buy-in, e.g. the CUPS libs would need to link to the hardware library.
The other approach is to bail and break the abstraction for this purpose; i.e. allow getting at the underlying kernel device or CUPS data structure, and pass that around. This is probably the most practical thing at first.
An outstanding question is whether applications would use subsystem APIs directly, or whether the single hardware abstraction library would provide APIs for each type of device. For example, is there a Printer object in the base hardware abstraction library that uses CUPS on the backend; or do applications just use CUPS directly, after asking CUPS for the printer that corresponds to a Device*. The answer to this may vary according to the properties of the particular subsystem - is it already a portable abstraction layer, or is it Linux specific, for example.
A complication: Not all uses of a device are device-type-specific. For example, the Device object might have a generic operation to disable the device (presumably this would normally map to unloading its driver). Thus, the base hardware abstraction library might support these operations.
Here are the major areas of implementation that I see.
Given some raw hardware information about a device, such as its PCI or USB ID, load the right driver for the device. This involves maintaining a database mapping hardware ID information to kernel driver names and parameters.
It must be possible for an OEM to ship an addition to the database along with a product. It must also be possible for local sites to add their own overrides or provide missing mappings.
There should be a hook from this layer to the desktop, where the desktop is provided with a) whatever description of the device can be scrounged up, if only "the device you just plugged in," and b) a list of possible drivers. The desktop then asks the user to pick one of the drivers, and invokes a routine to record the choice for future reference and then load the driver. This hook would probably be in the form of a few simple functions in the hardware abstraction library.
Once a driver is loaded, we should have zero or more new kernel devices. This means that the list of Device* provided from the hardware abstraction library would be updated. So the hardware abstraction library needs some way, from inside an application, to monitor additions and deletions from the list of loaded devices. This should not be done via polling, but rather via notification.
In an ideal world, /dev files are completely hidden from applications. Using them always involves knowledge of specific kinds of hardware, and thus breaks the abstraction barrier that allows us to add new device support without changing all the apps. /dev file naming may also vary by platform, and even by local site.
Currently, /dev naming is sometimes used to specify preferences such as "the default audio device" or "the default CD-ROM" (/dev/audio, /dev/mouse, etc.). Using /dev as an implementation detail for this is fine, but applications should have a way to get the default audio Device*, then play sound to the default audio Device*, without ever hardcoding the /dev/audio path in the app. Hardcoding /dev/audio means that the application must make platform-specific and devicetype/bustype-specific assumptions. These assumptions are ideally buried in a library, for all but the most specialized applications.
Anyhow, from an application standpoint, /dev naming is something that should magically go on behind the scenes of the hardware abstraction library. But it still has to be implemented somehow.
This is the primary task of the hardware abstraction library. It has several components:
This is the secondary task of the hardware abstraction library. Given a Device known to be a printer or a camera or whatever, the library must chain to CUPS, gphoto, cdrecord, or whatever is appropriate, allowing the application to use the device.
In some cases, the hardware abstraction library may offer its own APIs to use a type of device. This would be especially useful when the existing subsystem is low-level/crufty, hard to use properly, or unportable.
In other cases, the subsystem may be quite complex, something like GStreamer or CUPS; wrapping these APIs probably doesn't make sense. But there still has to be a bridge between the hardware abstraction layer and the subsystem, so they work together in a coherent way, and above all agree on the list of available devices.
Suggestions:
I don't have much opinion on this otherwise.
As long as this is suitably buried beneath the hardware abstraction library, it's fairly irrelevant to desktop application developers how it works. Anything that works is great.
The hardware abstraction library should use a "model-view" architecture, where the Device object and list of Device objects make up the model, and the application implements some view of it.
It seems appropriate to use D-BUS to notify the hardware abstraction library of changes to the list of kernel devices, changes to the list of CUPS printers, etc.
Each Device object would have a backend that was either a kernel device (major/minor pair), a CUPS printer, or some other kind of device representation. Information from this backend would then be integrated with override or supplementary information from a system database, and the result would be exposed as the public API of the Device object.
The implementation here is very specific to each type of device.
The core implementation idea presented in this document is to maintain a list of device objects, where each device object has a set of properties (key-value pairs).
Blacklists/whitelists of any kind simply become additional properties to be merged, or perhaps simple rewrite rules that modify a property in some way. Something like:
if DeviceModel matches blahblah then set BrokenFeatureFoo to true
This is extensible to support any kind of device, device feature, blacklist/whitelist, user configuration, translations, or whatever; we just store arbitrary data associated with any device, more or less.