A portfolio of technology projects I’ve been involved with in some capacity. I’m trying to include some that didn’t turn out so well too, because they were mostly interesting (or educational) in some way.
As of 2016, I’m working for Continuum Analytics, the Python data science company. So far I’ve worked on Bokeh, anaconda-project, and the Anaconda Platform flagship enterprise product.
anaconda-project tries to provide reproducibility and automation around Python data science projects, by going a little higher-level than plain conda. It provides a “project” abstraction which can be shared among data scientists and deployed to web hosting, attempting to load dependencies and run code in the same way on all machines. Continuum’s Anaconda Platform is built with the Project abstraction as its core.
The codebase is notably robust; 100% test coverage, 0 linter errors, lots of error handling.
Bokeh allows Python developers to create interactive data visualizations all in Python (without writing JavaScript/HTML/CSS by hand). It’s a sort of special-purpose web application server accompanied by a dataviz-specialized UI toolkit.
I reimplemented the Bokeh server for the 0.11.0 release, with tons of improvements to quality, architecture, and UX.
I also developed a neat layout API for Bokeh.
I worked for a while at Typesafe (now called Lightbend), an open source company around Scala, Akka, Play, and Spark. Together these are a pretty comprehensive technology stack on the Java VM, designed for modern application development (parallel, functional, message-based, and all that good stuff).
At Typesafe I had a series of different “odd job” roles, including both development and management.
Sounds unexciting! But it is kind of a cool project.
This configuration file library replaced two ad hoc made-up config file formats used in older versions of Akka and Play. I designed a superset of JSON which is also a 90%-if-you-squint superset of the old ad hoc formats and of Java properties format. HOCON also responded to a wishlist of ideas from colleagues. Despite this, it isn’t outrageously complicated… the first shipping version, with pretty complete test coverage, spec, and docs, was done in a few weeks.The library has seen a fair amount of adoption beyond Akka and Play, and gets a lot of positive comments from users.
Typesafe faced a “how do I get started?” problem, with lots of different technologies (Scala, Akka, Play, plus some smaller ones), each with their own websites and downloadable packages. Documentation was piecemeal. sbt, commonly used by Scala developers, had some neat capabilities but also a considerable learning curve.
As part of a small team, we developed Activator, which includes a server-side “template catalog” full of tutorials and sample code, a single unified download containing the Typesafe tech stack, and a friendly UI to help people try out templates and sample apps. This project required building a scalable web service with Play, Redis, and Heroku; extensive modifications to sbt internals; and a good-sized single-page JavaScript application.
The Scala world uses the sbt build tool. We refactored it from a command line interactive prompt to a client-server arrangement, with the command line prompt as one client. This allows IDEs (such as IntelliJ) and UIs such as the one in Typesafe Activator to share the same build logic, powered by sbt on the backend.
As technical director for litl OS, I hired a team and shipped a brand new high-design OS on brand new hardware (the “litl webbook”) in less than two years from a standing start.
This was essentially the same technical challenge as ChromeOS and Chromebook, except that we weren’t Google - we were a small startup team. litl OS predated both ChromeOS and the iPad by a little bit (though it ultimately couldn’t compete with them).
In litl OS we emphasized design a lot more than ChromeOS does, and implemented some pretty cool ideas that have since appeared in other products. Unfortunately the product didn’t work out overall, but we did an awful lot with what we had.
The UI for litl OS was built in JavaScript, but using native tech rather than HTML (this made a lot more sense back then than it would today). To do this, I created a standalone JavaScript system called gjs, which is technically pretty similar in spirit to node.js. Node didn’t exist at the time.
gjs outlived litl, though, and was used to build GNOME Shell which is the heart of the GNOME 3 desktop.
I was actively involved with the Linux desktop environment, GNOME, from roughly 1998-2007. I still keep track of it (and use it daily). I was chair of the GNOME Foundation board for a couple of years, and personally implemented large parts of the desktop. Other people have been steadily replacing my code though!
I was at Red Hat from 1999-2008, mostly working on the UI in various ways, but also pretty involved in other parts of the company including the discovery of the RHEL business model (many still don’t understand why this worked for Red Hat or how it was done, believing things like “open source companies sell support”).
I originally joined a “research” team called Red Hat Advanced Development Labs, which was formed to build a working web browser for Linux, then switched to GNOME after Mozilla went open source.
After the dot-com bubble burst, RHAD Labs became the Red Hat desktop team. At one point I almost was the entire desktop team, owning almost all the desktop-related packages in Red Hat Linux. Thankfully that didn’t last too long.
Over time we were the leaders in transforming the Linux UI from fvwm2+xterm, into a reasonable modern user experience and API, with far fewer people than Microsoft, Apple, or Google.
I managed the team for a while and was asked to grow the team from I think about 6, to 25, in a 6-month period. I had a spreadsheet of 100+ candidates, ran everyone through phone screens and interviews, and we ended up with a very strong group.
Lots of other things happened at Red Hat in 9 years, of course.
I was frequently the “technical guy” on special projects, such as due diligence for acquisitions or handling intellectual property issues.
At one point in 2005 or so, some of us at Red Hat moved to a separate building “pirate style” and built an entirely unrelated-to-Red-Hat consumer web app called Mugshot, which was a Facebook-style news feed built from lots of other social media sites popular at the time.
Facebook didn’t have a news feed then; it was still about the “wall,” if you remember that. About the same time we launched Mugshot publicly, Facebook also launched their news feed, causing a huge uproar among Facebook users who hated the news feed.
This was my first serious journey into development on the Java virtual machine, using JBoss, ant, Java 5, JSP, and all that other old-school tech. I then left the Java world for several years only to return to Typesafe.
I designed and implemented the Metacity window manager, which was the default window manager for the GNOME 2.x desktop environment. A fair amount of its code lived on into GNOME 3.x as well, refactored into Mutter.
Metacity was an opinionated synthesis of user feedback on earlier window managers; its most important “innovation” was that, as with equivalent Mac and Windows functionality, it wasn’t a user-visible entity. It had no web site, no logo, no separate configuration dialog, was well-integrated into the overall OS, and generally faded into the background as an OS component. That may sound like common sense, but at the time (we’re talking old-school Linux here) it was a big deal.
Another “feature” of Metacity: it was pretty rock-solid with few bugs compared to the other window managers at the time. It may or may not be a coincidence that I was in charge of fixing bugs in the Red Hat Linux window manager at the time that I replaced it with Metacity.
I wrote the GNOME terminal application, which is simply a “shell” (menu bar, preferences dialog) around a terminal emulator widget. I didn’t write the terminal emulator, that was by emulation mastermind Nalin Dahyabhai.
This wasn’t a piece of code but an initiative to clean up Linux, which was sharply divided by “GNOME vs. KDE wars” in 2000. You can see a year 2000 email announcing freedesktop.org here and the tons of stuff still happening there to this day.
To get this going I had to reach out to key developers in GNOME and KDE and talk them into it, and then keep momentum by doing some real work under the freedesktop.org umbrella.
I wrote a lot of code in GLib and GTK+. Some examples:
This is a different UI toolkit used by litl OS and later by GNOME Shell. Clutter is based on OpenGL and the idea of a 3D canvas; I contributed a few significant patches to it, but wasn’t one of the main developers.
I designed and implemented an IPC system called D-Bus which has become central to how Linux works. It’s now being reimplemented in the Linux kernel.
The Linux desktop (which is really the Linux OS, if you think of an OS in the way that Mac/Windows/Android/iOS/ChromeOS do) has a “swarm of processes” architecture. The main point of D-Bus is to facilitate this architecture. Prior to D-Bus, processes communicated with and discovered each other only over X11 (which was per-user and not secure), or they used a variety of ugly hacks.
D-Bus was both a technical and a political solution. Politically, it had to make both GNOME and KDE happy for wide adoption. This involved talking to key decision-makers from both projects, and avoiding dependencies on either project. Technically, it had to be a good way to coordinate a swarm of processes into a single user experience.
D-Bus enabled a bunch of other projects, including NetworkManager and the very-controversial-these-days systemd. It solved a real problem and it’s still a core component of Linux over a decade later.
pkg-config
is a simple utility to output compile flags for libraries on
Linux. It’s still widely used; my up-to-date Fedora system has 349
.pc files (pkg-config package descriptions) on it right now, for
example.
James Henstridge wrote the first pkg-config in shell, and my C
rewrite became the widely-deployed version. At the time we created
pkg-config there were a bunch of scripts that came with
different libraries (gtk-config, xml-config, and so on for
every library). The scripts were all slightly different, but more
importantly they didn’t handle the problem of merging flags. So
they could output conflicting or duplicate flags, and you’d have
to deal with this in your Makefile.
pkg-config replaced the many foo-config scripts with one
program, plus metadata files (.pc files) installed by each
library.
pkg-config was part of the Linux solution to API evolution.
GConf was the GNOME configuration API for many years.
GConf was the first thing I wrote in my career which was widely adopted. The high-level concept was sound, and it worked, but the code was awful. Many small details were wrong and even bizarre. Maintaining this API taught me a lot about what not to do and how to be a good programmer.
At some point I documented all the stuff I’d fix about GConf if I were redoing it, and by some miracle Ryan Lortie ran with all of that plus came up with a bunch of his own improvements. So GConf was replaced, finally.
GConf was pretty cool at the time; the state-of-the-art on Linux back then was for every program to have its own config file in its own invented format. GConf was cross-process and model-view, meaning you could have a setting shared among many processes, and changing it would affect all those processes instantly.
This was a book about how to write GNOME applications, written in 1999. It wasn’t a great book but for a while it was the only one available, and some people found it useful or even liked it.
My blog has a lot of dated posts that don’t matter anymore, but here are a few that may be of ongoing interest.
Software product management:
Other technical:
Finance:
Misc:
Most of what I’ve written is on my blog or these days on Twitter, but some old stuff is here. A few of these were fairly influential: