Pat Gunn (dachte) wrote,
Pat Gunn

Abstract strategies for abstraction

There are a few purposes of abstraction in programming; one of them is to construct a uniform API that is independent of the backend that can work with multiple backends. The classic example of this is a database layer; while it's possible to develop for a specific database like PostgreSQL, by creating a database abstraction layer, one can smooth over the differences between SQL dialects/capabilities and let users of one's software pick a database. The simplest way to do this is to go with baseline SQL, but that's not the only option; one can also write higher-level abstractions and expect the coder to implement them in terms of the specific SQL dialect. An example that better examples the use for this is a hardware abstraction layer for an operating system; these use whatever processor features are helpful and present and fallback to baseline as a last resort; there are often big performance (or security, or reliability) benefits to be had by using these features (like setting a NX bit).

One of the trickier tasks of a programmer, and one of those that's most judgement-laden (which I expect would be harder to instruct fresher programmers on), is writing this kind of abstraction layer when one only has intimate knowledge of a single backend target. The most recent time I wanted to do a large-ish version of this was when working on my wiki/blog software; one of the features it has is the ability to autopost mirrors of its content to other content engines. Right now, it can do this to LiveJournal, but I wanted to hedge my bets a bit and make it easier to later autopost to anything else I could get a decent API for; I needed to abstract two parts of the code for this:

  • The sync functionality, that knows how to post to foreign content engines, how to delete content from them, and how to update content on them. All of these are important because sometimes I update posts after I make them, or decide to remove them. There are additional functions to deal with post metadata, which might or might not exist on some target content engines; on POUND different icons are shown for a post depending on what time-of-day I make the post (not LJ supported), and I can note the music I'm listening to (LJ supported) and arbitrary other metadata to post (not LJ supported, generally). The second abstraction I needed for this is the ability to format a post for whatever markup is appropriate for that content engine. This is kind of a black art, given how many potential content engines there are. Of these two abstraction layers (both written now), the second is the one that's brought me benefit; I still only sync to LJ (so far) and have never written another sync module (although writing one would be easier now), but the ability to autoformat my posts for other media was very useful for RSS/Atom feeds as well as textdumps.
How do we actually write these layers given only one example?
  • Sometimes we don't; we might just wait until we have two examples and use the difference between them to figure out the least work we need to do to support both. This avoids work that might not be needed (parallel to well-known rules of optimisation), although it's not necessarily so great for primary-author opensource development (you know your code best and optimise for your preferences, but someone else who, say, uses a different database, might find it much harder than you would to extend your code), and if you know you eventually will want this kind of abstraction layer and are implementing things for the first time, you may be wasting time not building it in from the start. Plus, if you know you'll need more than two, you might get the abstraction wrong if you stay too close to whatever backend you're targeting right now.
  • Sometimes you have a nice language barrier that's a good hint; in the case of targeting multiple SQL backends, most of the time SQL is a guest in code written in some other language, and you can use the language bridge as the abstraction (or something near that language bridge, like a high-level "database-stuff API"). Sometimes you might even use something on the other end of the language bridge (using stored procedures for database compatibility), although this won't help that much with things like datatypes.
  • Other times you'll find yourself guessing at what anything you're going to want to interface with will have to provide, and try to leave yourself enough flexibility to deal with that (saving yourself some general-purpose space to save information that's probably usable to lookup keys in an unknown content system, perhaps)
  • Even if you get a good design, you might find yourself tweaking the abstraction layer when your provisioning is wrong for some new target, although if you really aim for flexibility these tweaks might not be too painful; the less-structured your data structured, the more flexibility you might have here, at the cost of performance and integrity. This is one of the times in programming where free-form is good for more than aesthetics.
  • For many situations, it's worthwhile to think about baseline and optional functionality while designing this kind of API; some things are performance hints or informational that won't be implementable (or will be unimportant) with some backends, or perhaps you'll want to focus on baseline functionality first (e.g. a plaintext export of your genomics data for API access) and add the cool formatted structured form later once you've whacked all the bugs out of the linkage.
Making these abstractions doesn't have a lot of set rules, but it's still a learnable skill.
Tags: programming

  • Testing functions in Perl

    (Nothing particularly profound or my-idea-centric here, and I was tempted to post it to my personal blog instead, but it's worth trying to learn…

  • Statistical Software Components

    A few months ago I mentioned my big library of useful generic C/Perl functions (libpgunn). There are plenty of other general-purpose libraries out…

  • Lessons from Refactoring

    Recently, I've been going through a number of my programming projects and moved any function that's either generally useful or used in more than one…

  • Post a new comment


    Anonymous comments are disabled in this journal

    default userpic

    Your reply will be screened

    Your IP address will be recorded