Sunday, September 27th, 2009 12:00 pm
It occurred to me recently that programming comes in several flavours, and that how much I enjoy a programming task depends strongly on which flavour predominates. The flavours I've identified so far, in descending order of how much I like them, are the following:

Data munging: You have a large mass of data in some fairly generic form, and must fold, spindle and mutilate it with well-understood tools to extract the information of interest to you. Writing Unix scripts is the classic example, but list manipulation in Haskell or Lisp and array manipulation in J or APL have this flavour too.

Clever algorithms: You have some calculation or task to perform, and brute force has proved inadequate. Now you must apply the Power Of Your Mind to find a cunning and better approach. I haven't actually done very much of this stuff, but I have had to solve a couple of problems of this nature at my current employer, and have another one waiting for me on Monday morning.

Twisty if-statements, all alike: You want to zom all the glaars, but only if (the moon's in (Virgo or Libra), unless of course the Moon's in Libra and Hearts are playing at home), or (the engine-driver's socks are mismatched xor (the year ends in a seven and the month contains an R)). And you meanwhile want to wibble the odd-numbered spoffles if the Moon's in Libra, Hearts are playing at home and (the year ends in a seven or the engine-driver has mismatched socks). The challenge lies in making sure you've identified all the exceptions and special cases, and in actually coding them up correctly. Not remotely elegant, but better than...

Doctor X-style wizardry: Making a system do things that it was never intended to do. If you squint at the problem just right you have all the tools you need to do the job, sort of, but it's at best a witty hack and at worst a horrible bodge, and certainly not something you'd want to put much weight on. All non-trivial TeX programming has this nature. This kind of thing is quite fun when you're doing it as a joke or a proof-of-concept, but it's downright horrible when you need to do it to get something important done. But it's still far more fun than...

API spelunking. You have a candle, a slice of cheese, and a pair of old boots. You need a fork-handle. Is it even possible to construct one out of what you have? Is there a chain of method calls and constructors that will lead you from what you have to what you need? And if you succeed in constructing your fork-handle, is it the fork-handle you need? Or some nearby, but completely inappropriate, fork-handle? This is in some sense dual to data-munging. The worst examples I've encountered have actually not been in relation to documented APIs (though try reading lines out of a zipped text file in Java, if you want to see what I'm talking about), but rather in large crufty systems with complicated and ill-thought out data models. The Law of Demeter addresses this problem, but I have yet to work on a project that sticks to it. I'd settle for some sort of coherence theorem, but that would require coherent API design, which is kind of the problem to begin with...
Sunday, September 27th, 2009 12:29 pm (UTC)
This post is Official Infocommie approved.

:D
Sunday, September 27th, 2009 07:07 pm (UTC)
Thank you!
Sunday, September 27th, 2009 12:41 pm (UTC)
That last one made me laugh out loud. Oh the number of times I've been there. What you didn't mention is the worst bit, which is when you have finally crafted a vaguely working, if slightly shonky fork-handle out of the sole of a boot, some melted cheese, and a bit of wax, losing a couple of fingers and a large portion of your sanity in the process. So you show it to someone, and go "look!", and they open a hidden door and say, "but we have a cutlery drawer here..."
Sunday, September 27th, 2009 02:44 pm (UTC)
I don't think that's the worst bit: that's frustrating and humiliating, but it at least means that you can throw away your horrible fork-handle and start working with a nice one. The worst bit is when you think you've got it, and that the magical incantation new Cylinder(new SwissRoll(new Layer(boot.getSole(), new Mixture(cheese.melt().reform(), wax.melt(new CigaretteLighter(pocket.getContents(WordHandbag.LEFT).get(0)))))))¹ will Do What You Want, and then you discover that the getSole() method is only implemented for running shoes and sandals...

¹ Why yes, I have been writing a lot of Java recently. How did you guess? :-)
Sunday, September 27th, 2009 07:06 pm (UTC)
On reflection, I don't think my description really got across the precise horror of API spelunking. Actually trying to construct a fork handle out of cheese, a candle and an old boot would have more in common with Doctor X programming than with API spelunking, which is essentially a navigational challenge: the question you're always asking is "how do I get there from here?" The kind of programming that you need a D&D Minotaur (http://badgods.com/minotaur.html) for. The difference between API spelunking and data munging is that in data munging (as I use the term here), any of your operators can be used at any stage, whereas in API spelunking you're limited to a small set of available operators at each turn, which may advance or retard your progress. In Unix scripting, pretty much everything takes a sequence of lines of text and turns it into a sequence of lines of text, but if you have a Boot object you can't decide to call getWick() on it. You're working in a category (or rather, a multicategory) rather than a monoid.

And it's especially horrific when the thing you have is really really like the thing you need, but for some insane reason the library author doesn't agree :-(
(Anonymous)
Sunday, November 15th, 2009 11:58 pm (UTC)
You've just fairly accurately described programming the BlackBerry.

Want to make a font appear anti-aliased? Well, you might spend a few days toying with different values to feed the style int, find that what looks good on the 8320 looks terrible on the 8700, tear your hair out, give up on any easy solution, and then somebody pops up in a forum to tell you: yes, just use net.rim.font.FONT_DO_SOME_ANTI_ALIAS_PLEASE, and no, that's nowhere in the published documentation. How did they find out about it? Beats me, but the BlackBerry's been around long enough to spawn some serious beardage on its acolytes.

Or, if you want to save an email to mass storage, you discover there's no way to access the attachments. Unless the email author has titled the attachment in question x-blackberrys-are-so-secure.zip. (I'm not kidding, only exaggerated the precise name a little.) So you keep going with the rest of the message, save it all to disk, great. Now, to pull the data out and recreate it... Oh, those MessagePart subclasses you need to use that you were just reading data from to save to disk? None of 'em have public constructors. So sorry, it's all mostly read-only in the end.

Hate that (*@# platform, and that's probably why I've been sitting on a great little app for a year, because finishing it is just so painful.
Sunday, September 27th, 2009 03:17 pm (UTC)
You want to zom all the glaars

HAHAHAHAHA
Sunday, September 27th, 2009 07:59 pm (UTC)
"You have a candle, a slice of cheese, and a pair of old boots. You need a fork-handle." Sounds like the beginnings of a text adventure game!
Friday, June 25th, 2010 03:44 pm (UTC)
you picture write what?
Sunday, September 27th, 2009 11:29 pm (UTC)
Do you think of "Clever datastructures" programming as a subcase of "Clever algorithms" programming?

If so, then you might (in some cases) be able to turn some of your third-fave problems into some of your second-fave ones... Although I guess that the "squinting" feel of your fourth-fave problems can happen here too...
Monday, September 28th, 2009 08:40 am (UTC)
I suppose I do. And yes, that's what I try to do with type-3 problems, but it isn't always possible: or rather, after staring at it for a while and trying to come up with an elegant data-driven design, sometimes I conclude that the path of least resistance is just to bang out the if-statements :-(
Monday, September 28th, 2009 07:22 am (UTC)
I find languages like Ruby or Python (to name two that I am familiar enough with to be able to say this) help with the API spelunking in that if the API sucks, there are other ways to introspect and get at the insides of the software to get what you need done. Although that does turn into Dr X wizardry very quickly...
Monday, September 28th, 2009 08:41 am (UTC)
Yep. Duck typing obviates a lot of the problem, too. And in Perl, where the bias is against creating object hierarchies, the problem is further reduced. Will the increasing dominance of Moose (http://www.iinteractive.com/moose/) end this situation? I hope not...
Saturday, June 26th, 2010 07:31 am (UTC)
You are the few people who wrote something about that, I like it.