Doing The Right Thing

One of the hardest things about being a framework developer is getting things right. There are a lot of tough choices you face when you're looking at an open set of applications. Getting things adequate is often fairly easy, but getting them right is a lot more conceptual work. As a framework developer you're often stuck with your choices for a lot longer. Three related examples:

Standards vs. Sanity

Who wins in this battle? If a tag claims it's 10 MB long, how do you handle it? What if it claims 50? You can get away with assuming it's not longer than the file, but that's still a lot of room (and RAM) for error. Or what about when a string list is null delimited and a text field was padded with a kilobye of zeros? A single string and some non-compliant junk or a really long array with the last several hundred empty?

Standards are funny. I tend towards the pedantic in their implementation, at least as much as users will let me get away with. But since the whole function of standards seems to be interoperability, allowing for less than perfection in the alternative implementations seems within the spirit. But where's the line? How many bytes is "silly"? This is one that I struggle with.

Everything In Its Right Place

I find myself shying away from reasonable suggestions at times because they don't seem like they're in the right place or don't conceptually fit. There are times that guarding against something bad at one level may prevent common errors, but mask subtler ones that could be more robustly handled at a higher level.

I hit this today when it was suggested that file locking via fcntl be added to TagLib. The problem is that I don't really trust fcntl locks. They're buggy on certain file systems (network, notably) and it wasn't really practical to propage useful information up the TagLib stack as to why a save failed in the case of a previously existing lock. It also assumed that shared locks were properly used by other programs, which, while I've not worked with them significantly in the past, I don't trust the world of tag editor authors to not misuse locks in a way that wouldn't screw all manner of things up.

Locks in TagLib probably would have prevented some errors. Even some that were pretty serious. But it would have also led to some really difficult to track down corner cases like, "Why won't KFileMetaInfo work on our NetApp's shares?"

It's tough to know the balance between damage control and predictability.

Usability for Application Developers

In all of the talk about software usability, API usability is one of those things that doesn't get as much formal treatment. I've noticed this in a number of places, both professional and in Open Source. Matthias's talk at Akademy a couple years back hit some of the important points. Often shorter isn't better and often making it possible isn't enough. I've seen a lot of APIs that work, but are difficult (or slow) to work with and where the end result isn't easy to read.

KDE, and other similar projects, fortunately have a culture where API usability is, while not formalized, a comunity value that's shared through patch review and similar mechanisms. But it's not one that's permeated to software design at large or taught in any formal context that I'm aware of. Design concepts (patterns and whatnot), domain knowledge and programming languages are taught, but one of the most important skills, API design, has been left as something of a black art that's at best acquired in a community.

Even the best software engineering programs in the world are turning out people that might be bright, able programmers, but that have never really been taught how to design an API -- how to get it right, not just working.


I think good API design is one of those things in life, specifically in software engineering, that can really only be acquired through experience. To know what makes good API you really need to have used a number of different APIs, and have used both good ones and bad ones. One of the very first designs I did way back was atrocious. I made decisions about certain things because I thought that was the way they were "supposed" to be, and not because they made sense. But I wasn't too worried because my only client at the time was me. I learned to never underestimate who might later end up using it (ugh).

Anyway, I just thought I'd mention that even in a large coroporate environment, API design is still best put before community review. That community will be others in the company of course. I recently saw a design go through a number of revisions based upon the collective knowledge and opinions of more than a dozen people around here.

By Rajpaul at Wed, 08/16/2006 - 17:48

what we call good design also depends a lot on who will maintain it. Many (especially C++ coders) tend to just make everything public, lots virtual and supply getters for internal data structures without at least making those const.
And many I spoke to think its the right thing to do since you may need to add that later anyway.

Teaching people how you should design with a minimum of public methods (only those you actually need) and add a public method or a virtual when someone external needs it (after checking if it indeed is needed) is a way to make things easier to maintain and refactor.

So, not only experience, but also quite some time maintaining classes and an occasional refactoring of a library without breaking the external API.

But best practices (which also depend on the language used, btw) can be written down and taught. And I think thats very much needed with the amount of young programmers we have nowadays, all doing their own things.

By Thomas Zander at Thu, 08/17/2006 - 08:28