Sunday, August 28, 2011

Major Changes Afoot

Well, it's been about forever since I last bothered to post anything here. Since my last post, we've released several times, introducing and fixing a bunch of bugs. Now, however, we're shifting our focus to new development of a sort that we don't do often—compatibility breakage and big internal changes. This means that our main development effort is now in working toward Pidgin and libpurple 3.0.0. I'm going to try to explain some of the work going on for the benefit of anyone who reads my ramblings, so here goes.


First of all, there seems to be some confusion about how Pidgin and libpurple version numbers work. I'd like to try to clear some of that up.

Pidgin, Finch, and libpurple use what's called “Semantic Versioning.” That is, each part of the version number has a particular meaning for users and developers. We chose this scheme to assist plugin developers in knowing when significant changes would require effort on their part to maintain compatibility with current Pidgin and/or libpurple versions. It also helps our users by letting them know when their existing plugins will stop working. So, how does this semantic versioning work? Let's look at the format of our version numbers and find out.

Pidgin, Finch, and libpurple version numbers have three components, separated by dots. At the time of this writing, 2.10.0 is the current version number. These components are called major (currently 2), minor (currently 10), and micro or patch (currently 0). Let's look at what each means.
  • Major: the major version doesn't change often. The last time we changed it was in 2007 with the release of Pidgin 2.0.0, and before that was in 2004 with 1.0.0 when we started using semantic versioning under our previous name. Whenever this number changes, we've made changes to Pidgin, Finch, or libpurple that break compatibility with every UI and plugin that currently exists. Usually this means that we've removed something from the API exposed to plugins and UI's, or that we've changed something about a function (its name, arguments, return type, or the header file it's in). Sometimes plugins or UI's can just be recompiled when this happens; other times they need maintenance to become compatible with the new release. Additionally, the major version never decreases. It will always increase when it changes.
  • Minor: the minor version changes more frequently than the major version, but generally less often than the micro or patch version. Whenever the minor number increases, we've added things to our API that do not break compatibility with existing plugins or UI's. A prime example of this is adding the voice and video support we added in 2.6.0. We added a bunch of stuff to the existing API, but didn't change anything that would cause a break in backward compatibility with existing plugins and UI's. When the minor version decreases (gets set back to 0), it means that the major version has changed, and the whole cycle starts over.
  • Micro or patch: The patch version changes with almost every release. When this version increases, it means that we haven't touched API at all; instead we've done nothing but fix bugs or add small features that don't affect compatibility with plugins or UI's. When this version decreases (gets set back to 0), it means that the minor version has changed.

Related to all this, and important only for plugin and UI developers (so skip this paragraph if you're not a developer!), is the behavior of the PURPLE_VERSION_CHECK macro. Many developers expect PURPLE_VERSION_CHECK(2, 5, 0) or similar to expand to a statement that evaluates to 1 or TRUE in the case of building against libpurple 3.0.0. This, however, is not the case. Because our major versions are incompatible with each other, we have intentionally written PURPLE_VERSION_CHECK to fail if the major version is not an exact match. We understand this can be confusing and inconvenient, and we sincerely apologize for that, but we're not going to change it.

What all this means is that if the first number (major version) changes, you're going to need to update your plugins when you upgrade Pidgin, Finch, and libpurple. If the second (minor version) or third (micro version) numbers change, it means you need to upgrade Pidgin, Finch, and libpurple, but your existing plugins will still work. No matter what the Pidgin, Finch, and libpurple version numbers are, you should always be using the newest version.

Structure (“struct”) Hiding

This paragraph is for those who aren't programmers. Feel free to skip it if you don't care about it. Pidgin is written in the C language. C has a type system, which means that if you declare a variable (imagine this as a box somewhere in your computer's memory) you can store only one type of data in it. A structure, or struct in C parlance, is a type made up of other types arranged sequentially. This is pretty easy to picture if you think of Lego blocks--stack a red block, a blue block, a green block, a yellow block, and a white block on top of each other and connect them together and you now have a structure made out of Lego blocks. It's pretty similar in C, except that you're telling the compiler to assemble something out of sequential boxes of memory instead of little plastic blocks.

Pidgin uses structs everywhere. We use them to represent things like your buddies, conversations, accounts, etc. Currently in Pidgin 2.10.0, most of the structs are in the public API—that is, anyone can directly access the members of the structs and do whatever they like. This is all fine and well, but it means that if our code changes such that a particular struct needs to grow significantly by the addition of new members, we can't always do that without breaking backward compatibility. (Yes, we included padding in a number of our structs, but we've burned through the padding in several of them, and although there are ways to work around it, I and a few other developers don't like them.) It also means that if we discover, for example, that switching the order of members in a struct allows the compiler to improve its optimizations or if we think a different order makes more logical sense for those of us reading and maintaining the code, we absolutely can't do this without breaking compatibility. We also can't rename members of structs for the same reason—it breaks compatibility with existing plugins, UI's, etc.

Because having these structs in the public API limits us so much, we're striving to hide as many of them as possible. By hiding, we mean that we're removing the struct definitions from the header files and moving them to the .c files, thus making them private. Plugin and UI authors will still be able to reference the “objects” with pointers, passing them to functions and operating on them with the appropriate sections of our public API, but no longer will the members of the structs be directly accessible outside of the .c files that define the functions that interact with them. For example, we have a PurpleRoomlist struct in libpurple/roomlist.c and libpurple/roomlist.h. For 3.0.0, the struct definition is in roomlist.c; thus the individual members are not directly accessible outside roomlist.c, even in other parts of libpurple. This “hiding” of the strcuts allows us significant internal flexibility in each file to modify the struct as we see fit.

Clean-Slate API Documentation

Because we generally change so much each time we increase our major version number, the API documentation can get a bit confusing if we keep doing @since tags and whatnot in our doxygen documentation. Our general overall feeling is that we prefer just starting with a clean slate at each new major version. This means that each time we do a new major version, all existing @since tags will disappear, any functions marked as @deprecated will be removed, renamed, or replaced as described in the @deprecated statement, and so on. We realize this decision may make some things more difficult for some plugin and UI authors, and we apologize for that, but our aim is to have overall cleaner documentation for everyone.

Merging of Old Projects

Over the years that we've participated in Google's Summer of Code program, we've accumulated a number of branches that have been sitting for quite some time. Most of these need some form of TLC and integration work. We want to try to incorporate at least some of them into 3.0.0 so we can finally benefit from the fruits of the students' labor. Most notably, we've been talking about merging the webkit integration branch into what will become 3.0.0. Eventually, this would allow the support of Adium's message styles, although it may not happen right away.

Another project of notable interest is some of the logging changes that went on in a previous Summer of Code project. One of our new Crazy Patch Writers took some of that work and made some progress on it; we don't know if this will make it for 3.0.0 or not yet, but it would be nice to have some of the features, such as non-blocking log writing.

Other Changes As Wanted

We may decide to make other changes since a major version change gives us the opportunity to break so much. There have been a number of ideas floated, ranging from supporting that XDG directory spec that I can't stand to doing away with the xml files in .purple and replacing them with something else (with what, in particular, has not seriously been discussed). There is a whole range of possibilities of things we could do for 3.0.0; it's just a matter of one of us wanting it and sitting down to write it.

So everyone should stay tuned, as we'll be making more and more changes over the course of the 3.0.0 development cycle. We don't yet know when we'll be releasing 3.0.0—this is another one of our famous “when it's ready, and not a minute before” time frames. We do know, however, that it will change a lot!