Some Clarification on Yahoo! Issues

From the recent activity on our support mailing list and in our IRC channel (#pidgin on irc.freenode.net), it's plain to see there is still a LOT of confusion about the recent troubles logging into Yahoo Messenger via Pidgin. I'm going to try to clear some of the confusion up. I apologize for the technical nature of this post. Feel free to skip ahead to the last paragraph of "Solving the Problem" below if you don't care what the problem is and just want to fix it. Or take the simple approach and just upgrade to Pidgin 2.5.7 and skip down to "But I already upgraded! It still doesn't work!" if you still have trouble.

The Problem

The specific problem that affected us is the authentication mechanism. Yahoo Messenger 6 used a spectacularly complicated password obfuscation method to "encrypt" the password as it was being sent over the wire to Yahoo's servers. Back in 2004 when we were preparing our 0.79 release, Cerulean Studios, the creators of the Trillian client, were kind enough to implement this authentication mechanism for us. As part of this change, we began to identify to the Yahoo servers as Yahoo Messenger 6. This was all fine and well.

The real problem came relatively recently. At some point in our recent history, which I honestly don't feel like tracking down now, we started identifying as Yahoo Messenger 7. At some point after that we again updated to identify as Yahoo Messenger 8. Both of these changes were related to file transfer and other feature enhancements. When we made these changes, however, we never updated our authentication code. This means we were claiming to speak Yahoo Protocol version 15 but we authenticated the same way protocol version 13 clients do. Even this worked for quite some time.

Yahoo began upgrading their servers at some point recently to phase out the old Yahoo Messenger 6 client. In effect, they want to force users of older software to update to the current client. Whether this is a good thing or not depends on your perspective. From Yahoo's point of view, I'm sure this is an excellent move, as it should make their server software simpler (speak one less protocol, one less auth scheme, etc.) and reduce their support load (fewer client versions to deal with). Where it became a problem for us is that at the same time, they started requiring protocol version 15 clients to speak the version 15 authentication scheme, which we never implemented. Since we still spoke version 13's authentication, this cut us off entirely.

[Note: Some users have discovered that not all of the Yahoo Messenger servers are rejecting the old authentication mechanism from Pidgin. The number of servers that still accept it appears to be shrinking, however. We advise upgrading to Pidgin 2.5.7 as soon as possible.]

Solving the Problem

A few months ago, a kind soul pointed out to us documentation he had assembled on how the current Yahoo Messenger client authenticated and communicated with the servers. It turns out that the convoluted authentication mechanism we had used for years was replaced with a significantly simpler one that uses a few simple HTTPS requests. The beauty here is the login data is encrypted in the same manner as your communications are encrypted whenever you purchase merchandise at a retailer like Amazon. This makes it significantly simpler for us, and probably allows Yahoo to simplify things on their end as well while increasing security during the login process for all their users.

As I mentioned in my previous post, two of our Summer of Code students, Sulabh Mahajan and Mike Ruprecht (keep an eye out for these names--they're responsible for chunks of Pidgin 2.6.0's changelog!), implemented this new authentication scheme. We eventually merged it into what will become Pidgin 2.6.0. This code has had some quality testing already, thanks to the guys over at Adium.

In response to the Yahoo server changes, I (quickly) yanked the most important changes for this new authentication scheme and slapped them onto Pidgin 2.5.6. After some testing and some other important bug fixes, we kicked Pidgin 2.5.7 out the door. In theory, upgrading to Pidgin 2.5.7 should make most people's Yahoo problems go away.

But I already upgraded! It still doesn't work!

Yeah, we have had a few people with this problem. Here are a few suggestions:

• Change your Yahoo account's Pager Server field (located on the Advanced tab when editing the account) to 'scsa.msg.yahoo.com', especially if you previously read this document
• If you're crashing when you connect your Yahoo account, again, set your Pager Server to 'scsa.msg.yahoo.com' and try to log in again. If you can't edit your account, you have two options:
• Run Pidgin from a shell (command prompt). 'pidgin -n' will cause Pidgin to start up in an Offline status, thus allowing you to edit your accounts. Set the Pager Server as described above.
  
• If you know the name of the server that you entered in the Pager Server field, you can change this manually. Locate your .purple directory (covered in the FAQ!). In this directory is a file called accounts.xml. Open this file with an editor (Wordpad on Windows; any text editor will do on Linux and other Unix-like systems) and find the server name you entered. Change this value and save the file. Start Pidgin and you should be fine.
  
• If you upgraded by compiling Pidgin yourself, make sure you removed your distribution's existing libpurple package (for Ubuntu and Debian users this is libpurple0; for RedHat/Fedora users it should be libpurple). Then run 'sudo ldconfig' (or 'ldconfig' if you have a root shell already) and try again.
• If you're behind a proxy server that proxies HTTP connections but not HTTPS connections, and you have to explicitly configure the proxy in Pidgin, you're probably not going to have any luck. Our current proxy code can't handle this kind of configuration. Sorry.
• If you're on Windows and you get errors about a corrupted installer, clear your browser's cache, then restart your browser and try to download again.

But I can't upgrade!

If you really can't upgrade, then try setting the pager server to either 'scsc.msg.yahoo.com' or 'cs101.msg.mud.yahoo.com' and try again. But you really, really, really, really should upgrade. If you do eventually upgrade, you'll have to change this again, as I describe above.

I upgraded but now ______________ doesn't work!

We've had some reports that after upgrading there are some issues such as missing buddy icons and buddies never being shown as signed off. We are looking into these issues and hope to have them fixed for Pidgin 2.6.0. These issues didn't show up in my testing before the 2.5.7 release.

Tracking Yahoo Problems

Please don't open any new tickets about these issues or about not being able to log in. There are already open tickets to track them. Please also don't comment with "Me too!" or similar on these tickets--it just creates more noise for us to sort through when we're working on Pidgin. If you want to express a "Me too" on one of the tickets, click the arrow near the top of the ticket page that's pointing up. This will cast a vote for the ticket.

I hope this information is helpful to people.

The Great Yahoo Debacle

By now it's no great secret that Yahoo has decided to break clients that use outdated authentication code. The result of that change is that Pidgin 2.5.6 and older no longer work unless you connect to one of a very few servers left that still speak the old authentication mechanism. We knew this change was upcoming, but all indications we saw pointed to us having almost another two months to push a release with updated authentication code.

All that aside, we've had some updated authentication code for a while now, and were originally waiting to include it in the Pidgin 2.6.0 release. However, since Yahoo made the authentication changes earlier than we expected, we had to react as quickly as possible to stop the flow of incoming complaints. As a result, we backported the Yahoo authentication changes so we could cut a Pidgin 2.5.7 release fairly quickly.

Pidgin 2.5.7 is now released and does work to connect to Yahoo.

Just for the record, I'd also like to point out to those who complained about waiting three days to get a working release that the last time Yahoo screwed with authentication, it took over a week to even get code to fix the problem, let alone prepare a release and actually get it out to users.

Additionally, we fixed the annoying MSN disconnect-on-block issue and changed AIM's behavior such that the "could not retrieve your buddy list" error appears only once per AIM account.

Enjoy the new release!