The EVE API Project

Þriðja bloggið mitt

Translated: "My third blog."

Reykjavik is great. The weather has been a balmy 5-10 degrees Celsius and beautifully sunny the past few days! I almost wore my shorts into the office today, but a bout of sensibility made me put on jeans. Had a nice 3km bike ride to work around the pond and ran into one of our Content developers on his way in so we spent a while discussing life in Iceland as we finished heading in. Good people.

Anyway, enough of that. On to business.

A few weeks ago I sat down with Hammerhead and he went over an idea he wanted to incorporate into Revelations 2. Data exports. I immediately started salivating over the idea, but it ended up floating around without much of a home for a bit while we had some issues that came up as a result of some downtime and EVEMon's subsequent overzealous server activity.

Let me explain a bit about the existing system, and then I'll talk about what's wrong with it and how we're making it better.

The Existing MyEVE

Right now MyEVE runs on a cluster of webservers. The machines are Windows and run IIS. The site is written in Classic ASP (VBScript). All of this you probably already know. We have two separate database machines - one for the EVE data and the other for website data. Most of the website information is all stored on the website database. The forums and account information still reside on the main EVE database, however.

This system works okay, but it has some problems. Notably, it's never a good idea to have your website on the same database as the game. We don't want the two to be codependent at all. If for no other reason, it'd be nice to be able to use the forums while TQ is undergoing maintenance! There are other reasons of course - stability, security, etc. It is really unfortunate that things like EVEMon can have a detrimental effect on the game.

Improving the Old

One of the big projects here is to move the forums to use a separate database. When that is done we will hardly ever need the main EVE database. This project is pretty much done, I'm just waiting on some support from the database guys to get the last few kinks sorted out. Then there will be rather a lot of testing before it actually gets pushed live.

This has been bouncing around for months and months, since before I started actually. As to when it will happen - well, other things have been given a higher priority, as you'll see. Hopefully sometime, though.

Another project has been to rewrite the way the API works. I started this project by splitting out the currently training skill information into its own page. Cut down and streamlined, it enables EVEMon to work without hitting us for the resource hog we call the character sheet every time they want some data. I had started going in this direction - creating much smaller pages that people could access to get the individual bits of data they need. However, Hammerhead's idea bubbled to the front and we realized that we could do so much more with just a little different approach.

Building the New

Enter the EVE API project.

In a nutshell, the idea is to provide a new interface that will enable developers to programmatically access data about their account, characters, corporation, the map, alliances, and whatever else we can think of (and get approved). The site is designed to solely be an API and not a user-facing website, so there are many optimizations that we can do to make the whole process easier on our databases and easier for programmers to use.

We have chosen to support XML and CSV for most of the outputs. We are also moving away from sessions and have chosen to go with a stateless design. Authentication will be done on each request and you are expected to obey any "don't retry until X" or "cached until Y" markers that you come across. The system uses heavy caching (with support for a distributed cache if we need to) as well as throttling so it can't overload the database with too many queries (well, in a simple sense of "queries per second" and not "badly formed query takes down the database").

As an example, let's look at what a pretty standard request will look like when talking to the API.

• POST to //SITEURL/char/SkillInTraining.xml.aspx with arguments:
  • characterID: 123412341234
  • username: test
  • passhash: y7Rp6nWNVogIcdymYTwFWyLp+2A=

Note that the password is a salted hash so you never send the actual password over the wire. Also note that this hash cannot be used to log in to the client or MyEVE, so even if someone did end up with it the worst they can do is access the API with your credentials. This gives us a side benefit of allowing people to provide their hash to third party sites, giving them a relatively secure way of letting a web site access the API on their behalf without the site getting their login credentials.

When you get the server response back, it will look something like:

 5/10/2006 8:11:37 PM
    3/9/2006 11:07:03 AM
    3400
    1280000
    4096000
    5
    5/15/2007 11:32:16 AM 

The exact output is subject to change and will be documented.

At any rate, this is just a taste of what is under development. It is scheduled to be released with Revelations 2 and will have much more than just your currently training skill. Right now the big two are corporate information (roster/tracking information) as well as your personal and corporate wallets. Other functionality may or may not be included at release, but the foundation is strong so we can add all sorts of functionality going forward.

Wait, what about MyEVE?

I know it looks something like we're going down a rabbit hole, what with the existing issues with MyEVE. It is in dire need of a rewrite, that much we agree on.

The rewrite of MyEVE will begin after the API project is launched - so sometime after Revelations 2. It won't just be a rewrite, though, there are lots of plans for extending MyEVE. I'm not going to say anything about that just yet, however. You'll just have to wait and see!

Thanks for reading. Now back to your regularly scheduled pew pew!