The new EVE forums - technical underpinnings

As most of you are aware, we recently launched the new EVE forums with a bit of a resounding thud.

CCP Sreegs has done an outstanding job both addressing the issues as they occurred and messaging to you folks the current state of affairs and what we are doing about it – I think we web guys owe him a case or two at least. He is best placed to continue answering such questions so I will endeavor not to muddy his waters but I did want to take the opportunity to address some of your questions about how the new forums came to be – i.e. why did we choose YAF, what changes did we make and why.

A little bit of history

To set the stage a bit, here's a bit of background to EVE Gate's development. When we first drafted up the initial development approach for what was then called "Cosmos," we looked at a number of possible approaches and ended up looking in earnest at Community Server (by Telligent) to form the backbone we would develop upon. They had a robust forum solution with a lot of integrated features we could build upon. They utilized the same development environment that our web team used (.Net) and had some proven success in large scale implementations.

We began more in-depth discussions with them and utilized an evaluation copy of CS for the White Wolf forums for a bit to get a better look at it. Unfortunately, there were a number of areas where it really was not a good fit both technically and with their licensing so we had to look at other options. Opening the gates wide there were a lot of alternative approaches we could have taken and some of you have alluded to a few of them – everything from writing it from scratch to utilizing proven packages like vBulletin/phpBB, etc for some of the features that made up what is now called EVE Gate. Different languages, different packages, different components and architectures were all looked at with more than a few heated debates in the mix.

If you look back to my old blog on EVE Gate's architecture and all its web techno babble, the architecture we decided to go with is outlined. This approach mixed best of breed .Net development, a strong layered application architecture for performance and security with the flexibility to integrate other .Net solutions for specific features as needed. The core application of EVE Gate was thus written in house, and then for each feature (like the forums) we could look to see if there was a suitable component out there we could integrate, modify or if we needed to write it from scratch we could.

Choosing YAF

We have known for quite some time that the forum desperately needed to be replaced so it was always pretty high on the priority list for EVE Gate development. To that end, our Community department had prepared a white paper outlining the features that any new forum solution had to have to meet your needs. This included all the functionality you are used to from the existing forum, functionality you can expect from most modern forum packages and additional functionality that the community department really needed to serve you better. It was a pretty daunting list so the challenge was to come up with an option that met as many of those needs as possible and still fit within our technical criteria. Then we could work on anything that was missing as iterations. As part of our look into options we looked at the alternatives in the .Net space and learned of YetAnotherForum.Net

However, not only did we learn about YAF and start looking into it but we also took notice of sueetie.com Dave Burke of Sueetie.com has done a great job of integrating a number of community focused .Net components into a package and we began seriously looking into what he was doing. Looking at it further, we determined that there was some stuff we needed, some we didn't, some we would do differently, etc so his particular approach wasn't a good fit in the end, but it was an inspirational approach and at the very least I want to give him some props for his great work.

Open Source components

In choosing YAF for the forum piece of EVE Gate, it was not so much to pick a solution to meet all of our needs but to choose a package that we could build off of. Whether it was Open Source piece or not was not the issue for us, the question was more about the feature set and quality of the code which is comparable to other solutions out there. Is it perfect? No, really there is no such thing, but it is pretty solid and we can now build on top of it and integrate it further into EVE Gate.

Now some have worried that by choosing an Open Source solution we have to reveal the source code thus making your accounts and activities in EVE Gate vulnerable. This is not the case here as we have purchased a commercial license to YAF so we can properly protect our efforts. That said, we are big fans of Open Source initiatives and if we note issues in YAF that we come up with improvements for we will communicate that back to their project team to benefit the YAF community.

Building on YAF

So we had our base to build on, then we needed to integrate it into EVE Gate and build on top of it. At a cursory glance that may seem like a just a small bit of work on login mechanisms and a bit of skinning but in fact there was an awful lot more work necessary.

The entire authentication mechanism was rebuilt to tie the forum into EVE Gate but we also had to rework the permissions to be based off of role information retrieved from Tranquility to give the community team, GMs, devs, CSM members, volunteers, etc. their appropriate access, capabilities and badges. We rebuilt and added a lot of functionality to the admin functions to give the community team all the custom tools they needed including reporting, various banning options, moderation abilities, custom filters and searches, etc. We did a lot of cross integration with EVE Gate so that if you clicked on a user's name you were taken to the EVE Gate profile page, etc. There was an awful lot of stress and load testing done with subsequent performance improvements made under the covers. And not least of all we spent considerable effort on writing from scratch a more powerful search capability and then spent a lot of time optimizing it. Also from an appearance side we certainly skinned it but we also did all the integration work to pull in the new character avatars efficiently.

Incorporating your feedback

I'm sure I'm missing no small number of tasks the team handled as well, the more we delved into it the more places we found to better integrate or improve the solution, and frankly that's a process that continues as I write this. A lot of the feedback you have given over the past weeks are going in right now on top of the obvious work being done to fix the security issues

So choosing YAF as a base was an important choice as it gave us a very solid foundation to build on, and we did further work on top of that based on direction from the community team. Once that development effort was pretty much done first pass, we deployed it in test mode to get valuable feedback from all of you. That was incredibly important. From that beta phase we got back both bug defects that were filed and reports from the community team on what else players were saying - from things that were broken to the font being too small, to some folks don't like fixed width sites, etc. That was then prioritized by the team and they worked through it.

What next

I didn't address the significant flaws that made it into release and how that came to be, that will be for another dev blog to detail. We are doing a post mortem right now and doing some serious soul searching. We don't like making mistakes, let alone obvious ones we should have caught at various stages but much more importantly we do not want to repeat mistakes or gloss over flaws in process or skillsets that caused it.

So, we are both evaluating what went wrong while working to get the forums back in shape and properly scrutinized. There will be more details on that forthcoming but there will not be any time pressure to release it before it is ready. We will do another public test phase to get your feedback and there will be more details on that soon. The goal is to fix the flaws, make some improvements based on your feedback and release the new forums when they are truly ready.

If you have any questions about the choice of YAF or anything I've detailed here please let me know and I'll do my best to provide further detail. Thanks for your patience and all your feedback (positive and negative) while we work on getting the new forums to where they need to be