Unscheduled downtime 20:00-20:30 and CTD

The reason for the CTDs that have the following symptoms:

Player comes out of warp and CTD's; then for each login of that character, a CTD occurs when “tuning into subspace frequency”. Only fix is to petition for a GM move.

I will now attempt to explain how the crash occurred.
EVE is a deterministic distributed simulation; the server maintains a master state, which it evolved periodically. When a client enters a solar system the server calculates the set of data needed for that client to be able to run its subset of the simulation. This is the union of the spheres of influence of the objects that "touch" the client. This can be referred to as the causality bubble of the client.

Given how sparse space is this has proven to be a good way to partition up the data needed for the client to make a true simulated evolution of the state.

Once the client has received the initial state, it evolves to create a believable illusion of space flight. The server only sends the client the state patches needed to represent external actions of the world.

Flying in an asteroid field while avoiding the asteroids and being orbited by an Angel Hijacker is no client<->server interaction but as soon as a player in the same belt changes direction the client gets a small time stamped packet notifying it of it and the client re-evaluates the state. If a state update involving ones own ship arrives late, because of lag for example, the determinism is wrong and then you notice the "bungee effect" as some call it, in most cases the re-eval is not noticeable). This is what makes EVE use comparably little bandwidth and relatively immune to "Radar hacks" (where a hacker creates an interface, showing information, that is know to the client but not displayed in the game's normal user interface).
This makes EVE a bit voulnerable to something which could be called a "Jovian precognition unit" and if someone creates such a unit before the Jove do, we’ll salute you

Now after this little prologue I can finally explain what the problem was. The problem was a NPC entity in orbit mode around a proximity sensor. This state was almost in all cases built as an NPC kill mission close to an Encounter.

When a client came out of warp close to such a setup or arrived there through a Stargate, the server sent the client all the contents of the causality bubble, apart from the proximity sensor (as it doesn't affect the state directly). Because the proximity sensor was not sent to the client, then the client crashed as soon as it attempted to evolve the state as it didn't have the follow target of the entity (as the server hadn't sent it).

We have now fixed it so the entities in this case are not orbiting the proximity sensor, which was an error; this is the server side fix we applied in the unscheduled downtime. We will also work on making the differential equation evolver (commonly referred to as Destiny) to be more resilient to these kind of issues, as it is better to warn and evolve incorrectly than to CTD.