Code of Honor - Latest Comments

Re: Uh-oh: was the company site hacked?

Stefan — Mon, 20 May 2013 07:10:39 -0000

As someone who regularly does security work for a big (vBulletin based) gaming forum I can't tell you how many times we get attacked a day. Normally we get simple SQL Injection attacks that don't work anymore but I have had to fight hackers who got through our system using a 0day exploit. In my experience it's always been fun to try to outsmart hackers.

Re: Choosing a game network library

stone — Thu, 16 May 2013 07:39:29 -0000

very useful thanks.

Re: Uh-oh: was the company site hacked?

Patrick Wyatt — Wed, 15 May 2013 20:05:40 -0000

Custom written: the socket library we used was built on top of Windows IoCompletionPorts (IOCP) since we had lots of good experiences with them for battle.net, but that meant we had to write our own code for *everything* because -- in those days -- most libraries were not written to handle the "inversion of control" that comes with using IOCP.

These days -- assuming I was using IOCP -- I'd use something like this: http://twimgs.com/ddj/images/a...

Re: Uh-oh: was the company site hacked?

alin — Wed, 15 May 2013 14:36:16 -0000

Was this AcctHttpSrv server a completely inhouse solution or did you use a certain framework/base server ?

And as always: very good article!

Re: Uh-oh: was the company site hacked?

Dev Swali — Sat, 11 May 2013 20:32:13 -0000

Interesting read! Surprisingly, I can actually relate to an incident I had while I was a volunteer-GM at a MMO company six years ago. From my experience, the company was very experienced with security and had a well-built system to prevent attacks. The incident happened when one of the “paid” GMs working from the offices decided to register at a fansite that was created by one of the volunteers. Long story short, he used the same password as his GM account, and ended up causing the company to do a major security overhaul because they were not sure how a random “volunteer” had surprisingly gained access to their GM tools (which I don’t know why they were client sided in the first place). The volunteer even claimed that he had hacked the system, causing more questioning.

Re: Uh-oh: was the company site hacked?

Patrick Wyatt — Sat, 11 May 2013 02:26:27 -0000

Why would you fire someone over that? People make mistakes; mistakes aren't firing offenses.

Re: Uh-oh: was the company site hacked?

airplane — Fri, 10 May 2013 06:48:22 -0000

"I have to give him props, though."

Was he fired anyway?

Re: Uh-oh: was the company site hacked?

WulfTheSaxon — Thu, 09 May 2013 18:57:23 -0000

In case anybody ever wants to do something like this (successfully), in Firefox go to about:config and multiply network.http.max-persistent-connections-per-server and network.http.max-connections (and now network.websocket.max-connections and dom.workers.maxPerDomain if those are relevant to you) by the number of tabs you’re going to try opening.

You might also want to set browser.cache.memory.enable and browser.cache.disk.enable to false.

(Needless to say, make sure you set them back when you’re done, and don’t try this on anybody else’s server.)

Re: Uh-oh: was the company site hacked?

xboi209 — Thu, 09 May 2013 14:51:15 -0000

he must have been embarrassed

Re: Uh-oh: was the company site hacked?

Twilit Soul — Thu, 09 May 2013 10:36:24 -0000

That's a funny story.

Re: Uh-oh: was the company site hacked?

Stilgar — Thu, 09 May 2013 04:29:46 -0000

The performance benefits of systems like Jekyll can easily be duplicated by many web frameworks via the concept of output caching. Basically the system caches the whole string generated for the response for a second or two. Of course the real motivation behind using Jekyll is security.

Re: Uh-oh: was the company site hacked?

James Green — Thu, 09 May 2013 04:25:22 -0000

I've always been scared at the rise in popularity of Wordpress sites as an easy target for hacking. While not quite as user friendly, we use Jekyll to generate static pages we push out to our web server, which I guess is also good for performance.

Re: Uh-oh: was the company site hacked?

Stilgar — Thu, 09 May 2013 04:16:45 -0000

This reminds me of this one time when I was testing a new real time functionality in our web system. Real time web things leave a connection open and respond when the server has to notify the client. So I deployed the code and decided to perform a poor man's load test before declaring it ready for QA. I opened 20 tabs in the browser just to see what happens. The first tabs opened just fine but then the next set of tabs wouldn't be able to load resources like images and the last set of tabs would not open at all. I spend the next day in a performance monitor on the server checking various stats and I could not find a reason why 20 simultaneous users could bring the site down. No resource was a bottleneck. Then I realized that browsers had a limit on simultaneous connections to a single domain. When we introduced our real time web feature every tab would leave a connection hanging and some tabs will be left with no connections to work it. It was the browser throttling the connections not the server. Sometimes a bug really is not a bug :)

Re: The StarCraft path-finding hack

Adam Heinermann — Wed, 08 May 2013 20:51:17 -0000

They use A* at a higher level region/node system, with an array-based binary min heap to determine the costs. I don't know exactly what's done afterwards but I think they use string pulling against their sorted, pre-computed list of all terrain edges(AKA contours, as they seem to be called) to bring the path down to the tile/pixel level. There is a function called GetClosestReachable (known because of its obvious left-over debug string) that tries to get the closest edge from a unit, which is called for each node in the rough path.

Re: Avoiding game crashes related to linked lists

Patrick Wyatt — Wed, 08 May 2013 00:56:10 -0000

Yeah, what happened to it! Oh wait, I was supposed to write that. Two things happened.

First, I got more busy -- like, a lot more: writing code and advising several companies in addition to raising four kids. I'm constitutionally unable to not work so my free time is always trending towards zero. That's why no articles recently, ssssooorrrry.

Second: this was my most controversial article. It occasioned quite a lot of negative responses because I killed several sacred cows without providing enough evidence for said slaughter. Example: coders all put out because I used the low bit of the pointer as an end-of-list sentinel. Actually, boost intrusive lists do that too but the gory bits are buried fourteen layers deep in (type-safe) templates so likely no one has ever read that deep into the code. But it means that I have to actually, y'know, write a scintillating and generally spectacular article to address those criticisms. Perhaps that's why I can't seem to write short blog posts, only book chapters. See -- even this comment is long!?!

So that's why.

But read my response to "Guest" below 'cause I did provide some details :)

Re: Avoiding game crashes related to linked lists

Peter Cormack — Fri, 26 Apr 2013 17:11:00 -0000

Whatever happened to part 3? I was looking forward to it. ^_^

Re: Avoiding game crashes related to linked lists

Patrick Wyatt — Mon, 15 Apr 2013 12:57:45 -0000

You're correct that I'm using the lowest bit of the next pointer as a sentinel to indicate end-of-list. Incidentally this is the same trick that the Boost library uses, though Boost wraps it in several layers of template indirection so it's hard to see right off.

You're absolutely correct that, for this trick to work, it's necessary for the memory manager to only return memory that's aligned on an even memory address, and further that any structures which are defined using #pragma pack(1) must carefully align members to ensure that linked list nodes (TLink<t>) do not start add odd memory addresses.

Any general-purpose memory manager which returns allocations that are not aligned on *at least* a four-byte boundary would be very poorly written. There are substantial CPU clock-cycle penalties for reading and writing non-aligned memory on some processors (ex: writing a dword to 0x.....3), and some processors will even generate an exception. Specialized memory allocators can return whatever they want, of course, but would generally be used for byte-aligned data if they're not guaranteeing alignment.

In regard to #pragma pack(1) structures, they should only be used for serializing data to disk or network where it wouldn't make sense to include a list node. In the event that you do need to use a pack(1) struct just make sure that the list-node fields are dword-aligned (on 32 bit) or quad-word-aligned (on 64 bit).

Incidentally, the linked-list code I wrote uses assertions in three TLink functions to ensure that link pointers are never stored at odd addresses.</t>

Re: Avoiding game crashes related to linked lists

Guest — Mon, 15 Apr 2013 10:20:35 -0000

With all due respect, if I understood your implementation correctly, it seems like your implementation is making the "next" pointer as an odd numbered address (by adding 1) to indicate that the pointer is an un-linked pointer. It is for sure if byte alignment is even numbered byte alignment such as 2 or 4 (8 or 16) byte alignment, however I was wondering if your implementation still satisfies when the byte alignment is 1 byte alignment (ex. #pragma pack(1) ).

Re: Avoiding game crashes related to linked lists

Guest — Mon, 15 Apr 2013 03:27:25 -0000

With all due respect, if I understood your implementation correctly, it seems like your implementation is making the "next" pointer as an odd numbered address ( by adding 1) to indicate that the pointer is an un-linked pointer.
It is for sure if byte alignment is even numbered byte alignment such as 2 or 4 (8 or 16), however I was wondering if your implementation also satisfies when byte alignment is 1 byte alignment ( ex. #pragma pack(1) )

Re: Using transaction rate-limiting to improve service reliability

Chris Hecker — Fri, 12 Apr 2013 14:49:03 -0000

Ah, I see it now, I was more confused than I thought! I was missing that the currTimeMS in the second if-statement is eating away at the m_timeMS as time passes, which is the "leak". Thanks, very nice!

Re: Using transaction rate-limiting to improve service reliability

Patrick Wyatt — Fri, 12 Apr 2013 01:42:30 -0000

Ah, I see your point. Fortunately it is possible to know the time until trying another login: 30 seconds. If I login 10 times in the first second, I have to wait 300 seconds until the bucket is empty (so I could try another 10 logins), but only 30 seconds until I can try the one more login. It really is the leaky bucket algorithm!

Re: Using transaction rate-limiting to improve service reliability

Chris Hecker — Thu, 11 Apr 2013 12:42:50 -0000

Yeah, making it scalable is an important thing, but for my current usage I just need a simple single-threaded rate limiter like this. Hopefully I'll need to make it scalable at some point. :)

My point about the max was just that it acts differently than the 'real' leaky bucket algorithm, because there isn't actually a way to define the timeout in the same way. With leaky bucket, you can immediately log in once more after the advertised timeout even if you were rate limited by the bucket filling (but your ability to burst again is limited for a longer period), whereas here the timeout for even a single login actually depends on how big your burst was, so it'd be hard for a user to know when they should try again (or for the login page to even say what it was). The advantage here is you are only storing a single int and also not having to tick the counter...it seems like to implement the real leaky bucket you either need to tick it or store two ints, one to track when the last call was so you can leak the right amount out of the buffer on the next call. But, it seems like the real behavior is slightly preferable since the timeout is predictable.

Anyway, it took me a while to figure out why I was confused by your code after reading the leaky bucket wikipedia page, so hopefully these comments help people from The Future! You might want to make a note in the text near the wikipedia link or something.

Thanks again!

Re: Using transaction rate-limiting to improve service reliability

Patrick Wyatt — Thu, 11 Apr 2013 00:38:36 -0000

Hola Chris; good to hear from you again.

You're right: if you sent 10 login requests in 1 second you'd have to wait another 299 seconds before you could attempt another login.

Another way to look at it is that the algorithm allows a long-term sustained rate of of 1 login attempt per 30 seconds, but it allows enough variability that someone could attempt up to 10 in one brief interval, as many folks do when they try to guess which of their several mostest favoritest passwords they actually used for the site.

Incidentally, these days I'd do the whole thing a bit differently. Since it's necessary to run a whole slew of servers for scalability and fault-tolerance, I'd actually use something like a Redis cluster to store the CRateLimiter values, and use a Redis Lua script to perform the same calculation as I've done here in C. By setting the Redis key-expiration time, the CRateLimiter values fall out of memory when they're no longer needed.

Re: Using transaction rate-limiting to improve service reliability

Chris Hecker — Wed, 10 Apr 2013 16:21:33 -0000

Hey Pat, thanks for posting this (also, hi!). I'm a bit confused by something, though. It seems this algorithm is different from Leaky Bucket because your max-cost burst limit is also your cooldown period. In other words, if somebody spams 10 logins with your constants above, then it will take 300 seconds before they can login again, whereas with the constant "drip" from Leaky Bucket, they'd be able to try a single login again in 30 seconds. Am I missing something?

Re: Avoiding game crashes related to linked lists

Konstantin Oznobikhin — Tue, 02 Apr 2013 16:05:20 -0000

Isn't this thread-safe deletion just wrong? If you need locking there it means object is being deleted multiple times. Can't imagine how it could be corrected without introducing new issues.

Still thanks for a great article. It's astonishing how quickly standard containers become unusable when performance is really important (and that's why we use C++ afterall).