Debugging ASP.NET memory leaks

by benl

A site we develop here had suddenly experienced a spike in usage and as a result had begun to (noticeably, since the rise in traffic) hemorrhage memory and ultimately crash. Nice! The site in question interfaces with an Autonomy back-end and the results of search queries issued to this server were being cached via the ASP.NET caching API. On entry to the cache the items were set to expire two hours from now. This being fine under low to moderate load but given the load the site has been experiencing of late the problem was compounded and the cache became rather large, rather quick!

It took some time to determine the cause of the leak – which had manifested itself as a huge amount of objects sitting around in the GC Gen 2 collection. However, by grabbing a non-intrusive memory dump using the Debugging Tools for Windows and the adplus.vbs script and then mining the dump using the WinDbg tool, the cause became quite apparent. It was clear that a lot of objects of a particular type (those returned from the Autonomy service) were the offenders. After changing the caching to use sliding rather than static expiration the problem was resolved.

This article is a great starting point for anyone interested in WinDbg. I've also found the "If broken it is, fix it you should" blog to be an excellent resource regarding WinDbg.

UPDATE: Useful WinDbg/SOS command reference 

Advertisements