Build a VM Test Lab using Unraid - Working From Home Part I
Understanding how hypervisors work is key to setting up a test lab on a home server. In this article, read about lessons I learned setting up a VM test lab on Unraid.
Redgate's ANTS Memory Profiler is a fantastic tool to help track potential memory leaks. But, it is important to have realistic expectations when getting started using the tool. When used properly, the tool can be very helpful in tracking down memory leaks. By no means is it a silver bullet. There is a learning curve with the tool. It will do all the heavy lifting of profiling all the items in memory, but it will not point at a specific piece of code and tell you that is where the memory leak is happening.
A few weeks ago I was tasked with tracking down a potential memory leak for one of the applications at my new job. I say potential memory leak because this application has been rock solid for many years. There was some doubt of a memory leak, the only way to know for sure is to use on of the many memory profilers out there. ANTS Memory Profiler is among the best of the best in that field. One of the perks of being a "Friend of Redgate" is it includes free licenses to their various products. I downloaded Redgate's ANTS Memory Profiler and started digging into the problem.
It has been a while since I have had the chance to use this tool, or a memory profiler in general. The next series of articles will cover some of the lessons I had to relearn in order to use the tool effectively. This article will cover some of the fundamentals in memory profilers in general. Future articles will dive into the tool itself.
This one should seem fairly obvious, but I've been party to a number of times where the person investigating the memory leak didn't bother to do this step. Without it, they were just randomly clicking around the application.
Memory leak reports come in many forms. The two most common ones are instances reported by users or a rogue process consuming all the memory on a server causing alerts to fire. Server memory leaks are an extra joy to debug, it requires tooling such as NewRelic to find out what calls were being made on the server.
In the case where I was asked to help track down the memory leak, it was reported by users of the application. Having users report the issue can be very helpful, more people having a problem indicates a trend which can help track down the problem.
Some of the common questions I ask when a user reports a memory leak are:
Once you have that use case in place put together a test plan.
In the application I was looking at recently I noticed there were a lot of references to resource files. I thought that was odd, so I asked the development manager. He said each client of the application gets their own resource file, this is the application is sold as customizable to the client's liking. Not much I can do about that. So I moved on to other items.
Because of the repetitive manual testing, tracking down memory leaks is often assigned to more junior members of the development teams. Or newer members of a team. If you are on the side doing the assigning make yourself available to answer any and all questions about the application. If you are on the side where the assigning is happening to you pretend you are five years old and keep on asking why to the senior members of the team. Maybe the right question will be asked to trigger something.
Most memory leaks are very hard to discover. They would have shown up almost right away during testing if they were apparent. Memory profilers are not magic tools. You don't just fire them up and they will instantly point at the memory leak. Prepare yourself mentally for the long haul. I've looked at many memory leaks in the past with the expectation it would be solved in a couple of hours. That is almost never the case. It takes a long methodical process to find the cause of the memory leak. Don't be surprised if it takes days to uncover.
Have a good and repeatable test plan. And don't be afraid to alter the plan as more information comes to light. A couple of years ago I was tasked with finding a memory leak in a Single Page Application where it would only show up after several hours of usage when the user was using IE 11. When I asked the user how they were using the application they told me they just left it running on a particular page (it was a dashboard that auto refreshed). My plan was to take a memory snapshot at set intervals.
When you believe you have solved the problem then repeat the same test. It is super fun to do. Just like a root canal is fun. But it is the only way to know for certain if the fix took.
In this particular case, I was lucky because the application auto refreshed. Having to manually click through the application for that length of time would have driven me mad. If anything, it sure would have been a good case for automation!
I love that .NET handles all the memory allocation and garbage collection automatically. I didn't even bother to learn the basics of memory for a long time. I blissfully kept my head in the sand the first time I was assigned a task of tracking down a memory leak. I got a crash course in .NET memory management. It is not a very easy topic to wrap your head around, it takes some time to learn everything and as a result, it is something you should avoid having to do in the middle of a crisis.
Key topics to learn:
Anytime I go into an application looking for a memory leak I first look for
If you see a lot of these in the memory profiler it might be something worth looking into.
.NET manages the memory, that is one of the big perks of using that framework. The garbage collector was written by people much, much smarter than you or I. What I am trying to say is if you find yourself in a situation where you are looking into ways to trick the garbage collector or change how memory is allocated in .NET then stop what you are doing. Nothing good will come from that.
This article was written to provide a foundation for future articles. In my next article, I will walk through the ANTS memory profiler.