Handling Large Data Sets in ASP.NET and Crystal Reports

I receive a lot of feedback from my ASP Alliance articles, usually comments.  I don’t typically answer via e-mail, since that doesn’t help the community.  Today I received the following message, and felt it was blog worthy.

Referring URL: http://aspalliance.com/532
Dear Rechard,

I have always been using Datasets and they have given me consistent Results. In fact, I DO NOT USE Report Viewer but export CR Report to a PDF File and then give it to the user as a URL.

BUT, WHEN DATA IS VERY LARGE, THE DATASETS GIVE PROBLEMS such as out of Memory, server not available, Timeouts, hanging etc. I have asked this question to many forums as to how to handle large data sets in ASP.Net with CR.Net. But, nobody has given me any satisfying answer.

I really want to know as to how to solve this problem.

Will You please Help Me.

With Warm Regards,

DN

Here’s the problem–the amount of memory which can be utilized by an ASP.NET web app is 2GB (3GB if you use the /3GB switch).  If your dataset exceeds 2GB on a Windows 2003 Server, you will get a Server Not Available error.  Less than but close to 2GB will cause a significant decrease in performance or warnings about memory.

The answer is simple–you need a smaller dataset!  A 2GB dataset is far too large, especially when you’re feeding it into Crystal Reports.  It sounds like you are dumping every bit of raw data into the report, and having the report process and present the data.  You really need to have your database queries doing more of the filtering and processing, then return a significantly smaller dataset.  In a web environment, it’s best if you only do presentation in your reports–leave the calculations and filtering to the database.