DPlot Forum Index DPlot
http://www.dplot.com
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Error 7

 
Post new topic   Reply to topic    DPlot Forum Index -> Bug Reports
View previous topic :: View next topic  
Author Message
Phillip Mathis



Joined: 10 Dec 2009
Posts: 21

PostPosted: Tue Aug 17, 2010 8:25 pm    Post subject: Error 7 Reply with quote

I'm working with large .csv files (above 500,000 rows) and when I try to plot more than three columns from either Dplot or Excel I'm prompted with a run time error 7 "Not enough Memory". Is this a limitation of Dplot, lack of RAM(2G installed), or an Excel limitation?

Is there a work-around, besides splitting the files?
Back to top
View user's profile Send private message
DPlotAdmin
Site Admin


Joined: 24 Jun 2003
Posts: 2310
Location: Vicksburg, Mississippi

PostPosted: Tue Aug 17, 2010 9:06 pm    Post subject: Reply with quote

Definitely not a DPlot limitation, though the Add-In may be fouling something up. What version of Excel? 32-bit or 64-bit?
_________________
Visualize Your Data
support@dplot.com
Back to top
View user's profile Send private message Send e-mail Visit poster's website
DPlotAdmin
Site Admin


Joined: 24 Jun 2003
Posts: 2310
Location: Vicksburg, Mississippi

PostPosted: Tue Aug 17, 2010 9:29 pm    Post subject: Reply with quote

FYI I just now tried a sheet with 5 columns of 500,000 points and the XYYY command. It took at least a minute, but it worked: in DPlot I get 4 curves with 500,000 points each. This is with Excel 2010 64-bit. When saved as a CSV file DPlot read the file, no problem.

If there's any more information you can give me about what you're trying to do, what Add-In command you're using, etc., that might be helpful.

When reading a CSV file directly into DPlot, if you know how many data rows are in the file then you should first double-click the "Array size" area of the status bar and enter values large enough to accommodate your data. The reason for that is that if, for example, the max. number of data points is N and DPlot reads N rows but doesn't reach the end of the file, it estimates how many rows are in the file based on the file size and how many bytes were processed for those N rows. It might get this badly wrong if the first N rows use many fewer characters per row than the remainder of the file, and so overestimate the required number of points.

2Gb should be plenty for what you're talking about, assuming you aren't running everything under the sun. With the example above, Task Manager shows DPlot eating up ~46Mb.
_________________
Visualize Your Data
support@dplot.com
Back to top
View user's profile Send private message Send e-mail Visit poster's website
DPlotAdmin
Site Admin


Joined: 24 Jun 2003
Posts: 2310
Location: Vicksburg, Mississippi

PostPosted: Wed Aug 18, 2010 8:14 am    Post subject: Reply with quote

PS: With that same sheet open in Excel, Excel eats up 242Mb. You should still be able to do this on a 2Gb system. If not you might want to try rebooting. If that doesn't help then you'll obviously be better off reading the CSV file directly into DPlot with Excel closed.
_________________
Visualize Your Data
support@dplot.com
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Phillip Mathis



Joined: 10 Dec 2009
Posts: 21

PostPosted: Wed Aug 18, 2010 8:33 pm    Post subject: Error 7 Reply with quote

David, my computer crashed. I reloaded Dplot and got updates on my new computer(Windows 7 64 bit). I bought office 2010 and loaded it on my computer. I added Dplot as an add-in and this when i started having this problem. I was not having this problem with 2007.

When I adjusted the array size, Dplot will now plot these large files from the import function, but when plotting an XYYY from the Excel add-in it still gives the error 7 code when plotting more than 3 columns. The Debug function on the error form highlights at the ReDim X(DataRowCount * NumCurves -1) in DplotData.XLA...

I have feeling that I haven't set something up correctly in 2010. Just noticed that Dplot is in xla, and if I remember correctly 2007 is xls. Am I having a backwards compatibilty problem? Anyway, I hope this is the info you need.

Thanks
Back to top
View user's profile Send private message
DPlotAdmin
Site Admin


Joined: 24 Jun 2003
Posts: 2310
Location: Vicksburg, Mississippi

PostPosted: Wed Aug 18, 2010 9:00 pm    Post subject: Reply with quote

Quote:
I have feeling that I haven't set something up correctly in 2010. Just noticed that Dplot is in xla, and if I remember correctly 2007 is xls. Am I having a backwards compatibilty problem? Anyway, I hope this is the info you need.


The Add-In has always been an XLA file, not XLS. It is possible that I sent you an XLS file previously to help debug something. But there's nothing magic about an XLA file other than once set up it is automatically loaded when you start Excel.

"Error 7" is an Excel thing, nothing to do with DPlot. It indicates that you have insufficient memory for the operation intended. (And really, 2Gb is pretty lean for Windows 7.) On my same setup, for example (same OS and same Excel version) I have 6Gb. I know that doesn't make your day, sorry.

You can kinda sorta predict this with Task Manager. It will show how much memory Excel is gobbling up and you can also tell pretty quickly how much memory you need to send the data to DPlot. dplot.exe with a small default array size takes ~9Mb. The data you want to plot will require NC*NP*16 bytes, where NC is the number of curves and NP is the number of points/curve.

The bad thing (for low memory situations) is that the requirements are at least doubled when sending data to DPlot from Excel: Excel owns one copy, DPlot the other. Actually it's quite a bit worse than that as Excel stores more information than just the raw data values (formulas, formatting, etc.). As mentioned previously, a data set that takes ~46Mb in DPlot takes ~240Mb in Excel. This relationship will vary depending on the Add-In command used, but the important part is that the total memory requirement will be, at the very least, twice what is eventually required in DPlot. So, short version, if you're running out of memory, save your sheet as a CSV file, exit Excel, and import that file directly into DPlot (for XYXY, XYYY, XYZSurface. For other data types... no surefire solution other than installing more RAM.)

I'll at least think about intercepting that particular error and offering suggestions within the Add-in. For the XYYY, XYXY, and XYZSurface commands the solution is fairly simple: save as CSV, exit Excel, import into DPlot. Some of the other Add-In functions aren't that straightforward, but those 3 will cover the majority of cases (he says, crossing his fingers).
_________________
Visualize Your Data
support@dplot.com
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Phillip Mathis



Joined: 10 Dec 2009
Posts: 21

PostPosted: Thu Aug 19, 2010 7:07 am    Post subject: Error 7 Reply with quote

Got it, Thanks David.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    DPlot Forum Index -> Bug Reports All times are GMT - 5 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group