DPlot Forum Index DPlot
http://www.dplot.com
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

NANs in your text files?

 
Post new topic   Reply to topic    DPlot Forum Index -> General discussion
View previous topic :: View next topic  
Author Message
DPlotAdmin
Site Admin


Joined: 24 Jun 2003
Posts: 2310
Location: Vicksburg, Mississippi

PostPosted: Tue Apr 10, 2012 7:13 pm    Post subject: NANs in your text files? Reply with quote

DPlot has for a while interpreted the character string "NAN" (or "nan" or "NaN") as "not a number": divide by 0, log(0), or other math error. And it treats those labels as numbers, though it just skips over them. It doesn't restrict the search to the first position because some programs output, for example, "!NAN!" or some variation.

This all works well unless you have a column of labels containing the sequence "NAN" (or any of the variations mentioned above) in one or more rows. In that case if you have, say, 3 columns with the last column containing labels, and in that third column is a "NAN" in any row, DPlot sees the sequence in your file:
number,number,label
number,number,number (line containing NAN)
number,number,label
... and this causes it to start over and assume the data starts in the row following the NAN, since the data types don't match up. One user sent me a file today with the label "KONAN" and later let me know the label might be "NANCY", which fouled up my first solution. Both of these fail in version 2.3.2.3 and earlier, for reasons mentioned above.

What I want to do is lock down what character sequences other programs might output as NaN's and ignore the rest.

So if you have a program that is outputting NaN's, what exactly is the format? What I'm leaning towards is checking for "NAN", "nan" or "NaN" at any position. If it starts in the first position OR if the previous character is non-alpha, AND the character following that sequence is non-alpha or the end-of-line, that's a NaN. But before doing this I want to make sure I'm not overlooking something.

Thanks for your time.
_________________
Visualize Your Data
support@dplot.com
Back to top
View user's profile Send private message Send e-mail Visit poster's website
jsc



Joined: 02 Dec 2005
Posts: 222

PostPosted: Wed Apr 11, 2012 2:40 pm    Post subject: Reply with quote

Campbell Scientific export files contain #QNAN when data is out of range.

I'll send some small pieces of some old files that I have.

Jon

(I also found a scanned copy of the data file format from 2007, and I'm thinking that you may already have a copy?)
Back to top
View user's profile Send private message
DPlotAdmin
Site Admin


Joined: 24 Jun 2003
Posts: 2310
Location: Vicksburg, Mississippi

PostPosted: Wed Apr 11, 2012 11:29 pm    Post subject: Reply with quote

Thanks, Jon. I'm glad you spoke up because QNAN would have broken what I had in mind. I think I'll probably have to handle this on a case-by-case basis.
_________________
Visualize Your Data
support@dplot.com
Back to top
View user's profile Send private message Send e-mail Visit poster's website
PCJimmmy



Joined: 12 Jan 2009
Posts: 12

PostPosted: Sat Apr 06, 2013 7:05 pm    Post subject: NaN Issues with Visual Basic 2012 Express Reply with quote

David

Getting an error that kills DPlot when sending XYZ data using DPlot_Plot8 that contains NaN - Z values were the NaN's. It was low on my list of things to fix but I had the issue in both the x86 and more recently x64 libraries. As you know I needed to change my data array when I started using the x64 - so pretty confident that it was not just me Smile

Can't tell from this thread if NaN was just handled in reading of files or if you also expected it to be handled when using the libraries. I don't really now how VB2012 is sending the NaN but it certainly is a standard NaN text when I save a file containing my data.

As NaN was an indicator that I had something wrong in my program the DPlot crash did alert me to a possible issue - my program does a lot of trig calculations and I had a significant digits issue with very small angles in VB.

If you can't replicate this issue by breaking one of your VB examples (I used the examples for my code flow) let me know and I will try to break an example in VB2012.
Back to top
View user's profile Send private message
PCJimmmy



Joined: 12 Jan 2009
Posts: 12

PostPosted: Sat Apr 06, 2013 7:06 pm    Post subject: NaN Issues with Visual Basic 2012 Express Reply with quote

David

Getting an error that kills DPlot when sending XYZ data using DPlot_Plot8 that contains NaN - Z values were the NaN's. It was low on my list of things to fix but I had the issue in both the x86 and more recently x64 libraries. As you know I needed to change my data array when I started using the x64 - so pretty confident that it was not just me Smile

Can't tell from this thread if NaN was just handled in reading of files or if you also expected it to be handled when using the libraries. I don't really now how VB2012 is sending the NaN but it certainly is a standard NaN text when I save a file containing my data.

As NaN was an indicator that I had something wrong in my program the DPlot crash did alert me to a possible issue - my program does a lot of trig calculations and I had a significant digits issue with very small angles in VB.

If you can't replicate this issue by breaking one of your VB examples (I used the examples for my code flow) let me know and I will try to break an example in VB2012.
Back to top
View user's profile Send private message
PCJimmmy



Joined: 12 Jan 2009
Posts: 12

PostPosted: Sat Apr 06, 2013 7:06 pm    Post subject: NaN Issues with Visual Basic 2012 Express Reply with quote

David

Getting an error that kills DPlot when sending XYZ data using DPlot_Plot8 that contains NaN - Z values were the NaN's. It was low on my list of things to fix but I had the issue in both the x86 and more recently x64 libraries. As you know I needed to change my data array when I started using the x64 - so pretty confident that it was not just me Smile

Can't tell from this thread if NaN was just handled in reading of files or if you also expected it to be handled when using the libraries. I don't really now how VB2012 is sending the NaN but it certainly is a standard NaN text when I save a file containing my data.

As NaN was an indicator that I had something wrong in my program the DPlot crash did alert me to a possible issue - my program does a lot of trig calculations and I had a significant digits issue with very small angles in VB.

If you can't replicate this issue by breaking one of your VB examples (I used the examples for my code flow) let me know and I will try to break an example in VB2012.
Back to top
View user's profile Send private message
PCJimmmy



Joined: 12 Jan 2009
Posts: 12

PostPosted: Sat Apr 06, 2013 7:08 pm    Post subject: NaN Issues with Visual Basic 2012 Express Reply with quote

David

Getting an error that kills DPlot when sending XYZ data using DPlot_Plot8 that contains NaN - Z values were the NaN's. It was low on my list of things to fix but I had the issue in both the x86 and more recently x64 libraries. As you know I needed to change my data array when I started using the x64 - so pretty confident that it was not just me Smile

Can't tell from this thread if NaN was just handled in reading of files or if you also expected it to be handled when using the libraries. I don't really now how VB2012 is sending the NaN but it certainly is a standard NaN text when I save a file containing my data.

As NaN was an indicator that I had something wrong in my program the DPlot crash did alert me to a possible issue - my program does a lot of trig calculations and I had a significant digits issue with very small angles in VB.

If you can't replicate this issue by breaking one of your VB examples (I used the examples for my code flow) let me know and I will try to break an example in VB2012.
Back to top
View user's profile Send private message
DPlotAdmin
Site Admin


Joined: 24 Jun 2003
Posts: 2310
Location: Vicksburg, Mississippi

PostPosted: Sat Apr 06, 2013 7:14 pm    Post subject: Reply with quote

"I don't really now how VB2012 is sending the NaN"

I don't either. If you can send me source that might be helpful. For what it's worth, these are completely different problems. Your data is being sent to DPlot in its native Double format, not as a text string. If you tried to access that value in your VB program you'd most likely get a show-stopping error (though I may be wrong about that).

In my defense, the dplotlib routines assume that a developer isn't trying to send garbage to DPlot Very Happy
_________________
Visualize Your Data
support@dplot.com
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic    DPlot Forum Index -> General discussion All times are GMT - 5 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group