DPlot

Author Message
toddbatzler

Joined: 28 Aug 2009
Posts: 7

 Posted: Fri Aug 28, 2009 2:16 pm    Post subject: Generate->Y=f(X) X interval calculation off by 1 DPlot Version 2.2.6.5 Set from X = 0, to X = 0.049999, dx = 1e-006 and number of points reported is 50001. Generate waveform, edit data and waveform was generated up to and including X= 0.05, not 0.049999 Same behavior if numbers 0.1 to 0.199999 dx = 1e-6, but ok with 0.2 to 0.299999 or any subsequent interval where tenths digit is >1.

Joined: 24 Jun 2003
Posts: 2023
Location: Vicksburg, Mississippi

 Posted: Fri Aug 28, 2009 3:17 pm    Post subject: The trouble (in your example) is that even with double-precision floats, 49999 * 1.E-6 is equal to (roughly) 0.049999 - 6.9E-18. That's of course less than 0.049999, so a point is added. I'll modify the test to add a small (very small) bit of slop._________________David Hyde support@dplot.com
 Display posts from previous: All Posts1 Day7 Days2 Weeks1 Month3 Months6 Months1 Year Oldest FirstNewest First
 All times are GMT - 5 Hours Page 1 of 1