View previous topic :: View next topic |
Author |
Message |
toddbatzler
Joined: 28 Aug 2009 Posts: 7
|
Posted: Fri Aug 28, 2009 2:16 pm Post subject: Generate->Y=f(X) X interval calculation off by 1 |
|
|
DPlot Version 2.2.6.5
Set from X = 0, to X = 0.049999, dx = 1e-006 and number of points reported is 50001. Generate waveform, edit data and waveform was generated up to and including X= 0.05, not 0.049999
Same behavior if numbers 0.1 to 0.199999 dx = 1e-6, but ok with 0.2 to 0.299999 or any subsequent interval where tenths digit is >1. |
|
Back to top |
|
|
DPlotAdmin Site Admin
Joined: 24 Jun 2003 Posts: 2310 Location: Vicksburg, Mississippi
|
Posted: Fri Aug 28, 2009 3:17 pm Post subject: |
|
|
The trouble (in your example) is that even with double-precision floats, 49999 * 1.E-6 is equal to (roughly) 0.049999 - 6.9E-18. That's of course less than 0.049999, so a point is added. I'll modify the test to add a small (very small) bit of slop. _________________ Visualize Your Data
support@dplot.com |
|
Back to top |
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|