Extents precision

rborn · Post by **rborn** » Wed May 19, 2004 12:20 pm

Ok, your going to hate me now.

When it brings in the time for the X axis, it evidently converts it to an integer. I know this since the extents it uses are 0.2 and 0.8 for the time (X axis). How it actually displays the time correctly is beyond me but impressive.

So evidently you divide the actual time by 24 to get the integer value, Excel works the same way. Here is where I'm wondering if there is a workaround. My problem is that I'd like to round the extents off to round hours, specifically 5:00:00 am and 6:00:00 pm. If you do the math, this gives you 0.2083 and 0.75 for integer values. It seems the extents get rounded to the 10th's position so the values it rounds to are 0.2 and 0.8 respectively, which comes out to 4:48 am and 7:12 pm, not exactly what I was looking for.

Any suggestions?

Post by **DPlotAdmin** » Wed May 19, 2004 4:59 pm

Ok, your going to hate me now.

Nah. You'll have to try much harder

The Add-In passes all data as 4-byte (single precision) floating point numbers, never as integers. The stair-stepping you see is just due to a lack of precision. The time right now in Vicksburg, MS is 4:29 PM on May 19, 2004. In Excel serial number-speak that's 38126.6868055556. At that magnitude, the resolution is approximately 0.00227253 days, or about 3.27 minutes. In other words 5/19/04 16:29 is indistinguishable from 5/19/04 16:32. Using double precision the resolution is about 4.23E-12, or about 0.36 microseconds... quite a difference. With any luck I should have the double precision version of DPLOTLIB.DLL available very soon, so all of these problems will go away.

You should see that if you save your Excel file as .CSV and read it in DPlot then the precision is preserved.

As for rounding off values, I wouldn't want to do that in the Add-In. Instead, something I've meant to do for a while is add truncation and rounding operators to Y=f(X) and the other equation parsing routines. To round a date to the nearest hour, for example, you'd use "Operate on X" with X=round(X*24)/24.

rborn · Post by **rborn** » Wed May 19, 2004 5:21 pm

I'm not sure if you miss-understood me or I am just confused. I agree you would not want to mess with Add-in for rounding and I don't want to round my numbers.

I'll try to explain it another way. I go to Options, Extent/Tick Marks/Size. My X axis with the time is a bit uneven, so to speak, with the Automatic scaling (extents). It starts at 4:48:00 AM and ends at 7:12:00 PM. I'd like it to be more rounded and have the X-Axis start at 5:00:00 AM and end at 6:00:00 PM. There is no data before 5:00 AM or after 6:00 PM and this just narrows down the time and makes everything nice and clean, to me anyway. I think you can undersand that it would be easier to read an axis from 0 to 10 than from -0.134 to 12.567, for an extreme example. The Automatic function does a nice job, but I'm sure you put in the extents and such to customize it for this exact reason.

So the problem I have is, the Low and High extents only hold a x.x number, if I add a number such as 0.2834, it will become 0.2. With the time on the X-axis this will never allow me to get exactly 5:00:00 AM. I said in my last message what the decimal number is for that but I don't have it now...I guess it would be 5/24 or 0.2083, which always rounds to 0.2, which is 4:48:00 AM, kind of an unround number, for time anyway.

If your suggestion in the last post was to fix this, I guess I didn't understand it.

PS: I did try to add that function but it gave me "Syntax error in function", I simply added "round(X*24)/24" to the X= box.

Post by **DPlotAdmin** » Wed May 19, 2004 6:00 pm

Ah... well I completely misunderstood. That dialog needs a bit of work for times. Right now you have to enter the extents as days, and whatever you enter will very likely be rounded off to something else, dependent on whether you also specified a tick mark interval. I'll change this to input date + hour... or something.

Post by **DPlotAdmin** » Wed May 19, 2004 6:47 pm

BTW you can do what you want now, though it's a bit inconvenient. In addition to specifying the extents you need to also set the tick mark interval on the X axis to 0.0416666666666666666667. The extents will then be integer multiples of 1 hour.