Smaller than milliseconds?
Dar Scott
dsc at swcp.com
Fri Jul 11 15:13:00 EDT 2003
On Friday, July 11, 2003, at 12:54 PM, Ray G. Miller wrote:
> Adding three more zeros yielded:
> 1057948840.14027500152587890625000
> 1057948841.41388595104217529296880
> 1057948842.74860799312591552734380
> 1057948848.56447696685791015625000
> 1057948852.67774403095245361328120
> 1057948979.99965500831604003906250
> 1057948984.09295594692230224609380
> 1057948985.36483705043792724609380
> 1057948998.36404299736022949218750
> 1057948999.02833700180053710937500
^
And right about here (depending on how quoting is handled in your mail
client and whether you use a fixed-pitch font) is a column that
consists of only 0's and 9's. I think that means the real resolution
is 1 microsecond. The rest is all binary-to-decimal. The binary
floating point can only represent values to about 1/9 of a microsecond.
The extra digits are the results of the number not being able to get
closer than 1/9 microsecond.
Dar Scott
More information about the metacard
mailing list