How to distinguish different types of NaN float in Python

Question

I'm writing Python 2.6 code that interfaces with NI TestStand 4.2 via COM in Windows. I want to make a "NAN" value for a variable, but if I pass it float('nan'), TestStand displays it as IND.

Apparently TestStand distinguishes between floating point "IND" and "NAN" values. According to TestStand help:

• IND corresponds to Signaling NaN in Visual C++, while
• NAN corresponds to QuietNaN

That implies that Python's float('nan') is effectively a Signaling NaN when passed through COM. However, from what I've read about Signaling NaN, it seems that Signaling NaN is a bit "exotic" and Quiet NaN is your "regular" NaN. So I have my doubts that Python would be passing a Signaling NaN through COM. How could I find out if a Python float('nan') is passed through COM as a Signaling NaN or Quiet NaN, or maybe Indeterminate?

Is there any way to make a Signaling NaN versus QuietNaN or Indeterminate in Python, when interfacing with other languages? (Using ctypes perhaps?) I assume this would be a platform-specific solution, and I'd accept that in this case.

Update: In the TestStand sequence editor, I tried making two variables, one set to NAN and the other set to IND. Then I saved it to a file. Then I opened the file and read each variable using Python. In both cases, Python reads them as a nan float.

Answer 1

I dug a bit for you, and I think you might be able to use the struct module in combination with the information on at Kevin's Summary Charts. They explain the exact bit patterns used for the various kinds of IEEE 754 floating point numbers.

The only thing you probably will have to be careful for, if I read the topics on this IND-eterminate value, is that that value tends to trigger some kind of floating point interrupt when assigned directly in C code, causing it to be turned into a plain NaN. Which in turn meant those people were advised to do this kind of thing in ASM rather than C since C abstracted that stuff away.. Since it is not my field, and that I am not sure to what extent this kind of value would mess with Python, I figured I'd mention it so you can at least keep an eye for any such weird behaviour. (See the accepted answer for this question).

>>> import struct

>>> struct.pack(">d", float('nan')).encode("hex_codec")
'fff8000000000000'

>>> import scipy
>>> struct.pack(">d", scipy.nan).encode("hex_codec")
'7ff8000000000000'

Referring to Kevin's Summary Charts, that shows that float('nan') is actually technically the Indeterminate value, while scipy.nan is a Quiet NaN.

Let's try making a Signaling NaN, and then verify it.

>>> try_signaling_nan = struct.unpack(">d", "\x7f\xf0\x00\x00\x00\x00\x00\x01")[0]
>>> struct.pack(">d", try_signaling_nan).encode("hex_codec")
'7ff8000000000001'

No, the Signaling NaN gets converted to a Quiet NaN.

Now let's try making a Quiet NaN directly, and then verify it.

>>> try_quiet_nan = struct.unpack(">d", "\x7f\xf8\x00\x00\x00\x00\x00\x00")[0]
>>> struct.pack(">d", try_quiet_nan).encode("hex_codec")
'7ff8000000000000'

So that's how to make a proper Quiet NaN using struct.unpack()--at least, on a Windows platform.

Answer 2

CPython definition of nan

When Python reports a nan, where does that come from?

• Result of a calculation (platform specific values?)
• Py_NAN in the CPython C source code
  • defined as (Py_HUGE_VAL * 0.)
    • Value is platform-specific
    • Py_HUGE_VAL is probably defined as HUGE_VAL--it has a note to say it should be HUGE_VAL except on platforms where that is broken.
• float('nan') which is defined from Py_NAN in CPython's C source code.

Reading Python and pywin32 Source Code

I've had a look at the C source code for pywin32, in particular win32com, which forms the Python↔COM translation layer. That code:

• takes the input object
• calls PyNumber_Float() to convert it to a Python float (if it isn't already)
• calls PyFloat_AsDouble() to convert it to a plain C double value.
  • This simply returns the C double directly contained in the PyFloatObject member ob_fval.

So it looks as though I've traced a NaN from the COM interface back to a plain C double type containing Py_NAN, whatever that turns out to be on the Windows platform.

TestStand NAN Value

Now I've tried this with NI TestStand. First I tried:

quiet_nan = struct.unpack(">d", "\x7f\xf8\x00\x00\x00\x00\x00\x01")[0]
# Set the variable's value in TestStand
locals_prop_object.SetValNumber(var_name, 0, quiet_nan)

But that still appeared in TestStand as IND. So then I created a TestStand file with variables set to IND and NAN, and read the values from Python. It turns out that TestStand's NAN has a value of FFFF000000000001. According to Kevin's Summary Charts that is a negative quiet NAN. TestStand's IND does have the expected value for Indeterminate, FFF8000000000000.

Success

So, after all that, I have succeeded in setting a NAN in TestStand, from Python:

# Make a NAN suitable for TestStand
teststand_nan = struct.unpack(">d", "\xff\xff\x00\x00\x00\x00\x00\x01")[0]
# Set the variable's value in TestStand
locals_prop_object.SetValNumber(var_name, 0, teststand_nan)

Answer 3

John Cook had a nice post on this that might be helpful:

• IEEE floating point arithmetic in Python

Update: won't this work?

In [144]: import scipy

In [145]: scipy.nan
Out[145]: 1.#QNAN

In [146]: scipy.inf
Out[146]: 1.#INF

In [147]: scipy.inf * 0
Out[147]: -1.#IND

Answer 4

From what I can gather, it seems there is some confusion as to thinking that the sign of a NaN determines whether or not it is quiet. On the contrary, the convention is that the most significant bit of the mantissa determines this. From Wikipedia (emphasis added):

In IEEE 754 standard-conforming floating-point storage formats, NaNs are identified by specific, pre-defined bit patterns unique to NaNs. The sign bit does not matter. Binary format NaNs are represented with the exponential field filled with ones (like infinity values), and some non-zero number in the significand field (to make them distinct from infinity values). The original IEEE 754 standard from 1985 (IEEE 754-1985) only described binary floating-point formats, and did not specify how the signaling/quiet state was to be tagged. In practice, the most significant bit of the significand field determined whether a NaN is signaling or quiet... The 2008 revision of the IEEE 754 standard (IEEE 754-2008) makes formal recommendations for the encoding of the signaling/quiet state. For binary formats, the most significant bit of the significand field should be an 'is_quiet' flag. I.e. this bit is non-zero if the NaN is quiet, and zero if the NaN is signaling.

Since most implementations are IEEE 754-2008 conformant, this is the convention you should follow. In general, you cannot plan on the sign bit being consistent for NaNs, even for different NaNs on the same platform. Under this convention, float('nan') and scipy.nan both seem to be quiet NaNs, at least in the cases discussed above.