I have a DataFrame in pandas where some of the numbers are expressed in scientific notation (or exponent notation) like this:
id value id 1.00 -4.22e-01 value -0.42 1.00e+00 percent -0.72 1.00e-01 played 0.03 -4.35e-02 money -0.22 3.37e-01 other NaN NaN sy -0.03 2.19e-04 sz -0.33 3.83e-01
And the scientific notation makes what should be an easy comparison, needlessly difficult. I assume it’s the 21900 value that’s screwing it up for the others. I mean 1.0 is encoded. ONE!
This doesn’t work:
np.set_printoptions(supress=True)
And pandas.set_printoptions doesn’t implement suppress either, and I’ve looked all at pd.describe_options() in despair, and pd.core.format.set_eng_float_format() only seems to turn it on for all the other float values, with no ability to turn it off.
Answers:
Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.
Method 1
quick temporary: df.round(4)
global: pd.options.display.float_format = '{:20,.2f}'.format
Method 2
Your data is probably object dtype. This is a direct copy/paste of your data. read_csv interprets it as the correct dtype. You should normally only have object dtype on string-like fields.
In [5]: df = read_csv(StringIO(data),sep='s+')
In [6]: df
Out[6]:
id value
id 1.00 -0.422000
value -0.42 1.000000
percent -0.72 0.100000
played 0.03 -0.043500
money -0.22 0.337000
other NaN NaN
sy -0.03 0.000219
sz -0.33 0.383000
check if your dtypes are object
In [7]: df.dtypes Out[7]: id float64 value float64 dtype: object
This converts this frame to object dtype (notice the printing is funny now)
In [8]: df.astype(object)
Out[8]:
id value
id 1 -0.422
value -0.42 1
percent -0.72 0.1
played 0.03 -0.0435
money -0.22 0.337
other NaN NaN
sy -0.03 0.000219
sz -0.33 0.383
This is how to convert it back (astype(float)) also works here
In [9]: df.astype(object).convert_objects()
Out[9]:
id value
id 1.00 -0.422000
value -0.42 1.000000
percent -0.72 0.100000
played 0.03 -0.043500
money -0.22 0.337000
other NaN NaN
sy -0.03 0.000219
sz -0.33 0.383000
This is what an object dtype frame would look like
In [10]: df.astype(object).dtypes Out[10]: id object value object dtype: object
Method 3
Try this which will give you scientific notation only for large and very small values (and adds a thousands separator unless you omit the “,”):
pd.set_option('display.float_format', lambda x: '%,g' % x)
Or to almost completely suppress scientific notation without losing precision, try this:
pd.set_option('display.float_format', str)
Method 4
If you would like to use the values as formated string in a list, say as part of csvfile csv.writier, the numbers can be formated before creating a list:
df['label'].apply(lambda x: '%.17f' % x).values.tolist()
Method 5
quick fix without rounding:
pd.options.display.float_format = '{:.0f}'.format
Method 6
I tried all the options like
- pd.options.display.float_format = ‘{:.4f}’.format
- pd.set_option(‘display.float_format’, str)
- pd.set_option(‘display.float_format’, lambda x: f’%.{len(str(x%1))-2}f’ % x)
- pd.set_option(‘display.float_format’, lambda x: ‘%.3f’ % x)
but nothing worked for me.
so while assigning the variable / value (var1) to a variable (say num1) I used round(val,5).
num1 = round(var1,5)
This is a crude method as you have to use this round function in each assignment. But this ensures you control on how it happens and get what you want.
All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0