Beginner with panda dataframes. I have this data set below with missing values for column A and B (Test.csv):
DateTime A B 01-01-2017 03:27 01-01-2017 03:28 01-01-2017 03:29 0.18127718 -0.178835737 01-01-2017 03:30 0.186923018 -0.183260853 01-01-2017 03:31 01-01-2017 03:32 01-01-2017 03:33 0.18127718 -0.178835737
I can use this code to fill in values using forward propagation, but this only fills in for 03:31 and 03:32, and not 03:27 and 03:28.
import pandas as pd
import numpy as np
df = pd.read_csv('test.csv', index_col = 0)
data = df.fillna(method='ffill')
ndata = data.to_csv('test1.csv')
results in:
DateTime A B
01-01-2017 03:27
01-01-2017 03:28
01-01-2017 03:29 0.18127718 -0.178835737
01-01-2017 03:30 0.186923018 -0.183260853
01-01-2017 03:31 0.186923018 -0.183260853
01-01-2017 03:32 0.186923018 -0.183260853
01-01-2017 03:33 0.18127718 -0.178835737
How could I include the ‘Bfill’ to fill in the missing values for 03:27 and 03:28 using the backfil?
Answers:
Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.
Method 1
You can use ffill and bfill if need replace NaN values forward and backward filling:
print (df)
A B
DateTime
01-01-2017 03:27 NaN NaN
01-01-2017 03:28 NaN NaN
01-01-2017 03:29 0.181277 -0.178836
01-01-2017 03:30 0.186923 -0.183261
01-01-2017 03:31 NaN NaN
01-01-2017 03:32 NaN NaN
01-01-2017 03:33 0.181277 -0.178836
data = df.ffill().bfill()
print (data)
A B
DateTime
01-01-2017 03:27 0.181277 -0.178836
01-01-2017 03:28 0.181277 -0.178836
01-01-2017 03:29 0.181277 -0.178836
01-01-2017 03:30 0.186923 -0.183261
01-01-2017 03:31 0.186923 -0.183261
01-01-2017 03:32 0.186923 -0.183261
01-01-2017 03:33 0.181277 -0.178836
Which is same as the function fillna with parameters:
data = df.fillna(method='ffill').fillna(method='bfill')
All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0