I am looking for a gradient pattern on my dataframe as follows:
df.loc[(
(df['A'].shift(-0).lt(1.7)) &
(df['A'].shift(-1).lt(1.7)) &
(df['A'].shift(-2).lt(1.7)) &
(df['A'].shift(-3).lt(1.7)) &
(df['A'].shift(-4).lt(1.7)) &
(df['A'].shift(-5).lt(1.7)) &
]
The latter will return a df where 6 previous values are smaller than 1.7 for example:
the data frame will look like this (before and after):
A
329 15.1252
330 13.1251
331 1.3125
332 1.5625
333 39.5625
346 45.6875
347 11.0000
348 11.0000
354 1.8125
355 1.8125
358 1.4375
359 1.4375
360 1.5000
361 1.5000
362 1.5000
363 1.5000
364 1.4375
365 1.4375
366 1.5000
A
364 1.4375
365 1.4375
366 1.5000
It works but I want to improve it. I tried many things, I think it could be something like:
parameters = [
[0, 1.7],
[1, 1.7],
[2, 1.7],
[3, 1.7],
[4, 1.7],
[5, 1.7],
]
conditions = ([ ' & ' .join(['(df["A"].shift(-{0}).lt({1}))'.format(x[0], x[1]) for x in parameters])])
conditions = '(' + conditions ')'
df.loc[conditions]
It seems that ‘conditons’ is returned as string between quotes litterally as ‘conditions’, so df.loc[conditions] returns a ‘KeyError’
Is my first question on the website. Thanks in advance.
Answers:
Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.
Method 1
You can try use reduce on list
from functools import reduce
m = reduce((lambda df1, df2: df1&df2), [df['A'].shift(-s).lt(v) for s, v in parameters])
print(df.loc[m])
A
358 1.4375
359 1.4375
360 1.5000
361 1.5000
Method 2
One solution, could be to concatenate all the shifts, check where the values are less than 1.7 and then keep the ones where all the shifts pass the test.
cond = pd.concat((df["A"].shift(-i) for i in range(6)), axis=1).lt(1.7).all(axis=1) df.loc[cond]
Gives
A 358 1.4375 359 1.4375 360 1.5000 361 1.5000
All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0