Rolling window for 1D arrays in Numpy?

Is there a way to efficiently implement a rolling window for 1D arrays in Numpy?

For example, I have this pure Python code snippet to calculate the rolling standard deviations for a 1D list, where observations is the 1D list of values, and n is the window length for the standard deviation:

stdev = []
for i, data in enumerate(observations[n-1:]):
    strip = observations[i:i+n]
    mean = sum(strip) / n
    stdev.append(sqrt(250*sum([(s-mean)**2 for s in strip])/(n-1)))

Is there a way to do this completely within Numpy, i.e., without any Python loops? The standard deviation is trivial with numpy.std, but the rolling window part completely stumps me.

I found this blog post regarding a rolling window in Numpy, but it doesn’t seem to be for 1D arrays.

Answers:

Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.

Method 1

Just use the blog code, but apply your function to the result.

i.e.

numpy.std(rolling_window(observations, n), 1)

where you have (from the blog):

def rolling_window(a, window):
    shape = a.shape[:-1] + (a.shape[-1] - window + 1, window)
    strides = a.strides + (a.strides[-1],)
    return np.lib.stride_tricks.as_strided(a, shape=shape, strides=strides)

Method 2

Starting in Numpy 1.20, you can directly get a rolling window with sliding_window_view:

from numpy.lib.stride_tricks import sliding_window_view

sliding_window_view(np.array([1, 2, 3, 4, 5, 6]), window_shape = 3)
# array([[1, 2, 3],
#        [2, 3, 4],
#        [3, 4, 5],
#        [4, 5, 6]])

Method 3

I tried using so12311‘s answer listed above on a 2D array with shape [samples, features] in order to get an output array with shape [samples, timesteps, features] for use with a convolution or lstm neural network, but it wasn’t working quite right. After digging into how the strides were working, I realized that it was moving the window along the last axis, so I made some adjustments so that the window is moved along the first axis instead:

def rolling_window(a, window_size):
    shape = (a.shape[0] - window_size + 1, window_size) + a.shape[1:]
    strides = (a.strides[0],) + a.strides
    return np.lib.stride_tricks.as_strided(a, shape=shape, strides=strides)

NOTE: there is no difference in the output if you are only using a 1D input array. In my search this was the first result to get close to what I wanted to do, so I am adding this to help any others searching for a similar answer.

Method 4

With only one line of code…

import pandas as pd

pd.Series(observations).rolling(n).std()

Method 5

Based on latter answers, here I add code for rolling 1-D numpy arrays choosing window size and window steps frequency.

a = np.arange(50)

def rolling_window(array, window_size,freq):
    shape = (array.shape[0] - window_size + 1, window_size)
    strides = (array.strides[0],) + array.strides
    rolled = np.lib.stride_tricks.as_strided(array, shape=shape, strides=strides)
    return rolled[np.arange(0,shape[0],freq)]

rolling_window(a,10,5)

Output:

array([[ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9],
       [ 5,  6,  7,  8,  9, 10, 11, 12, 13, 14],
       [10, 11, 12, 13, 14, 15, 16, 17, 18, 19],
       [15, 16, 17, 18, 19, 20, 21, 22, 23, 24],
       [20, 21, 22, 23, 24, 25, 26, 27, 28, 29],
       [25, 26, 27, 28, 29, 30, 31, 32, 33, 34],
       [30, 31, 32, 33, 34, 35, 36, 37, 38, 39],
       [35, 36, 37, 38, 39, 40, 41, 42, 43, 44],
       [40, 41, 42, 43, 44, 45, 46, 47, 48, 49]])

Method 6

def moving_avg(x,n):
    mv =  np.convolve(x,np.ones(n)/n,mode='valid')
    return np.concatenate(([np.NaN for k in range(n-1)],mv))

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0