Screen Capture with OpenCV and Python-2.7

I’m using Python 2.7 and OpenCV 2.4.9.

I need to capture the current frame that is being shown to the user and load it as an cv::Mat object in Python.

Do you guys know a fast way to do it recursively?

I need something like what’s done in the example below, that captures Mat frames from a webcam recursively:

import cv2

cap = cv2.VideoCapture(0)
while(cap.isOpened()):
    ret, frame = cap.read()
    cv2.imshow('WindowName', frame)
    if cv2.waitKey(25) & 0xFF == ord('q'):
        cap.release()
        cv2.destroyAllWindows()
        break

In the example it’s used the VideoCapture class to work with the captured image from the webcam.

With VideoCapture.read() a new frame is always being readed and stored into a Mat object.

Could I load a “printscreens stream” into a VideoCapture object? Could I create a streaming of my computer’s screen with OpenCV in Python, without having to save and delete lots of .bmp files per second?

I need this frames to be Mat objects or NumPy arrays, so I can perform some Computer Vision routines with this frames in real time.

Answers:

Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.

Method 1

That’s a solution code I’ve written using @Raoul tips.

I used PIL ImageGrab module to grab the printscreen frames.

import numpy as np
from PIL import ImageGrab
import cv2

while(True):
    printscreen_pil =  ImageGrab.grab()
    printscreen_numpy =   np.array(printscreen_pil.getdata(),dtype='uint8')
    .reshape((printscreen_pil.size[1],printscreen_pil.size[0],3)) 
    cv2.imshow('window',printscreen_numpy)
    if cv2.waitKey(25) & 0xFF == ord('q'):
        cv2.destroyAllWindows()
        break

Method 2

I had frame rate problems with other solutions, mss solve them.

import numpy as np
import cv2
from mss import mss
from PIL import Image

mon = {'top': 160, 'left': 160, 'width': 200, 'height': 200}

sct = mss()

while 1:
    sct.get_pixels(mon)
    img = Image.frombytes('RGB', (sct.width, sct.height), sct.image)
    cv2.imshow('test', np.array(img))
    if cv2.waitKey(25) & 0xFF == ord('q'):
        cv2.destroyAllWindows()
        break

Method 3

This is the updated answer for the answer by @Neabfi

import time

import cv2
import numpy as np
from mss import mss

mon = {'top': 160, 'left': 160, 'width': 200, 'height': 200}
with mss() as sct:
    # mon = sct.monitors[0]
    while True:
        last_time = time.time()
        img = sct.grab(mon)
        print('fps: {0}'.format(1 / (time.time()-last_time)))
        cv2.imw('test', np.array(img))
        if cv2.waitKey(25) & 0xFF == ord('q'):
            cv2.destroyAllWindows()
            break

And to save to a mp4 video

import time

import cv2
import numpy as np
from mss import mss


def record(name):
    with mss() as sct:
        # mon = {'top': 160, 'left': 160, 'width': 200, 'height': 200}
        mon = sct.monitors[0]
        name = name + '.mp4'
        fourcc = cv2.VideoWriter_fourcc(*'mp4v')
        desired_fps = 30.0
        out = cv2.VideoWriter(name, fourcc, desired_fps,
                              (mon['width'], mon['height']))
        last_time = 0
        while True:
            img = sct.grab(mon)
            # cv2.imshow('test', np.array(img))
            if time.time() - last_time > 1./desired_fps:
                last_time = time.time()
                destRGB = cv2.cvtColor(np.array(img), cv2.COLOR_BGRA2BGR)
                out.write(destRGB)
            if cv2.waitKey(25) & 0xFF == ord('q'):
                cv2.destroyAllWindows()
                break


record("Video")

Method 4

Here’s the Python 3 Implementation

This Function finds the Application in the List of Running Applications:

def capture_dynamic():
    toplist, winlist = [], []
    
    def enum_cb(hwnd, results):
        winlist.append((hwnd, win32gui.GetWindowText(hwnd)))
        
    win32gui.EnumWindows(enum_cb, toplist)

    wnd = [(hwnd, title) for hwnd, title in winlist if 'spotify' in title.lower()]

    if wnd:
        wnd = wnd[0]
        hwnd = wnd[0]

        bbox = win32gui.GetWindowRect(hwnd)
        img = ImageGrab.grab(bbox)
        return img
    else:
        return None

This Function Displays the Images until the Letter ‘q’ is pressed:

import cv2
import numpy as np

while(True):
#   Dynamic Version
    screen_grab =  capture_dynamic()
    
    if(screen_grab == None):
        print("No Window Found! Please Try Again")
        break
        
    screen_grab = np.array(screen_grab)
    cv2.imshow('window',cv2.cvtColor(screen_grab, cv2.COLOR_BGR2RGB))
    
    if cv2.waitKey(25) & 0xFF == ord('q'):
        cv2.destroyAllWindows()
        break

Make sure the Application you want to capture should be in the Foreground and not behind any other Application

Please Upvote!

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0