Python Subprocess.Popen from a thread

I’m trying to launch an ‘rsync’ using subprocess module and Popen inside of a thread. After I call the rsync I need to read the output as well. I’m using the communicate method to read the output. The code runs fine when I do not use a thread. It appears that when I use a thread it hangs on the communicate call. Another thing I’ve noticed is that when I set shell=False I get nothing back from the communicate when running in a thread.

Answers:

Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.

Method 1

You didn’t supply any code for us to look at, but here’s a sample that does something similar to what you describe:

import threading
import subprocess

class MyClass(threading.Thread):
    def __init__(self):
        self.stdout = None
        self.stderr = None
        threading.Thread.__init__(self)

    def run(self):
        p = subprocess.Popen('rsync -av /etc/passwd /tmp'.split(),
                             shell=False,
                             stdout=subprocess.PIPE,
                             stderr=subprocess.PIPE)

        self.stdout, self.stderr = p.communicate()

myclass = MyClass()
myclass.start()
myclass.join()
print myclass.stdout

Method 2

Here’s a great implementation not using threads:
constantly-print-subprocess-output-while-process-is-running

import subprocess

def execute(command):
    process = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
    output = ''

    # Poll process for new output until finished
    for line in iter(process.stdout.readline, ""):
        print line,
        output += line


    process.wait()
    exitCode = process.returncode

    if (exitCode == 0):
        return output
    else:
        raise Exception(command, exitCode, output)

execute(['ping', 'localhost'])

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0