‘yes’ reporting error with subprocess communicate()

I am using the following function to run a command in Python:

def run_proc(cmd):
    child = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
    stdout, stderr = child.communicate()
    returncode = child.returncode
    return stdout, stderr, returncode

It has always been working fine, however now I’m trying to use the yes program to pipe output to stdin. The command I’m trying to run is the following:

yes '' | apt-get -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" dist-upgrade

but I believe it could be substituted with a general example, like:

yes | head -3 | cat

My problem is that if I try to run any command which has yes | in it, the above subprocess.Popen will contain the error messages:

yes: standard output: Broken pipe
yes: write error

For me it seems that the piping still works, as can be seen from yes | head -3 | cat‘s answer: y y y.

I have the following questions:

1. Is the yes piping still functional, even though yes reports error?
2. How can I fix it?

Answers:

Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.

Method 1

The issue is that subprocess module before Python 3.2+ doesn’t restore SIGPIPE signal handler to default action. That is why you get EPIPE write error instead.

In Python 3.2+

>>> from subprocess import check_output
>>> check_output("yes | head -3", shell=True)
b'ynynyn'

yes is killed by SIGPIPE when head exits.

In Python 2:

>>> from subprocess import check_output
>>> check_output("yes | head -3", shell=True)
yes: standard output: Broken pipe
yes: write error
'ynynyn'

yes got EPIPE write error. It is safe to ignore the error. It communicates the same information as SIGPIPE.

To workaround the problem, you could emulate restore_signals in Python 2 using preexec_fn parameter :

>>> from subprocess import check_output
>>> import signal
>>> def restore_signals(): # from http://hg.python.org/cpython/rev/768722b2ae0a/
...     signals = ('SIGPIPE', 'SIGXFZ', 'SIGXFSZ')
...     for sig in signals:
...         if hasattr(signal, sig):
...            signal.signal(getattr(signal, sig), signal.SIG_DFL)
... 
>>> check_output("yes | head -3", shell=True, preexec_fn=restore_signals)
'ynynyn'

Method 2

the other question answers the why … I’ll try and give you a work around

could you not do something like

proc = subprocess.Popen(cmd, shell=True, 
                             stdout=subprocess.PIPE, 
                             stderr=subprocess.PIPE,
                             stdin = subprocess.PIPE)

for i in range(10):  #send 10 y's
    time.sleep(1) # 1 second apart
    proc.stdin.write("y") #you may also need to send a newline ...

print proc.communicate()

see below (I didnt bother with the delay since head isnt really doing much)

>>> import subprocess
>>> proc = subprocess.Popen("head -3",
...                          shell = True,
...                          stdout = subprocess.PIPE,
...                          stderr=subprocess.PIPE,
...                          stdin=subprocess.PIPE)
>>> for i in range(10):
...    proc.stdin.write("yn")
...
>>> proc.communicate()
('ynynyn', '')

Method 3

Saying:

yes | head -3

causes head to send a SIGPIPE to yes once it’s done reading 3 lines of input, i.e. it’d send a signal to terminate yes.

$ yes | head -3
y
y
y
$ echo "${PIPESTATUS[@]}"
141 0

The solution would be to avoid a SIGPIPE!

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0