Pipe subprocess standard output to a variable

I want to run a command in pythong, using the subprocess module, and store the output in a variable. However, I do not want the command’s output to be printed to the terminal.
For this code:

def storels():
   a = subprocess.Popen("ls",shell=True)
storels()

I get the directory listing in the terminal, instead of having it stored in a. I’ve also tried:

 def storels():
       subprocess.Popen("ls > tmp",shell=True)
       a = open("./tmp")
       [Rest of Code]
 storels()

This also prints the output of ls to my terminal. I’ve even tried this command with the somewhat dated os.system method, since running ls > tmp in the terminal doesn’t print ls to the terminal at all, but stores it in tmp. However, the same thing happens.

Edit:

I get the following error after following marcog’s advice, but only when running a more complex command. cdrecord --help. Python spits this out:

Traceback (most recent call last):
  File "./install.py", line 52, in <module>
    burntrack2("hi")
  File "./install.py", line 46, in burntrack2
    a = subprocess.Popen("cdrecord --help",stdout = subprocess.PIPE)
  File "/usr/lib/python2.6/subprocess.py", line 633, in __init__
    errread, errwrite)
  File "/usr/lib/python2.6/subprocess.py", line 1139, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

Answers:

Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.

Method 1

To get the output of ls, use stdout=subprocess.PIPE.

>>> proc = subprocess.Popen('ls', stdout=subprocess.PIPE)
>>> output = proc.stdout.read()
>>> print output
bar
baz
foo

The command cdrecord --help outputs to stderr, so you need to pipe that indstead. You should also break up the command into a list of tokens as I’ve done below, or the alternative is to pass the shell=True argument but this fires up a fully-blown shell which can be dangerous if you don’t control the contents of the command string.

>>> proc = subprocess.Popen(['cdrecord', '--help'], stderr=subprocess.PIPE)
>>> output = proc.stderr.read()
>>> print output
Usage: wodim [options] track1...trackn
Options:
    -version    print version information and exit
    dev=target  SCSI target to use as CD/DVD-Recorder
    gracetime=# set the grace time before starting to write to #.
...

If you have a command that outputs to both stdout and stderr and you want to merge them, you can do that by piping stderr to stdout and then catching stdout.

subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)

As mentioned by Chris Morgan, you should be using proc.communicate() instead of proc.read().

>>> proc = subprocess.Popen(['cdrecord', '--help'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
>>> out, err = proc.communicate()
>>> print 'stdout:', out
stdout: 
>>> print 'stderr:', err
stderr:Usage: wodim [options] track1...trackn
Options:
    -version    print version information and exit
    dev=target  SCSI target to use as CD/DVD-Recorder
    gracetime=# set the grace time before starting to write to #.
...

Method 2

If you are using python 2.7 or later, the easiest way to do this is to use the subprocess.check_output() command. Here is an example:

output = subprocess.check_output('ls')

To also redirect stderr you can use the following:

output = subprocess.check_output('ls', stderr=subprocess.STDOUT)

In the case that you want to pass parameters to the command, you can either use a list or use invoke a shell and use a single string.

output = subprocess.check_output(['ls', '-a'])
output = subprocess.check_output('ls -a', shell=True)

Method 3

With a = subprocess.Popen("cdrecord --help",stdout = subprocess.PIPE)
, you need to either use a list or use shell=True;

Either of these will work. The former is preferable.

a = subprocess.Popen(['cdrecord', '--help'], stdout=subprocess.PIPE)

a = subprocess.Popen('cdrecord --help', shell=True, stdout=subprocess.PIPE)

Also, instead of using Popen.stdout.read/Popen.stderr.read, you should use .communicate() (refer to the subprocess documentation for why).

proc = subprocess.Popen(['cdrecord', '--help'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = proc.communicate()

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0