subprocess wildcard usage

import os

import subprocess

proc = subprocess.Popen(['ls','*.bc'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)

out,err = proc.communicate()

print out

This script should print all the files with .bc suffix however it returns an empty list. If I do ls *.bc manually in the command line it works. Doing [‘ls’,’test.bc’] inside the script works as well but for some reason the star symbol doesnt work.. Any ideas ?

Answers:

Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.

Method 1

You need to supply shell=True to execute the command through a shell interpreter.
If you do that however, you can no longer supply a list as the first argument, because the arguments will get quoted then. Instead, specify the raw commandline as you want it to be passed to the shell:

 proc = subprocess.Popen('ls *.bc', shell=True,
                                    stdout=subprocess.PIPE,
                                    stderr=subprocess.PIPE)

Method 2

Expanding the * glob is part of the shell, but by default subprocess does not send your commands via a shell, so the command (first argument, ls) is executed, then a literal * is used as an argument.

This is a good thing, see the warning block in the “Frequently Used Arguments” section, of the subprocess docs. It mainly discusses security implications, but can also helps avoid silly programming errors (as there are no magic shell characters to worry about)

My main complaint with shell=True is it usually implies there is a better way to go about the problem – with your example, you should use the glob module:

import glob
files = glob.glob("*.bc")
print files # ['file1.bc', 'file2.bc']

This will be quicker (no process startup overhead), more reliable and cross platform (not dependent on the platform having an ls command)

Method 3

Besides doing shell=True, also make sure that your path is not quoted. Otherwise it will not be expanded by shell.

If your path may have special characters, you will have to escape them manually.

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0