How to redirect output with subprocess in Python?

What I do in the command line:

cat file1 file2 file3 > myfile

What I want to do with python:

import subprocess, shlex
my_cmd = 'cat file1 file2 file3 > myfile'
args = shlex.split(my_cmd)
subprocess.call(args) # spits the output in the window i call my python program

Answers:

Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.

Method 1

In Python 3.5+ to redirect the output, just pass an open file handle for the stdout argument to subprocess.run:

# Use a list of args instead of a string
input_files = ['file1', 'file2', 'file3']
my_cmd = ['cat'] + input_files
with open('myfile', "w") as outfile:
    subprocess.run(my_cmd, stdout=outfile)

As others have pointed out, the use of an external command like cat for this purpose is completely extraneous.

Method 2

UPDATE: os.system is discouraged, albeit still available in Python 3.

Use os.system:

os.system(my_cmd)

If you really want to use subprocess, here’s the solution (mostly lifted from the documentation for subprocess):

p = subprocess.Popen(my_cmd, shell=True)
os.waitpid(p.pid, 0)

OTOH, you can avoid system calls entirely:

import shutil

with open('myfile', 'w') as outfile:
    for infile in ('file1', 'file2', 'file3'):
        shutil.copyfileobj(open(infile), outfile)

Method 3

@PoltoS I want to join some files and then process the resulting file. I thought using cat was the easiest alternative. Is there a better/pythonic way to do it?

Of course:

with open('myfile', 'w') as outfile:
    for infilename in ['file1', 'file2', 'file3']:
        with open(infilename) as infile:
            outfile.write(infile.read())

Method 4

One interesting case would be to update a file by appending similar file to it. Then one would not have to create a new file in the process. It is particularly useful in the case where a large file need to be appended. Here is one possibility using teminal command line directly from python.

import subprocess32 as sub

with open("A.csv","a") as f:
    f.flush()
    sub.Popen(["cat","temp.csv"],stdout=f)

Method 5

size = 'ffprobe -v error -show_entries format=size -of default=noprint_wrappers=1:nokey=1 dump.mp4 > file'
proc = subprocess.Popen(shlex.split(size), shell=True)
time.sleep(1)
proc.terminate() #proc.kill() modify it by a suggestion
size = ""
with open('file', 'r') as infile:
    for line in infile.readlines():
        size += line.strip()

print(size)
os.remove('file')

When you use subprocess , the process must be killed.This is an example.If you don’t kill the process , file will be empty and you can read nothing.It can run on Windows.I can`t make sure that it can run on Unix.

Method 6

It will work if your args will look like ['sh', '-c', 'cat file1 file2 file3 > myfile'] it will mean that output of cat won’t pass Python and spawn in shell instead (insted of sh -c you can use bash -c)

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0