Popen with conflicting executable/path

I’d like to call the “convert” utility from ImageMagick from my Python script using Popen, like so:

Popen(["convert", input_path, "-flop", output_file_path])

(The above example simply reverses the image horizontally)

The problem is that when I run the script in Windows, it mistakenly calls the convert.exe utility that ships with Windows to convert FAT partitions to NTFS! (located in Windowssystem32)

Now, if I randomly open a command prompt at any directory other than system32, and type “convert”, it correctly runs the ImageMagick executable. So, this implies that Popen is automatically looking in system32. How can I make it not look in system32, and run the correct executable?

Answers:

Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.

Method 1

Search for a program is not trivial. I’d specify the full path to the convert.exe executable explicitly instead.

subprocess uses CreateProcess on Windows that looks in system32 directory even before any other directory in %PATH%:

… If the file name does not contain an extension, .exe is appended.
Therefore, if the file name extension is .com, this parameter must
include the .com extension. If the file name ends in a period (.) with
no extension, or if the file name contains a path, .exe is not
appended. If the file name does not contain a directory path, the
system searches for the executable file in the following sequence:

1. The directory from which the application loaded.
2. The current directory for the parent process.
3. The 32-bit Windows system directory. Use the GetSystemDirectory function to get the path of this directory.
4. The 16-bit Windows system directory. There is no function that obtains the path of this directory, but it is searched. The name of this directory is System.
5. The Windows directory. Use the GetWindowsDirectory function to get the path of this directory.
6. The directories that are listed in the PATH environment variable. Note that this function does not search the per-application path specified by the App Paths registry key. To include this per-application path in the search sequence, use the ShellExecute function.

Therefore convert is equivalent to convert.exe in this case. It first looks in a directory that contains sys.executable e.g., C:Python27. Then in the current directory: where you started the Python script from. Then in system32 where it finds convert.exe (filesystem utility, not imagemagick).

You could try to remove system32 directory from os.environ['PATH'] it may(?) suppress checking it: Popen(cmd, env=no_system32_environ) but it is fragile (worse than the explicit path).

There is a related issue on Python bug tracker: “Subprocess picks the wrong executable on Windows.”

cmd.exe (the shell) uses different algorithm. See How does Windows locate files input in the shell?

If you set shell=True then the search sequence for convert program:

1. convert is not an internal shell command
2. there is no explicit path, so the search continues
3. search the current directory
4. search each directory specified by the PATH environment variable, in the order listed

%PATHEXT% defines which file extensions are checked and in what order e.g., convert.com, convert.exe, convert.bat, convert.cmd if %PATHEXT% is .com;.exe;.bat;.cmd.

Method 2

As a completely different approach, you may want to try out PythonMagick, a Python wrapper for ImageMagick. This way you can access convert‘s functions from within Python, and you won’t have to spawn outside processes.

Method 3

Just ran into this myself. Running a php script in a shell has access to the shell $PATH, while running within Apache does not (at least not a $PATH with the Cygwin dll/exe).

Secondly, use the native Windows notation C:/dir/subdir/pgm within php when on a windows platform and you will get what you expect.

So multi-platform code needs to switch/case the pathing and then reference the decision.

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0