Using win32com with multithreading

I am working on a web app with CherryPy that needs to access a few applications via COM.

Right now I create a new instance of the application with each request, which means each request waits 3 seconds for the application to start and 0.01 for the actual job.

I would like to start each COM application once and keep it alive and reuse it for a few seconds on the following requests because most of the time it is used by a burst of 5-10 ajax requests, then nothing for hours.

Is it possible to share a COM abject across all the threads of a CherryPy application?

Here is the summary of a few experiments that show how it is working now on each request and how it does not work across threads.

The following code successfully starts and stops Excel:

>>> import pythoncom, win32com.client
>>> def start():
    global xl
    xl = win32com.client.Dispatch('Excel.Application')

>>> def stop():
    global xl
    xl.quit()
    xl = None

>>> start()
>>> stop()

But the following code starts Excel and closes it after 3 seconds.

>>> import pythoncom, win32com.client, threading, time
>>> def start():
    global xl
    pythoncom.CoInitialize()
    xl = win32com.client.Dispatch('Excel.Application')
    time.sleep(3)

>>> threading.Thread(target=start).start()

I added the call to CoInitialize() otherwise the xl object would not work (see this post).

And I added the 3 second pause, so I could see on the task manager that the EXCEL.EXE process starts and is alive for 3 seconds.

Why does it die after the thread that started it ends?

I checked the documentation of CoInitialize(), but I couldn’t understand if it is possible to get it to work in multithreaded environment.

Answers:

Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.

Method 1

If you want to use win32com in multiple threads you need to do a little bit more work as COMObject cannot be passed to a thread directly. You need to use CoMarshalInterThreadInterfaceInStream() and CoGetInterfaceAndReleaseStream() to pass instance between threads:

import pythoncom, win32com.client, threading, time

def start():
    # Initialize
    pythoncom.CoInitialize()

    # Get instance
    xl = win32com.client.Dispatch('Excel.Application')

    # Create id
    xl_id = pythoncom.CoMarshalInterThreadInterfaceInStream(pythoncom.IID_IDispatch, xl)

    # Pass the id to the new thread
    thread = threading.Thread(target=run_in_thread, kwargs={'xl_id': xl_id})
    thread.start()

    # Wait for child to finish
    thread.join()

def run_in_thread(xl_id):
    # Initialize
    pythoncom.CoInitialize()

    # Get instance from the id
    xl = win32com.client.Dispatch(
            pythoncom.CoGetInterfaceAndReleaseStream(xl_id, pythoncom.IID_IDispatch)
    )
    time.sleep(5)


if __name__ == '__main__':
    start()

For more info see: https://mail.python.org/pipermail/python-win32/2008-June/007788.html

Method 2

The answer from @Mauriusz Jamro was really helpful. Just to add to it, also ensure that you do:

pythoncom.CoUninitialize ()

in the end so that there’s no memory leak. You can call it somewhere after using CoInitialize() and before your process ends.

P.S.- I didn’t have enough reputation to comment under his answer, so had to resort to writing a new answer. Cheers!

Method 3

Try using multiprocessing. Worked for me, after a long search.

from multiprocessing import Process

p = Process(target=test, args=())
p.start()
p.join()

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0