changing order of unit tests in Python

How can I make it so unit tests in Python (using unittest) are run in the order in which they are specified in the file?

Answers:

Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.

Method 1

You can change the default sorting behavior by setting a custom comparison function. In unittest.py you can find the class variable unittest.TestLoader.sortTestMethodsUsing which is set to the builtin function cmp by default.

For example you can revert the execution order of your tests with doing this:

import unittest
unittest.TestLoader.sortTestMethodsUsing = lambda _, x, y: cmp(y, x)

Method 2

Clever Naming.

class Test01_Run_Me_First( unittest.TestCase ):
    def test010_do_this( self ):
        assertTrue( True )
    def test020_do_that( self ):
        etc.

Is one way to force a specific order.

Method 3

As said above, normally tests in test cases should be tested in any (i.e. random) order.

However, if you do want to order the tests in the test case, apparently it is not trivial.
Tests (method names) are retrieved from test cases using dir(MyTest), which returns a sorted list of members. You can use a clever (?) hack to order methods by their line numbers. This will work for one test case:

if __name__ == "__main__":
    loader = unittest.TestLoader()
    ln = lambda f: getattr(MyTestCase, f).im_func.func_code.co_firstlineno
    lncmp = lambda a, b: cmp(ln(a), ln(b))
    loader.sortTestMethodsUsing = lncmp
    unittest.main(testLoader=loader, verbosity=2)

Method 4

There are also test runners which do that by themselves – I think py.test does it.

Method 5

Use proboscis library as I mentioned already (please see short description there).

Method 6

The default order is alphabetical. You can put test_<int> before every test to specify the order of execution.

Also you can set unittest.TestLoader.sortTestMethodsUsing = None to eliminate the sort.

Check out Unittest Documentation for more details.

Method 7

I found a solution for it using PyTest ordering plugin provided here.

Try py.test YourModuleName.py -vv in CLI and the test will run in the order they have appeared in your module.

I did the same thing and works fine for me.

Note: You need to install PyTest package and import it.

Method 8

hacky way (run this file in pycharm or other unit test runner)

#!/usr/bin/env python
# -*- coding: UTF-8 -*-
import unittest


def make_suite():
    class Test(unittest.TestCase):
        def test_32(self):
            print "32"
        def test_23(self):
            print "23"

    suite = unittest.TestSuite()
    suite.addTest(Test('test_32'))
    suite.addTest(Test('test_23'))
    return suite

suite = make_suite()

class T(unittest.TestCase):
    counter = 0
    def __call__(self, *args, **kwargs):
        res = suite._tests[T.counter](*args, **kwargs)
        T.counter += 1
        return res

for t in suite._tests:
    name = "{}${}".format(t._testMethodName, t.__class__.__name__)
    setattr(T, name, t)

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0