Test Everything Easily in Python

How to use the coverage library to check which lines of code you are testing

Image by Author

Suppose you have just finished creating the project that will make you rich. You want everything to be perfect, so you wrote numerous tests, and they all passed. But then, on the day of the presentation, a bug occurs exactly when you are showing your project to the public for the first time.

How is that possible? Your tests passed, so shouldn’t everything be correct?

Well, the truth is that tests cannot prove that your program is bug-free: they just show that the situations that you tested do not occur with bugs. This means that there is no way for you to be absolutely certain that no bug will occur, but at the same time, it doesn’t mean that you cannot do anything about it.

The most important thing you can do is to try to test as much of your code as possible. But, especially with large programs, it is difficult to understand which lines of code you are testing and which you have forgotten.

Luckily, the Coverage library will do exactly that, and in this article, I will show you how to use it.

The Program

First of all, we need a program to test. I will use a really simple one, but it should be enough for you to understand the capabilities of this library.

Here is the code, which I saved in a file named main.py :

INFINITY = 10e9
def addition(a, b):
  return a+b
def subtraction(a, b):
  return a-b
def division(a, b):
  if b == 0:
    return INFINITY
  return a/b

The tests

Then, we will also need some tests. I will use the unittest library, but you can also use a different one.

If you don’t know how to test a python program, check this article:

Test Your Python Program the Right Way

For now, let’s just create (inside a file named tests.py) a code for testing the addition function:

import main
import unittest
class OperationTests(unittest.TestCase):
  def testAdd(self):
    self.assertEqual(main.addition(1, 2), 3)
if __name__ == '__main__':
  unittest.main()

Using coverage

Now is time to use the coverage library! first of all, we need to install it using pip:

pip install coverage

Then, we can use the coverage run command in the command line to run the tests and record how much of the code is actually executed.

To do so, you can just replace python with coverage run in the command that you use to execute the tests. for example, instead of using python tests.py we will execute:

coverage run tests.py

This will execute your tests, but also create a file named .coverage .

To see the results of this, we need to execute a new command:

coverage report

The output will be:

Name       Stmts   Miss  Cover
------------------------------
main.py        9      4    56%
tests.py       7      0   100%
------------------------------
TOTAL         16      4    75%

What does this mean? We have executed code from two files, tests.py and main.py . Inside tests.py we have executed every line of code, so we have covered 100% of the program, while in main.py we have only covered 56% of the code.

This is what one would expect since we have only tested the addition function. But knowing just the p[ercentage is not really useful: we also want to know exactly which lines are executed and which are not. To see this, execute this other command:

coverage html

This will create a htmlcov folder, with various files inside. Open the index.html in your browser. It will show this page:

Image by Author

This shows the same information as the command line. however, you can click on the name of a file to see the details. To try this, click on main.py .Thi sis what appears:

Image by Author

Here we can see that some lines are red: these are the parts of the code that are not executed by our test. As one would expect, the code inside the subtraction and division functions are not executed.

To solve this, let’s add two new tests inside tests.py :

def testSubtraction(self):
  self.assertEqual(main.subtraction(3, 2), 1)
def testDivision(self):
  self.assertEqual(main.division(4, 2), 2)

Now we can run again the commands to see if we are testing everything:

coverage run tests.py
coverage html

Now we can open again the details page:

Image by Author

We have reached a coverage of 89%. This is better than before, but we still need to check one case.

It is clear from the code that we are not testing when the division function returns infinity. Let’s add one last test:

def testDivisionByZero(self):
  self.assertEqual(main.division(3, 0), main.INFINITY)

Again, we run the tests. Now we have obtained coverage of 100%:

coverage run tests.py
coverage html

Image by Author

Excluding some files

We have seen how to check the coverage of our tests. However, you may have noticed that the coverage library was also checking the tests.py file. this doesn’t make much sense: we do not want to check this file since it is not part of our final program.

Luckily, coverage.py provides a configuration file where you can specify which files should be skipped.

Create a file .coveragerc and add inside the following lines:

[run]
omit = tests.py

Now if you run coverage , it will not record the values for the tests.py file:

coverage run tests.py
coverage report
Name      Stmts   Miss  Cover
-----------------------------
main.py       9      0   100%
-----------------------------
TOTAL         9      0   100%

If you want to learn more about the configurations for this library, check the official documentation:

Configuration reference - Coverage.py 6.3.2 documentation

Conclusion

Thank you for reading through to the end! I hope this article will help you write better tests.

If you want more information about this library (or testing in general), check out these links:

• Coverage.py official documentation
• Testing with unittest tutorial

More content at PlainEnglish.io. Sign up for our free weekly newsletter. Follow us on Twitter and LinkedIn. Join our community Discord.