I719 Fundamentals of Python/test results

Test Result Output

Example Code

app.py

def get_hello_world_1():
    return "Hello World!"


def get_hello_world_2():
    """Returns wrong text"""
    return "Wrong Text."


def get_hello_world_3():
    """Will raise an error when run"""
    return hello_world

test_app.py

import app
import unittest


class HelloWorld1TestCase(unittest.TestCase):
    def test_value(self):
        result = app.get_hello_world_1()
        self.assertEqual("Hello World!", result)


class HelloWorld2TestCase(unittest.TestCase):
    def test_value(self):
        result = app.get_hello_world_2()
        self.assertEqual("Hello World!", result)


class HelloWorld3TestCase(unittest.TestCase):
    def test_value(self):
        result = app.get_hello_world_3()
        self.assertEqual("Hello World!", result)

Output

.FE
======================================================================
ERROR: test_value (test_app.HelloWorld3TestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/test_app.py", line 19, in test_value
    result = app.get_hello_world_3()
  File "/tmp/app.py", line 12, in get_hello_world_3
    return hello_world
NameError: name 'hello_world' is not defined

======================================================================
FAIL: test_value (test_app.HelloWorld2TestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/test_app.py", line 14, in test_value
    self.assertEqual("Hello World!", result)
AssertionError: 'Hello World!' != 'Wrong Text.'
- Hello World!
+ Wrong Text.


----------------------------------------------------------------------
Ran 3 tests in 0.001s

FAILED (failures=1, errors=1)

Overview

• . is a passing test
• F is a failure
• E is an error

NOTE: The test results do not appear in a consistent order.

Passing Test

Since the assertion -- assertEqual -- is called with 'Hello World!' and 'Hello World!', no error is raised.

Failing Test

FAIL: test_value (test_app.HelloWorld2TestCase)

• FAIL, means the test failed
• test_value is the name of the method in the TestCase
• test_app.HelloWorld2TestCase is the python path to the TestCase

The assertion -- assertEqual -- is called with 'Hello World!' and 'Wrong Text.'. These two strings are not equal, so assertEqual raises an AssertionError. When an AssertionError is raised, the test fails.

You will see in the FAIL test result the values used. This is helpful to figure out what went wrong.

Error

The function -- get_hello_world_3 raises a NameError, since the name 'hello_world' is not defined. When an error is raised, and the error is NOT an AssertionError, then the test result is a special failure called an "error".