Auto Test Discovery Using Unittest

Anvil Q&A
#1

What I’m trying to do:
Discover and run unittest classes on the server automatically on app startup

What I’ve tried and what’s not working:

I have a sample test class defined as:

Code Sample:

import anvil.tables as tables
import anvil.tables.query as q
from anvil.tables import app_tables
import anvil.server
import unittest

class SampleTest(unittest.TestCase):
  
  def test_assert_is_positive_int_passing_1point1_fails(self):
    with self.assertRaises(Exception):
      assert_is_positive_int(1.1)

  def test_assert_is_positive_int_passing_None_fails(self):
    with self.assertRaises(Exception):
      assert_is_positive_int(None)

def assert_is_positive_int(num: int):
    assert isinstance(num, int), 'number ' + str(num) + " not of type int"
    assert num > 0, 'number ' + str(num) + " not greater than zero"

@anvil.server.callable
def server_auto_tests(verbosity=1):
  suite = unittest.TestLoader().discover('.', pattern = "_Test.py")
  runner = unittest.TextTestRunner(verbosity=verbosity)
  result = runner.run(suite)
  print(result.errors, result.failures)

And then I’m calling server_auto_tests in the first form.

The error log I’m getting is:
[(<unittest.loader._FailedTest testMethod=anvil.files>, ‘ImportError: Failed to import test module: anvil.files\nTraceback (most recent call last):\n File “/downlink/anvil/tables/v2/_app_tables.py”, line 26, in getattribute\n return self[name]\n File “/downlink/anvil/tables/v2/_app_tables.py”, line 32, in getitem\n table_args = cache[name]\nKeyError: 'files'\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File “/usr/local/lib/python3.7/unittest/loader.py”, line 470, in _find_test_path\n package = self._get_module_from_name(name)\n File “/usr/local/lib/python3.7/unittest/loader.py”, line 377, in _get_module_from_name\n import(name)\n File “/downlink/anvil/files/init.py”, line 34, in \n FILES_TABLE = getattr(app_tables, “files”) # TODO: Get table name from server config\n File “/downlink/anvil/tables/v2/_app_tables.py”, line 28, in getattribute\n return object.getattribute(self, name)\nAttributeError: 'AppTables' object has no attribute 'files'\n\n’), (<unittest.loader._FailedTest testMethod=anvil.users.mfa>, ‘ImportError: Failed to import test module: anvil.users.mfa\nTraceback (most recent call last):\n File “/usr/local/lib/python3.7/unittest/loader.py”, line 470, in _find_test_path\n package = self._get_module_from_name(name)\n File “/usr/local/lib/python3.7/unittest/loader.py”, line 377, in _get_module_from_name\n import(name)\n File “/downlink/anvil/users/mfa/init.py”, line 5, in \n from anvil.js import window\nImportError: cannot import name 'window' from 'anvil.js' (/downlink/anvil/js.py)\n\n’), (<unittest.loader._FailedTest testMethod=sandboxlib>, ‘ImportError: Failed to import test module: sandboxlib\nTraceback (most recent call last):\n File “/usr/local/lib/python3.7/unittest/loader.py”, line 470, in _find_test_path\n package = self._get_module_from_name(name)\n File “/usr/local/lib/python3.7/unittest/loader.py”, line 377, in _get_module_from_name\n import(name)\n File “/downlink/sandboxlib/init.py”, line 1, in \n from .virtualizedproc import VirtualizedProc, signature, sigerror\n File “/downlink/sandboxlib/virtualizedproc.py”, line 5, in \n from ._commonstruct_cffi import ffi\nImportError: dynamic module does not define module export function (PyInit__commonstruct_cffi)\n\n’)] []

My current guess at what’s going on
Going through the above logs, it looks like there are some fundamental anvil components that unittest is trying to run (App Tables, Downlink, Sandbox) but is getting stuck on.

Any help or suggestions are appreciated, as always.

Clone link:
share a copy of your app

#2

I can’t really see this bit working, or understand how it would find any tests? Are there .py files you have loaded on the anvil server in some location? Are they already there?

Maybe it doesn’t work the way it makes me think it works, I don’t set up tests very often in my day to day.

Is there a way to see if suite contains any results, before using it in runner.run() ?

#3

There aren’t any .py files other than the standard ones from interacting with the IDE by adding modules and forms.

I’ll see about analyzing the results of suite.

#4

I’ve never got test discovery to work because it’s file based. Here’s the best I’ve come up with:

2 Likes
String List of Server Modules with Pattern Matching Function
#5

Ok, back from my experiment:

Modified function…

@anvil.server.callable
def server_auto_tests(verbosity=1):
  suite = unittest.TestLoader().discover('.', pattern = "_Test.py")
  numberOfTests = suite.countTestCases()
  print('numberOfTests were ' + str(numberOfTests))

Returns 3 tests regardless of the number of test functions in SampleTest

I think it has to do with

So I’ll give that a shot :slight_smile:

#6

This definitely worked, so thank you! I’ll be using it until I can figure out a way to manually scan the Anvil project to do something analogous to

suite = unittest.TestLoader().discover('.', pattern = "_Test.py")

Except instead of files, it will be for modules.

I’ll be hiking in the anvil.server to see if there’s anything useful there that I can use to reflect on the current anvil project’s modules in the server code.

I didn’t see anything in the documentation that allowed this, so I created a feature request here

1 Like
#7

you might be able to use dis together with globals() to gather your module names, I’m not sure where it would take you but its definately a place to start.

I used it here: