Setting up an environment for testing modules locally

I always clone my apps and test the modules locally. Unfortunately sometimes the process is not smooth and I can have the following problems:

  • Sometimes server functions run on the server, so they can’t be debugged
  • Sometimes server functions run locally, so they don’t work (for example when they use SQL - I have a dedicated plan with SQL access that only works on the server)
  • Sometimes the production app, which runs on the server on a different environment and on a different branch, stops working while I run a test because its server functions run on my pc, where I am running the tests

A local test environment is easy to get to work, but I got it wrong so many times that I decided to come here and write down the steps required to:

  • Have one module with server functions that always run on the server, and only their return value is tested
  • Have one module with server functions that run locally and can also be debugged
  • Not bother the production app while running the tests

Here are the steps to create an app with two server modules, one with the functions that should be tested locally and one with the functions that should run on the server even while testing:

  • Create the app
  • Create a branch called dev
  • Create 2 environments, Published with master and Development with dev
  • In the Development branch click on “Show advanced settings” and enable server uplink. Make sure the “Send server calls from other environments to this Uplink” is unchecked (otherwise the production app would use your test Uplink while you test the Development branch)
  • Make sure dev is checked out
  • Add two server modules and call them RunOnServer and RunEverywhere
  • Add this function to RunOnServer :
@anvil.server.callable
def two():
    txt = f'Two * {__name__} * {anvil.app.branch} * {anvil.server.context.type}'
    return txt
  • Add this function to RunEverywhere:
if anvil.server.context.type == 'uplink':
    # running tests on pc
    def two():
        return anvil.server.call('two')
else:
    # running on Anvil server
    from RunOnServer import two

@anvil.server.callable
def one():
    txt = f'One * {__name__} * {anvil.app.branch} * {anvil.server.context.type}'
    return f'{txt}\n{two()}'
  • Add this to form_show:
def form_show(self, **event_args):
    one = anvil.server.call('one')
    two = anvil.server.call('two')
    alert(f'one:\n{one}\n\ntwo:\n{two}', large=True)
  • Click on “Merge changes into master” (this will also check out master)

At this point running the app from dev will show an alert saying it’s running from dev, running it from master will say from master.

  • Clone the app to a local repository
  • Make sure dev is checked out
  • Add the Tests folder
  • Add the test.py file with this code, replacing <uplink key> with the correct value:
import anvil.server
from unittest import TestCase
from server_code.RunEverywhere import one, two
class TestBackgroundTask(TestCase):
    def test_1(self):
        anvil.server.connect('<uplink key>')
        print('=== one ===')
        print(one())
        print('=== two ===')
        print(two())

At this point running the test the function one() will run locally and the function two() will run on the server.


I use type hinting on client modules. In the IDE it’s ignored. It doesn’t help, but doesn’t bother either. But it helps when developing locally, so I use it.

The problem is that type hinting doesn’t bother Skulpt, but the imports do.
The solution is to not import when running on the client.
This will import the typing classes when working locally with PyCharm but will not bother the browser:

if anvil.server.context.type != 'browser':
    # running on server
    from typing import List, Optional, Union, Tuple, Dict

Always make sure the “Send server calls from other environments to this Uplink” is unchecked. There may be cases where you want to run server functions on your uplink, but when that checkbox is checked, ALL the server functions defined in any running uplink ALWAYS run in the uplink.

This means that your production apps will pick uplink functions being tested and modified while the tests are connected and pick their own functions when all the uplinks have disconnected.

I had a few days hitting the head in the wall trying to figure out why an app was working for some users and not for other users, and for me was working only intermittently.


It is possible to debug HTTP endpoints by:

  • checking “Send server calls from other environments to this Uplink”
  • importing the module that defines them
  • using the correct URL, which may not be the URL shown at the bottom of the server module that defines them in the IDE. The correct URL for the HTTP endpoints can be found by adding _/api/ to the URL obtained by running the app from the IDE in its own tab

Careful: by checking the “Send server calls from other environments to this Uplink”, you are executing all the HTTP endpoints on your testing environment, even the production ones.

5 Likes

Imagine that I have the following folder hierarchy:

src\
  __init__.py
  client_code\
  server_code\

Where, in relation to client_code and server_code, would be a good, practical place to put folder Tests? (E.g., to make the import statements in Tests\*.py work cleanly.)

  1. At the same level (in the same folder) as `src’?
  2. At the same level as `client_code’?
  3. At a level higher than src?

This is how what I see in PyCharm with the app I’m working on now, following the suggestions from Ian:

image

And this is how I import some modules:

if anvil.server.context.type == 'uplink':
    # running tests on pc
    from client_code.Release import Project
    def execute_query_and_fetchall(cmd, parameters=None):
        return anvil.server.call('execute_query_and_fetchall', cmd, parameters)
else:
    # running on Anvil server
    from Release import Project
    from Sql import execute_query_and_fetchall

The imports are different because I only have access to SQL from the server, not from uplink and because… it works.

1 Like

This helps immensely! Thank you, @stefano.menci !

That is the first place I checked, but it didn’t say where to put Tests…

Do you also create a __init__.py for client_code and server_code?

Code in my tests folder has trouble finding modules from those locations.

1 Like

No, I don’t create a __init__.py for client_code and server_code.

Until yesterday I used two strategies:

  • A mess of if anvil.server.context.type == 'uplink' to decide how to import what
  • When that didn’t work, a mess of try - except
  • A mess of folders tagged as source code in PyCharm (I think it changes some environment variables before starting Python, not 100% sure)

The problem is that, once I get my local environment working, PyCharm does its best to figure out what’s going on, but sometimes it just fails to find the correct references. The problems are even bigger when I have dependencies and start tagging folders on other repositories as source folders. And the problems increase when some apps and some modules have the same name.


Right now I’m in the process of setting up the test environment for a little app that I have almost finished without doing any tests (yeah I know, not really test driven, but this is 50% UI, 50% dependencies and the remaining 0.0something% the new app itself). The dependencies don’t play nicely with each other, so I need to setup a test environment on my PC. Unfortunately (or rather luckily) I just got a new PC. I just installed Python, hopefully I will be ready to start working on the tests tomorrow.

Before the old PC gave up, I had just learned that the reason why the tests folder has trouble finding the modules is what Anvil does with the __init.py__:

__path__ = [__path__[0]+"/server_code", __path__[0]+"/client_code"]

The folder structure is flattened, so it’s easier to import from both server_code and client_code, but PyCharm doesn’t like it.

I tried adding:

__path__ = [__path__[0]+"/server_code", __path__[0]+"/client_code", __path__[0]+"/tests"]

and I got the tests to run, but I don’t like to mess with automatically generated code. And I don’t like it because PyCharm doesn’t understand it.

So, as soon as I have the new PC ready, I will try to figure out what is the best way to get the tests folder to import everything from both the current app and the dependencies, in the cleanest way possible, so PyCharm and type hinting will work without problems.

I will let you know if I figure out a pattern that makes sense to tell you about.

1 Like

Thanks, Stefano. Likewise, I will post here if I find a way.

The pytest books I’ve seen so far all assume that you’re testing an installable package, and have, in fact, actually installed it, in editable mode.

Neither of which apply to Anvil apps, of course. Perhaps there’s a pytest plugin that will help. If not, it might be easy enough to write one.

Edit: Just getting started with pytest. Maybe a pytest fixture? Or config setting?

3 Likes

Thanks @p.colbert and @stefano.menci , if you do find a way to flatten the directories please post here! I gave up and now I use in my server code:

try:
    from global_cls import PermitGrade
    from classes import ContactRowType, OwnerRowType, RoleRowType
except ImportError:
    from client_code.global_cls import PermitGrade, StatusCode
    from server_code.classes import ContactRowType, OwnerRowType, RoleRowType

and in my client_code:

if anvil.server.context.type == "laptop":
    from server_code.classes import ContactRowType, OwnerRowType, RoleRowType

It’s a little annoying.
BTW for my simple apps I use pytest to test client_code and server_code with pyDALAnvilWorks
Screen Recording 2022-06-11 at 3.04.38 AM

1 Like

For anyone else using pycharm:
You can flatten the directories by setting the interpreter paths in PyCharm (see screenshot)

1 Like