Server Environment Variables

I was frustrated with how I was using configuration variables especially when it came to variables that were embedded into a dependency app.

My problem

I don’t like having to search around for where a particular configuration value lives within the code across dependencies.

I don’t like having to remember the dependency app structure to override particular values in a parent app.

I don’t like how much inconsistency has come from my previous methods.

My solution

I create ENV dependency app that handles environment variables. It allows dependency apps to register environment variables but provide a default value even if none of the setup is complete. Giving dependencies a “batteries included” feel to them. If all of the default values work, great! There is no additional work on the parent app. If you want to change a config value in the parent app there is some additional setup and ENV provides tools to review the available configuration values.

Features

Common Interface

Variables can be retrieved by environ.get(VARIABE_NAME)
Varaibles throughout the app, dependencies included, can be set from a top level table called env.

Default values

Providing default values can make dependencies work right out of the box without any additional work from the parent app.

from ENV import environ
my_variable = environ.get('MY_VARIABLE', 1234)

Forced values

You can force values to be included in the environment variables.

from ENV import environ
api_public_key = environ.get('API_PUBLIC_KEY')

Which will raise a LookupError if it is not available in the environment variables. This is nice when you have some configuration values that will be specific to the parent app and should never be reused between apps.

Variable Tracking

ENV automatically collects environment variables throughout the app and provides access to them from the server console.

This allows me to place my variable definition closer to the point of use which is better for readability, debugging and adjusting default values.

This can be retrieved from the server console:

>> from ENV import environ
>> environ.VARIABLES

Environment Variables
in_use:
	API_PUBLIC_KEY
available:
	MY_VARIABLE
	API_ENDPOINT
	API_PUBLIC_KEY_ROTATION_SCHEDULE

available are variables that are currently using the default value and can be overridden.
in_use are variables that are currently being pulled from the environment variables table.

Setting variable

It is possible to set variables programatically as well. This can only be done if the env table is setup.

from ENV import environ
environ.set(variable_name, variable_value, variable_info)

Clone

ENV Clone

Or use as third party dependency with token:
AFKPYPDLJMH2TYVK

3 Likes

This is a neat idea and good of you to share. What are the downsides in your opinion?

Let me just stream of consiousness some downsides:

  • It is a dependency that has versions, breaking changes, etc.
  • If you want to override or define forced variables you must create the env table at the top level app.

As an aside, it would be great if I could programatically create this table to simplify setup and reduce creation errors. However, it is a very simple table.

  • If you are using this as a third-party dependency, you have to trust me.

  • If you don’t trust me, you need to clone the app and review it to make sure I’m not up to something nefarious.

  • You lose namespacing of variables, so variable names can become obnoxious or potentially overlapping.
    good variable names:
    STRIPE_WEBHOOK_ENDPOINT - From dependency A
    MY_SENSOR_WEBHOOK_ENDPOINT - From dependency B

    bad variable names:
    WEBHOOK_ENDPOINT - From dependency A
    WEBHOOK_ENDPOINT - From dependency B

These overlapping names between modules would be resolved to the same environment variable if they were to be overridden. So, things could work as expected while using default variables, then blowup after overriding.

I’ll ponder this some more, but these are some of the downsides/pitfalls that I could pull from the top of my head.

1 Like

That’s great! Thanks!

Here is another opinion question: what is the difference between using this dependency and using a table shared across all the apps?

I have one environ table shared across all the apps and one Admin tools app that allows admin users to edit all the values in that table.

The table has a text key column and a simple object value column. Using a simple object column allows to store more info in one single key and limits the proliferation of keys.

All I need is to share the table and use app_tables.environ.get(key='my setting').

What are the advantages of this dependency compared to the simple key/value shared table I described?

2 Likes

I think it comes down to a lazier version of what you are doing. I can, but don’t have to populate or even create the env table.

Development is easier

Because this system allows for default values, you are free to develop without defining variable in your table. If the the env table or variable is not defined, it utilizes the default value you provided in your code. In the future if you think updating a value is necessary, you can just query the environment variable manager to see what can be updated and create the override value in the env table.

Allows for dependency development

When I’m developing a dependency app, I add the env table with values that are convenient for development. For instance, I was just working on a customer specific Stripe pricing table. This uses a short lived client secret value that expires after 30 minutes. I could have the default expiration be the expected 30 minutes and then override it with 45 seconds in the dependency app to test my key rotation system. Adding this dependency to my app didn’t require me to do anything with this value since the override values are determined at the top level not the dependency level. So, since the top level env table didn’t have a expiration override, the default 30 minute value was automatically used.

I’ve been tempted to integrate this into anvil.secrets to allow this type of behavior for development vs production API keys.

Non-exhaustive table

In most cases for me the value I conjure on the spot don’t need to be changed. In this case the default value can just be used and I never have to populate a table with a value. This also means that if the value is in the available list that I am free to rename it within the code without needing to also update a name in a table. If it is in the in_use I know that I need to update the table as well.

Hmmm, a global table

I like the idea of a global table like you are using. It could be interesting to have both global and local env tables. You could look for variables in global envlocal envdefault and select the first found.

You could use this dependency with your global table. I could see your admin panel having your normal view showing the in_use variables and a list of options of variables that you could add from the available list.

1 Like

Using a shared table presents both a challenge and a benefit:

  • Challenge: There is a risk of conflicts when two apps access the same variable, mistakenly assuming it belongs to them. The solution is to include the app name in the variable name for variables that are app-specific.
  • Benefit: Some settings can be made global across all apps. For example, a variable like EmergencyNotificationEmailAddresses could store a list of email addresses accessible to multiple apps.

Another potential benefit of introducing a dependency to manage the list is that the table could include last_modified_time and last_accessed_time columns. These timestamps would help identify old, unused variables that were only relevant to apps that have been deleted. Currently, I have some variables in the table whose origins or usage are unclear, perhaps created by a colleague for a now-defunct app, and I’m scared to delete them.

If timestamps were available, I could safely remove variables that have gone unused for two years.

1 Like

Interestingly, app secrets sort of behave this way.

If you have a secret defined in a dependency app but not the main app, your main app will use the secret defined in the dependency app.

I have been using secrets so far for this purpose and it does get a bit confusing sometimes but I got used to it

1 Like