I’m piecing this together from existing code. It was written to solve a problem, not as a general “how-to”. So I’ve had to edit extensively. Apologies if I have, in the process, introduced syntax or execution errors.
I’m deliberately omitting module setup (imports and anvil.server.connect() calls). They don’t illustrate any important points about this technique.
First, you need a db table to record each outstanding task. In this example, I name it Tasks (python_name: tasks), with the following columns:
- request: string
- when_started: datetime
- when_completed: datetime
You may want to add other bookkeeping columns.
Second, a Server Module function to create and launch the task:
@anvil.server.callable
def launch_offsite_task(request):
"""Immediately launches a task,
with the string request as data.
Returns a row id for use as a task id."""
task = app_tables.tasks.add_row(
request = request,
when_started = datetime.utcnow(),
when_completed = None
)
task_id = task.get_id()
anvil.server.call('launch_task_nowait', request, task_id)
# note: all sorts of things can go wrong with the above call,
# so you should really wrap it in a try...except block.
return task_id
The caller might use the task_id, later, to look up the table row, and see whether the task has completed.
Third, an Uplink function, off-site, to launch the task locally.
job_inbox = '/tasks/in_box/'
@anvil.server.callable
def launch_task_nowait(request, task_id):
"""Immediately launches a local task,
with the string request as data.
Returns immediately."""
task_id_as_file_name = task_id.encode().hex()
full_file_name = job_inbox + task_id_as_file_name + '.txt'
with task_file as open(full_file_name, 'wt'):
task_file.write(request)
Because several users might launch tasks at essentially the same time, this function may still be running when Anvil calls it again. In this case, it will run that call in its own thread.
So be careful with global variables! Avoid writing to global data, even indirectly (e.g., via some database wrapper), unless you can be absolutely certain that the write is thread-safe.
Fourth: Now that the request has been written as a file, it’s up to some worker program to pick up the file and actually get something done with it. For simplicity, we assume that the worker writes to an outbox folder when it is done. Then a separate Python program, monitoring that folder, can notify Anvil that the work really is done:
job_outbox = '/tasks/out_box/'
folder_to_watch = Path(job_outbox)
pattern_to_wait_for = '*.txt'
cycle_time_seconds = 30
def record_task_completion(task_id):
"""Records completion of a local task."""
task_row = app_tables.tasks.get_by_id(task_id)
task_row.update(when_completed = datetime.utcnow())
# begin watching for completed jobs:
while True :
for full_filename_found in folder_to_watch.glob(pattern_to_wait_for):
file_name = full_filename_found.stem
# convert file name back to a task id:
task_id = bytes.fromhex(file_name).decode()
record_task_completion(task_id)
# remove flag file, so we don't process it again:
full_filename_found.unlink()
# Once we've exhausted the out-box,
# let other programs use the file system and CPUs:
time.sleep(cycle_time_seconds)
Because this function handles just one file at a time, it does not need to use threads. It can safely use global state (e.g., a database connection).
I hope this gets the general idea across.
