SuspensionError calling datetime.strptime in an iterator

Anvil Q&A
1

I am getting this error while iterating my class:

ExternalError: TypeError: Sk.builtin.SuspensionError is not a constructor
at TruckDetails.ItemVersion, line 16

Here:

for truck in Trucks():  # this is line 16
    [...]

My class is defined like this:

class Trucks:
    [...]
    def __iter__(self):
        self._load_trucks()  # the failure is in here
        self._iter_index = -1
        return self

    def __next__(self):
        self._iter_index += 1
        if self._iter_index >= len(self._trucks):
            raise StopIteration
        return self._trucks[self._iter_index]

At first I thought the problem was on my iterator, but that seems to be working well. I can iterate my class in Python, and I kind of can iterate it also in Skulpt.

I traced it down to be at this line, called by self._load_trucks:

print('this is printed')
shipping_date = datetime.strptime(shp_date, '%m-%d-%Y').date()
print('this is not printed')

I replaced the datetime.strptime(shp_date, '%m-%d-%Y').date() with 'x', and the problem disappears from this line, goes through two cycles, then it appears again. Perhaps the self._load_trucks() is hitting another line with the same problem.

So here are my two questions:

  • What is causing this problem and how do I solve it?
  • Why is the problem reported on the for loop line which uses my iterator which calls a function which calls datetime.strptime instead of being reported at the datetime.strptime line?
2

The quick fix is to do import _strptime at the top of the module.

We’ll look into the problem our end and report back.

2 Likes
3

Thanks, importing _strptime does fix the problem, but it just kicks the can a little further and I get the same error after a few lines, when the app makes a server call.

I created a little test app that shows both the errors, and shows that after importing _strptime one of the two errors is solved:
https://anvil.works/build#clone:6TTRGWDCJZ2TDDTV=6POAUPI66Q5P2GEPVISCMZHX

In the real app I have a nested data structure where the children of each level are loaded lazily using server calls.
I don’t want to load everything at once, in one server call, because that call would return a few mega bytes of stuff.
Is it possible to fix the problem with server calls too?

PS:
Why does import _strptime fix the problem?
What does import _strptime have to do with datetime.strptime(...)?

PPS:
Figuring out where is the problem on the real app with iterators used by databinding triggered by iterators using databinding, in a place where the traceback points you to the wrong place, is a majestic nightmare!

1 Like
4

Yeah, I’m starting to just ditch the databinding whenever I run into issues and code that part manually instead.

1 Like
5

The correct error message should have been:

SuspensionError: you can't call a function that blocks or suspends here

The wrong error displaying is an internal bug that we’ll fix.

Now, assuming that the correct error was displayed.

Suspensions happen whenever we call the server or encounter blocking code.
Code that blocks includes:

  • sleep()
  • Any server call
  • Some imports that are lazily loaded (some client side python std-lib modules)

Most of the time you don’t notice the blocking code since it all happens without you thinking about it.
But occasionally you’ll find places in client code that don’t support blocking code (aka Suspensions).
When client code finds blocking code it doesn’t know how to handle it throws the SuspensionError.

For years this is why the fix for:

dict(my_row) # i require a server call to get my values

was to do

dict(list(my_row))

dict couldn’t handle blocking code but list could.
(We’ve now addressed this so that dict knows how to handle blocking code)

Creating an iterator is a place that can’t block.
So having a __iter__ with blocking code causes a problem.
The error doesn’t get thrown inside __iter__ because it’s only creating the iterator that has the problem.

Simple example

from time import sleep

class A:
   def __iter__(self):
        sleep(.1) # I block
        return iter([1,2,3])

a = A()
for _ in a: # SuspensionError: you can't call a function that blocks or suspends here
    pass

The reason for the fix with _strptime is that it is a module that is lazily loaded.
It loads the first time you call datetime.strptime so causes blocking code.
By importing it early we don’t block.
(the _strptime module is required for datetime.strptime)
The reason the can got kicked down the road is because you also have a server call which blocks.

How to fix
include a flag and do the blocking code in the first call to __next__.
(getting the next value from a created iterator is a place that knows how to handle blocking code).

Noted on the challenge of debugging this - we’ll try to improve the traceback in the SuspensionError.
And think about some other ways to improve this behaviour.

(There isn’t a quick fix for supporting creating an iterator on the client with blocking code).

4 Likes
6

Thanks for the detailed answer.

Moving the data loading out of __iter__ and into __next__ fixed the problem.

It would be even better if that “here” was replaced by “inside __iter__()”.
If the list of places where this call wouldn’t work is long, then a reference to a documentation page that lists those places.

Maybe this is not a bug after all, and this post should be back in the Q&A section with your answer marked as solution?

3 Likes
7

Quick update on this - you should now see the correct error message with a traceback to which function call caused the Suspension.

In the trivial example:

from time import sleep

class A:
   def __iter__(self):
        sleep(.1) # I block - line 5
        return iter([1,2,3])

a = A()
for _ in a: # line 9
    pass

You’ll now get this error message:

SuspensionError: Cannot call a function that blocks or suspends here
at Module1, line 5
called from Module1, line 9

(We’ll also look at improving the message itself - but hopefully having tracebacks would have made it somewhat easier to understand the location of the root cause)

4 Likes