Anvil JS - memory leak when returning data from JS, Python dicts aren't updated

Bug Reports
1

What I’m trying to do:
Hi Anvil and friends :slight_smile:

I’ve got an application with a WebSocket in JS that is handling some stream of data. My application then pulls that data on user demand and displays it.

I’ve noticed that every time the data is pulled, a reference to the copied object is kept internally by Skulpt, leading to a memory leak over time. This leak persists no matter if the passed object is a string (then it’s leaked in Skulpt’s str() builtin), a dict (dict() builtin), or list.

To try and circumvent this I thought of passing a dict from Python to a JS function, having it fill the data, and then using it back in Python so to avoid returning a new object from JS. This made me notice that the dictionary on the Python end doesn’t reflect changes made by JS.

I created a MRE in the clone link below. The code has a button that when pressed calls a JS script function with a Python dict. The JS function returns a random string and adds a value to the passed dictionary. The Python code then finally throws the result into a textbox. The returned string needs to be random to force a new string allocation and not let any constant-string optimizations happen under the hood. This chain of events serves to mock the on demand pulling of information from my WebSocket.

By running the cloned app and repeatedly pressing the button you can observe the following findings:

  1. the new key added to the dict by the JS function is absent after its return.
  2. monitoring the app with memory profiling (I tested with both firefox and Chrome’s developer tools) shows every press of the button causes the returned value to be leaked as it remains referenced internally by Skulpt. This memory isn’t freed, and the more you press the more the “baseline” memory consumption grows.

I tried deling the dict in the Python code, explicitly marking references as null or undefined in the JS code, and various other techniques but the root cause is that Skulpt keeps holding on to references after I’m done with them and marked them as such.

tl;dr:

  1. is it possible to populate a dict passed from Python to JS?
  2. returning a value from JS to Python leaks memory. How to handle?

Cheers!

Clone link:

3

Thanks for reporting, i’ve moved to bug reports.
This is an issue you’ve found with how Skulpt interns strings.
I’ll update this thread with any updates.

I think the post is really 2 questions, and I’d recommend creating a separate thread about Python dicts not getting updated.
This’ll make it easier for others to reply to that specific question.

4

Thanks Stu! I’ll do that right now. Should the dicts question go back in the QnA section or here under Bug Reports?

5

Yeah i’d create a new question based on the above, ignoring anything to do with the memory leak, it confuses the question you’re trying to ask.

2 Likes
6

Hey Stu - this might be a dumb question, but how do I edit the original post here? It says I have no permissions to edit or delete it anymore.

In the meantime I opened a separate topic for the dict update as you suggested. Thanks!

7

@jacob.levine91 no need to edit this post.

I’m aware of an issue with interned strings but your post also mentions problems with dict and list. I’m not aware of issues with either of these objects. Are you able to demonstrate the issue with these data structures? (Do ensure that you profile the memory after running garbage collection).

8

You can easily reproduce in the app I linked by returning a dict instead of a string, and putting the random string as a value in the dict. I can make another app and put a clone link if that helps.

9

yes - if you return { random_str } i don’t see an issue with dict.

Here’s a quick rundown:

  • Click the button a few times
  • in the memory tab in dev tools hit the garbage collection
  • record a snapshot
  • click the button again
  • click the garbage collection button
  • record a new snapshot
  • hit the “comparison” option to compare the snapshots

The only additional memory from skulpt is that of the interned str

If i’m missing something let me know.

10

I followed your steps and observed the random_str is still referenced after garbage collection. You’re right though that it’s not clear if the dict itself is still being referenced, just that the retainer for the internal string now resides within dict().

1 Like