Storing images in cache

Anvil Q&A
1

Hi everyone

Firstly, as others have said, very excited about the possibility that offline cache usage provides. However, it seems to have a problem with images:

This used to work fine, when I downloaded each row of a data table, one column of which holds images, as a list of dictionaries, first converting each row to a dictionary with dict(). This list was stored to a local global variable - any image component could load the ‘pic’ key-value from the local variable (list of dictionaries) no problem.

Now trying the cache, at first use, I load the data table to local in exactly the same way, list of dicts, each row converted to a dict with dict(), then save into the cache.

On load, the app looks for the cache, and if it’s available it loads the list of dictionaries to the same global variable, but now trying to access an image throws:

TypeError: The ‘source’ property can only be set to a Media object or a URL

…at this line:

i = Image(source=self.item[‘pic’],height=50,width=50)

when printing the self.item[‘pic’] I get this:

{‘v’: {’$d’: {‘size’: 0, ‘entries’: {}, ‘buckets’: {}, ‘in$repr’: False, ‘$version’: 0}, ‘$anvil_isLazyMedia’: True, ‘_spec’: {‘manager’: ‘table-media’, ‘key’: ‘bf7bb5ef4fe37edbd2817a0b2d476f6c’, ‘id’: ‘13866864’, ‘mime-type’: ‘image/jpeg’, ‘length’: None, ‘name’: ‘Baby Einstein Light and Rock Guitar.jpg’, ‘type’: [‘LazyMedia’], ‘path’: [‘response’, 0, ‘pic’]}}, ‘$isPyWrapped’: True}

Is there a special way I must handle media between a data table and the cache?

Thanks in advance!

Bruce

1 Like
2

You can try save image link, not the actual image or if you really one to use image, have a look at this tutorial

Browsers often cache images, audios… after first load.

1 Like
3

Which offline storage method are you using?

Indexed_db can handle images - I think they are automatically converted to bytes.

When you reload them you have to create a Media object from the bytes and then set the source of the image to Media object.

Does this make sense?

1 Like
4

I was referring to the anvil_extras 3rd party dependency recently talked about. So I was storing my data in local_storage[‘data’] for example.

How does one access indexed_db? I’ve heard about it but was not aware anvil could access it easily/reliably?

Thanks!

5

https://anvil-extras.readthedocs.io/en/latest/guides/modules/storage.html

from anvil_extras.storage import indexed_db

I think you’ll need to convert your media objects to bytes before storing them in indexed_db.
The indexed_db has the same api as local_storage

from the docs:

Which to chose?
If you have small amounts of data which can be converted to JSON - use local_storage.
If you have more data which can be converted to JSON (also bytes) - use indexed_db.

JSON typically means: int, float, str, list, dict, None, bool types.
So with indexed_db you get JSON plus bytes.

For anything else you’ll need to convert it to something JSONable first.
e.g. datetimes can’t be stored.
Unfortunately the wrapper around the browser Storage doesn’t fail loudly if you try storing something that you shouldn’t (which is what’s happened here)

(Feature requests are always welcome - just add an issue to anvil-extras)

6

Wow so helpful thanks Stu

I did try store as bytes first (with pic.get_bytes() ) but then when trying to put it into local_storage I got a size limit/quota exceeded error.

I’ll wade on into indexed_db!

1 Like
7

Hello again

Even less luck with indexed_db. When attempting to save the list of dictionaries (even without images) into indexed_db, I get the following error:

ExternalError: DataCloneError: The object can not be cloned.

Any ideas?

8

I wrote some code a month or so ago that successfully stored about 50 images into indexed_db with no problems… I do remember running in to this issue. Let me go and check…

Edit: Did you have any datetime objects in there? From memory, indexed_db doesnt like them so you have to convert to ISO format and back again when you load them

1 Like
9

The following seemed to work for me

To store a datetime object in indexed_db, convert to ISO Format:

from datetime import datetime

iso_datetime_string = datetime_obj.astimezone().isoformat()

When you want to go back from iso to Datetime, use:

from datetime import datetime

datetime_obj = datetime.fromisoformat(iso_datetime_string)

And yes as per @stucork suggestion you need to store bytes of images.
I wrote a function that did these conversions automatically before and after storing in indexed_db

Hope it helps

1 Like
10

You’re on the money!

datetime objects and images are the issue. date time sorted thanks

Only problem with images, to convert them in server code is quick and easy. But it seems you cannot pass a dictionary with a bytes type value from server to client. And to convert all the images to bytes client side takes 25 seconds (1sec max on server).
And I’m struggling to convert the bytes value to a string before sending to the client.

Thanks again for all your help…any more ideas on efficient image manipulation?

11

Question: Is the 25 seconds because of the fact that images are trasnferred first and then converted to bytes? I ask this since I think if you are loading these from data table rows, media objects are not cached so require to be fetched from the server.

I don’t remember seeing this kind of delay when using get_bytes on an image solely on the client side.

Have al ook at this solution to a similar problem I was having. It uses base64 endcoding to which you can then return from the server.

1 Like
12

Almost certainly because they’re being fetched yes. Thanks I’ll explore base64…