Storing multiple temporary files from app tables

Anvil Q&A
1

I have an app table with 5 images per row for N number of rows. I need to write 5XN temporary files to the file system so xlsxwriter can write all the image data at once since xlsxwriter doesn’t support file appending. I have to write everything before I close the workbook, and because my app tables images are stored in rows it becomes problematic because I can’t use a for loop to do this.

Normally I use something liket his

with anvil.media.TempFile(result[‘gray_image’]) as img_file:

To load 1 file, but how can I load a batch of files within 1 ‘with’ statement?

I tried workbook = xlsxwriter.Workbook(‘results.xlsx’, {‘in_memory’: True})

using the in_memory true to see if that would store the image data so I wouldn’t have to keep all the files open but it doesn’t seem to be working…

2

You can use multiple contexts in one single with:

with A() as x, B() as y, C() as z:
    do_something(x, y, z)
1 Like
3

Thanks, but how can I use that for multple rows of data (N). So I need A0-AN() … B0-BN() … etc

Normally I would put the with statement within a for loop, but that doesn’t work because the files would be deleted.

With xlfswriter, it stores the url of the file until the end when you close the workbook, so I need to keep the files open until I have all of the rows written.

4

You talk about xlsxwriter and images, but don’t tell what is the relation (xlswriter is usually not used for images).

You talk about a loop, but you don’t show an example.

Can you show a more complete example of what works and what doesn’t?

5

I’m not sure, but I think I have a feeling what @conraddonovan16 is talking about with the temp files?

xlswriter automatically closes a file when you close the workbook, when you close a normal tempfile in python the default behaviour is to delete it, making it impossible to get the finished excel file.

Is this the problem?

6

It works if I have workbook.close() within the with statement like this

workbook = xlsxwriter.Workbook('results.xlsx') 
worksheet = workbook.add_worksheet()
for i in range(0,n): #number of rows
    #select only current ower rows in database
    result = app_tables.biofilm_images.search(
    owner=anvil.server.get_session_id(),
    )[i]
    row = i+1
    with anvil.media.TempFile(result['original_image']) as img_file:
        worksheet.insert_image(row,2,img_file, {'x_offset': 5, 'y_offset': 5, 'positioning': 1})
        workbook.close()

Yes, so it deletes the file if I have the workbook,close() outside of the with statement in the below exampe. But I need to call multiple rows, so I need to use a for loop

workbook = xlsxwriter.Workbook('results.xlsx') 
worksheet = workbook.add_worksheet()
for i in range(0,n): #number of rows
    #select only current ower rows in database
    result = app_tables.biofilm_images.search(
    owner=anvil.server.get_session_id(),
    )[i]
    row = i+1
    with anvil.media.TempFile(result['original_image']) as img_file:
        worksheet.insert_image(row,2,img_file, {'x_offset': 5, 'y_offset': 5, 'positioning': 1})
workbook.close()
7

Why not:
(to make a code block on the forum enclose the text with triple backticks ``` on different lines )

with anvil.media.TempFile(result[‘original_image’]) as img_file:
    for i in range(0,n): #number of row
        worksheet.insert_image(
                                row,
                                2,
                                img_file,
                                {‘x_offset’: 5, ‘y_offset’: 5, ‘positioning’: 1}
                                    )
    workbook.close()
    #  Do something with the temp file here 
    #  before the end of the 'with' context block.

#  File no longer exists here outside of context

Would this work? (if not, then we are misunderstanding something)

Edit: I just looked at some code where I am using xlsxwriter and it looks more like this to create the workbook object:

workbook = xlsxwriter.Workbook(filename_string)

So again, I think I’m missing something?

8

Thanks, still learning the ropes with this forum. I have the code updated now. The issue is I need to call the with statement within the for loop because I have multiple rows of original images. There’s actually 5 images per row, but I have it simplied for this example.

9

I was able to assemble a list of media objects like this:

[<anvil.media.TempFile object at 0x7f2ef14b7b90>, <anvil.media.TempFile object at 0x7f2ef14b7d10>, <anvil.media.TempFile object at 0x7f2eb6b64a10>, <anvil.media.TempFile object at 0x7f2eb6b64750>, <anvil.media.TempFile object at 0x7f2eb6b649d0>]

Is there a way to call

with anvil.media.TempFile() as img_files:

for the entire list at once? The length of the file list will be different each time, depending on how many images the user uploads. Perhaps there is a way with a context manager?

10

I’m trying to get it to work using some examples I found here.

from contextlib import contextmanager
@contextmanager
def multi_file_manager(files):
    """ Open multiple files and make sure they all get closed. """
    files = [anvil.media.TempFile(file) for file in files]
    yield files
    for file in files:
        file.close()

then calling this from main program

with multi_file_manager(image_objs) as files:
    worksheet.insert_image(row,3,files[0], {'x_offset': 5, 'y_offset': 5, 'positioning': 1})

But I’m getting this error when I try to write to the worksheet “TypeError: stat: path should be string, bytes, os.PathLike or integer, not TempFile”

Any suggestions? Is this even possible to do like this?

edit: I should note, image_objs is a list of media objects shown in the previous post.

edit2: I think I may have found a better implementation

I think I may have found a better implementation seen here

from contextlib import ExitStack

 with ExitStack() as stack:
    files = [
        stack.enter_context(anvil.media.TempFile(obj))
        for obj in image_objs
    ]

But I’m getting the error “AttributeError: ‘TempFile’ object has no attribute ‘get_bytes’”

Any ideas?

11

If this:

Will take a bytes string of the image instead of a file, (as suggested by the error it gave you) then you should be able to cut out all of the turning it into a file part and just do:

image_objs[0].get_bytes()

worksheet.insert_image(row,3,image_objs[0].get_bytes(), {'x_offset': 5, 'y_offset': 5, 'positioning': 1})

You can get the bytes string directly from the anvil media object without messing around with temporary files of any kind.

12

for reference,

image_objs = []
  for i in range(0,n): #number of row
    #select only current ower rows in database
    result = app_tables.biofilm_images.search(
    owner=anvil.server.get_session_id(),
    )[i]
    row = i+1
    image_objs.append(anvil.media.TempFile(result['original_image']))
    image_objs.append(anvil.media.TempFile(result['gray_image']))
    image_objs.append(anvil.media.TempFile(result['gray_image_filtered']))
    image_objs.append(anvil.media.TempFile(result['bw_image']))
    image_objs.append(anvil.media.TempFile(result['bw_image_filtered']))

print(image_objs) yields...

[<anvil.media.TempFile object at 0x7fa577e4a710>, <anvil.media.TempFile object at 0x7fa577e4ab90>, <anvil.media.TempFile object at 0x7fa53d743890>, <anvil.media.TempFile object at 0x7fa53d743ad0>, <anvil.media.TempFile object at 0x7fa53d743850>]

Thanks Ian, but when I run

print(image_objs[0].get_bytes())

alone I get the same error “AttributeError: ‘TempFile’ object has no attribute ‘get_bytes’”

edit: I was able to get a list of lazy media objects by removing the anvil.media.TempFile from the above for loop, so

image_objs is now

[<anvil._server.LazyMedia object at 0x7fbfdc690890>, <anvil._server.LazyMedia object at 0x7fbfdc6907d0>, <anvil._server.LazyMedia object at 0x7fbfdc690850>, <anvil._server.LazyMedia object at 0x7fbfdc690790>, <anvil._server.LazyMedia object at 0x7fbfdc690810>]

update:

So I got xlfswriter to take the data doing this:

image_data = image_objs[0].get_bytes()
  worksheet.insert_image(row,3,'image1', {"image_data": image_data, 'x_offset': 5, 'y_offset': 5, 'positioning': 1})

But when it gets to workbook.close(), I get this error, “AttributeError: ‘bytes’ object has no attribute ‘getvalue’”

update2: looks like the data might have to be a BytesIO object, not the data itself.

update3:

Was able to get it working with what Ian suggested, thanks again!

for i in range(0,n): #number of row
    #select only current ower rows in database
    result = app_tables.biofilm_images.search(
    owner=anvil.server.get_session_id(),
    )[i]
    row = i+1
    for j in range(0,5):
      image_data = image_objs[i*5+j].get_bytes()
      data = io.BytesIO(image_data)
      worksheet.insert_image(row,2+j,'image'+str(i)+str(j), {"image_data": data, 'x_offset': 5, 'y_offset': 5, 'positioning': 1})
1 Like