Memory usage when changing image in an ui.ImageView

mrjk

Hi all

I've written a script which loops through a list of photos, displaying the current one while some processing happens behind the scenes. I've wrapped this up into an app in Xcode using the Pythonista3AppTemplate and have noticed that the memory consumption grows every time the photo changes. Minimum code to reproduce:

# Code 1
import photos
import ui
import time

v = ui.View()
v.add_subview(ui.ImageView(name = 'img'))
v.present()

all = photos.get_assets()

while True:
    for p in all:
        v['img'].image = p.get_ui_image(size=(300,300))
        time.sleep(.5)
    time.sleep(5)

Xcode shows memory usage steps up every time get_ui_image() is called. The rate of increase is linked to the size argument.

I tried a few things to narrow down where the issue is. This one still loops and changes the image forever but it gets all the ui_images once at the start.

# Code 2
import photos
import ui
import time

v = ui.View()
img = ui.ImageView(name='img')
v.add_subview(img)
v.present()

all = photos.get_assets()[0:5]
images = [p.get_ui_image(size=(300,300)) for p in all]

while True:
    for i in images:
        v['img'].image = i
        time.sleep(.5)
    time.sleep(2)

The memory usage jumps up at the beginning, as you'd expect, but then it stays constant while it loops. This is ok for a small number of photos but impractical if you want to look at the whole photo roll.

I changed the original code so it removes the ImageView entirely to try to get some garbage collection.

import photos
import ui
import time

v = ui.View()
v.present()
all = photos.get_assets()

while True:
    for p in all:
        for sv in v.subviews:
            v.remove_subview(sv)
        img = ui.ImageView(name = 'img')
        img.image = p.get_ui_image(size=(300,300))
        v.add_subview(img)
        time.sleep(.5)
    time.sleep(5)

The memory usage still grows with every change.

I think this shows it's caused by calling get_ui_image(). But is there anything I should (or could) be doing to kick start any garbage collection here? Is this a memory leak in the Pythonista photos module? And can I mitigate it?

Grateful for any advice. J

JonB

I suspect omz might be using a caching image manager. Might need to try stopping/clearing the cache every so often?. Or, use the objc manager yourself, then you might have control of it.
https://developer.apple.com/documentation/photokit/phcachingimagemanager?language=objc

ccc

It is probably a good idea to avoid masking Python built-ins https://docs.python.org/3/library/functions.html#all

What about this approach...

while True:
    for photo in photos.get_assets()[0:5]:
        del v['img'].image
        v['img'].image = photo.get_ui_image(size=(300,300))
        del photo

mikael

@mrjk, thank you for spending the time to create a very clear question.

As @JonB said, might need to use objc_util. @cvp has done a lot with the image library in the past, and might have some code for you to use.

cvp

@ccc said:

 del v['img'].image

I get an error

    del v['img'].image
TypeError: Cannot delete attribute

cvp

But this works

	v['img'].image = None

mrjk

Yes, the use of all in my example was very sloppy. I don’t know what came over me.

Thanks for all your suggestions.

I’ve tried setting image to None and deleting the looping variable p but the increasing memory consumption continues.

I was trying to avoid using the objc because I am much more comfortable in Python but I’ll look into it this evening.

cvp

@mrjk could you tell me how you check the memory consumption, so I can also try

cvp

@mrjk said:

I’ve tried setting image to None

That is only if you don't recreate the ImageView in the loop, thus no remove subview

mrjk

@cvp

In the left panel in Xcode which usually shows the project navigation / directory structure there’s a row of icons along the top. Click the 7th one across : “Debug navigator”.

If you select the “memory” from the list it shows charts in the main panel.

mrjk

@cvp said:

@mrjk said:

I’ve tried setting image to None

That is only if you don't recreate the ImageView in the loop, thus no remove subview

Yes exactly. But I did also try doing it in addition to removing the subview.

cvp

@mrjk said:

Xcode

Ok, understood, but I don't work in Xcode, only in Pythonista on my iPad, sorry

JonB

@cvp there was some code a while back that checked memory usage..
https://forum.omz-software.com/topic/3146/share-code-get-available-memory

LukasKollmer's code I think checks system mem usage. I had some code that checked the process usage, however it likely does not work on 64 bit. Now that I have a 64 bit iPad available to me, I should probably try to get it working again.

@mrjk have you tried the same code, but without the imageview? I.e just creating the ui.Image's?

cvp

@JonB Thanks. I had seen this topic and I have tested the script with/without image=None but not really sure that it has an impact

JonB

Another few things to try:. Put this while loop inside a thread. Where you have put it, it is run on the same background thread as some other iOS background tasks.

Number 2, setting an image should technically be done on the main thread -- so wrap the code that sets v["img"].image = ... with a def wrapped with on_main_thread (from objc_util library)

Number 3:. Draining an AutoRelease pool. Blah, this problem has been encountered before and solved, lol

https://forum.omz-software.com/topic/3844/photos-asset-get_image_data-is-there-a-memory-leak/6

I'm not sure if the objc_util AutoRelease retaining thing has been fixed or if you still need to patch it. Also, I seem to remember some built in AutoRelease pool stuff in pythonista ..

Don't put this while loop in the main code -- iOS background tasks can't run until

JonB

It appears that objc_util now has an autorelease context handler:

with objc_util.autorelease():
# set the new image

Might work. I'm not sure if the other bug -- related to clearing the cached methods is still needed.

mrjk

@JonB

@mrjk have you tried the same code, but without the imageview? I.e just creating the ui.Image's?

Thanks. I hadn't but it does suffer from the same issue.

import photos
import time
all_photos = photos.get_assets()
while True:
    for p in all_photos:
        image = p.get_ui_image(size=(300,300))
        time.sleep(.5)
    time.sleep(5)

mrjk

@JonB said:

Another few things to try:.

Thanks very much. That link looks like what I've been trying to wrestle together in objc without success. I'll try to implement some of these suggestions and report back.

mrjk

This is a working version without significant leak.

import photos
import time
import concurrent.futures
import objc_util
import ui

@objc_util.on_main_thread
def update(image_view, ui_image):
    image_view.image = ui_image

def loop_photos():
    all_photos = photos.get_assets()
    while True:
        for p in all_photos:
            with objc_util.autoreleasepool():
                ui_image = p.get_ui_image(size=(300,300))
                #image_data = p.get_image_data()
                update(v['img'], ui_image)
            time.sleep(.5)
        time.sleep(5)

v = ui.View()
v.add_subview(ui.ImageView(name='img'))
v.present()

with concurrent.futures.ThreadPoolExecutor(max_workers=1) as executor:
    executor.submit(loop_photos)

Thanks very much for all the suggestions. I'm grateful for the tip about threading and will try to incorporate it into the rest of the project but the thing that does the lifting here is the autoreleasepool context manager; with just this addition and removing the threading it works just fine.