Load PIL image in scene with retina resolution

upwart

I would like to paste PIL images with load_pil_image to a retina screen at full resolution (e.g. 2224 * 1668 for a 10.5" iPad Pro). It now works actually at half that resolution, ie. 1112 * 834). Is there any way to use the double resolution (in Scene)?

mikael

@upwart, you could experiment with converting to ui.Image first. Or using an ui.ImageView instead of scene_drawing.

mikael

@upwart, also, how are you measuring the resolution of the image at different points of your code? Maybe share some code?

upwart

@mikael
Essentially my code is

                capture_image = Image.new("RGB", (1122, 834, (0, 0, 0))
               #  build up the image with Pillow
                ims = scene.load_pil_image(capture_image)
                scene.image(ims, 0, 0, *capture_image.size)
                scene.unload_image(ims)

This results (on an iPad Pro 10.5") in a full screen picture.

I would like to double the resolution, so build a Pillow image of 2244 x 1668 pixels and then copy that to the screen.

Any idea how to do that?

By the way, this is part of a much bigger open source simulation library that supports animation on both tkinter and Pythonista platforms. If you are interested have a look at www.salabim.org, where you will find the link to the GitHub repository.

brumm

look for screen scale

upwart

I can see the screen scale.
But how do I put a Pillow image to a scene in pixels and not points?

mikael

@upwart, sorry, moving slowly, but just to confirm:

If you create and display the image as you show, image quality is not sufficient
If you use a full-scale image, only one quarter of the image is shown?
You are committed to using scene
Do you have some fps requirement?

brumm

PhotoTextV2 Class

upwart

@mikael
Here are the answers to your questions:

If you create and display the image as you show, image quality is not sufficient
Sometimes it is not sufficient and then double resolution would be helpful

If you use a full-scale image, only one quarter of the image is shown?
Yes, that's true.

You are committed to using scene
Yes, because I use the same code base for tkinter I have to rely on a PIL images that are shown in a loop.

Do you have some fps requirement?
Not really, because my package automatically corrects for missed frames, but as it is an animation, there are some performance requirements, for sure.

mikael

@upwart:

Here is one way. You can update the texture of self.bg in an update method.

Note that since we go via ui.Image, we could do this all in the ui module as well, and I am not sure if the performance meets your requirements.

This is not perfect (you see me fudging with the value 102), let me know how it works for you.

import scene
import Image, ImageDraw
import ui, io

class MyScene (scene.Scene):
    ims = None
    def setup(self):
      scale = scene.get_screen_scale()
      full_width = int(self.size[0]*scale)
      full_height = int(self.size[1]*scale)
      full_size = (full_width,full_height)
      im = Image.new("RGB", full_size, (0, 0, 0))
      draw = ImageDraw.Draw(im)
      
      # Draw lines to show we have every pixel
      for y_block in range(int(full_height/scale)):
        for y_in_block in range(int(scale)):
          colors = ('white','black','black')
          y = int(y_block*scale+y_in_block)
          draw.line(((0, y), (full_width,y)), fill=colors[y_in_block])
          
      # Draw lines to show we are seeing the whole picture
      draw.line(((102, 102),(full_width-102,102),(full_width-102,full_height-102),(102,full_height-102),(102,102)), fill=128)
      del draw
      
      with io.BytesIO() as fp:
        im.save(fp, 'PNG')
        img = ui.Image.from_data(fp.getvalue(),scale)
        self.bg = scene.SpriteNode(scene.Texture(img))
        self.add_child(self.bg)
        self.bg.position = self.size / 2

scene.run(MyScene())

upwart

@mikael
Thank you so much for the code you provided.
Based on that I have made a small proof of concept in which I let your rectangle shrink with one point on every draw().
I do that by just changing the texture of self.bg.
That works excellent.
I suppose due to the overhead of

        im.save(fp, 'PNG')
        img = ui.Image.from_data(fp.getvalue(),scale)

, I can reach just 4 frames per second.
Not ideal, but enough for me to try and implement it in my salabim package.

Thanks a lot.

upwart

@mikael
I just found out that if change the file format from PNG to BMP speeds up the animation significantly: from 4 to > 16 fps.

mikael

@upwart, excellent, thank you for the info. Do you know if the BMP format handles transparency?

JonB

How are you generating the image?

See https://forum.omz-software.com/topic/5155/real-time-audio-buffer-synth-real-time-image-smudge-tool

This has the fastest methods to update images although in a ui.View. you can get 60 fps or higher with a recent device.

Not sure if this would work with scene.

upwart

@JonB
The PIL image is created for each frame by combining several (sometimes hundreds or thousands) small PIL images. For non-retina resolution it is easy to show them in a scene event loop. For retina resolution I have to escape to a SpriteNode which is filled with a texture for each and every frame.

upwart

Now that I can display the built up picture (as a SpriteNode), I run into another problem. In my scene.draw method, I now update the SpriteNode's texture and then I want to draw some rectangles and texts with the scene_drawing primitives rect and text.
Unfortunately these do not show up at all, as if they are hidden behind the SpriteNode. I tried to set the SpriteNode's z_position to -1, but that doesn't help.
So my concrete question is: "How can I show the scene_drawing results in front of the SpriteNode?".

cvp

@upwart As I'm very far to be a Scene specialist, I'm not sure that scene_drawing is compatible with SpriteNode.
Perhaps, you could try path and ShapeNode.

Or perhaps a shader with SpriteNode, like

        img = ui.Image.named('test:Peppers')
        self.bg = scene.SpriteNode(scene.Texture(img))
        rect_shader = '''
          precision highp float;
          uniform sampler2D texture;
          varying vec2 v_tex_coord;
          void main( void ) {
           vec2 uv = v_tex_coord;
           vec4 rect = vec4(0.2, 0.3, 0.4, 0.5);
           vec2 hv = step(rect.xy, uv) * step(uv, rect.zw); 
           float onOff = hv.x * hv.y;	// off if in rect
           vec4 color = texture2D(texture, vec2(uv.x, uv.y)); // original color
           gl_FragColor = mix(color, color+vec4(1,0,0,0), onOff);texture2D(texture, vec2(uv.x, uv.y));
           }
           '''
        self.bg.shader = scene.Shader(rect_shader)

cvp

@upwart and this?

        self.bg = scene.SpriteNode(scene.Texture(img))
        self.red = scene.ShapeNode(ui.Path.rect(0, 0, 150, 150), 
                             parent=self.bg,
                             fill_color = (1,0,0,0.5),
                             position=(10,10))

upwart

@cvp
Well, that's maybe the way to go, although I prefer to stick with my old fashioned scene_drawing code.

Thanks just the same.

cvp

@upwart said:

Thanks just the same

What do you want to say?