omz:forum

    • Register
    • Login
    • Search
    • Recent
    • Popular

    Welcome!

    This is the community forum for my apps Pythonista and Editorial.

    For individual support questions, you can also send an email. If you have a very short question or just want to say hello — I'm @olemoritz on Twitter.


    3D rotations in a UImage

    Pythonista
    5
    18
    223
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • madivad
      madivad last edited by madivad

      G'day guys, it's been a while since I've played with Pythonista and I was stoked today when I opened up the forums and found a beta out there. That's great!

      I have some UI ImageViews that I would like to transform the images in, about one of the 3D axis. The image has the silhouette of a phone in it and I want it to be representative of the orientation the phone is being held.

      We have access to CGAffineTransform the transforms of which cover rotation, translation, skew and scale, but I can't get it to rotate about the axis I'd like because they're 2D related.

      I have started looking at how to write the wrapper function and would seriously like some help to do it :)

      This example code:

      CATransform3D transform = CATransform3DIdentity;
      transform.m34 = 1.0 / -500;
      transform = CATransform3DRotate(transform, 45.0f * M_PI / 180.0f, 0, 1, 0.0f); 
      _bgView.layer.transform = transform;
      

      I believe that gives the rotation I am after (at this point I'm only doing a single axis, not the whole device, however I may do that later).

      I've opened up https://omz-software.com/pythonista/docs/ios/objc_util.html
      and now I'm here.

      If someone can point me into the right direction for researching the transform I actually need, I'd appreciate that. I've had my head stuck in CGAffineTransform for the last few days and I've skewed my own searching results and I just don't have the right terminology under my belt.

      This page has a lot of information that is WAY beyond my knowledge, mathematically, practically and programmatically. I realise I have NO idea which one or where to start. But if the above function is a good place to start and I need any others later, I can always extend the code and include more of the other functions myself (hopefully).

      https://developer.apple.com/documentation/quartzcore/transforms

      Any help appreciated.

      I have read that this is easy enough to achieve in scenekit(?) but I'm designing a GUI style app. It's a page of data with some icons, one of which I want to display an orientation. I don't think that scenekit wouldn't fit my needs.

      edit: I had to remove all of the links except the one I really wanted as this question got flagged as spam. I have done the research, I'm just confused about what it is I actually need.

      Thanks.

      cvp JonB 2 Replies Last reply Reply Quote 0
      • cvp
        cvp @madivad last edited by

        @madivad said

        I have read that this is easy enough to achieve in scenekit(?) but I'm designing a GUI style app. It's a page of data with some icons, one of which I want to display an orientation. I don't think that scenekit wouldn't fit my needs.

        I think that SceneKit is the best way to do what you want and you can always add a scene as a subview of an ui.View

        You would find a lot of samples of SceneKit in the forum

        madivad 1 Reply Last reply Reply Quote 0
        • JonB
          JonB @madivad last edited by JonB

          @madivad This should be doable.

          Something like this though I haven't tried yet

          from objc_util.import *
          from ctypes import *
          import math
          
          '''we need to wrap the CATransform structure '''
          def CATransform (ctypes.Structure):
              _fields_ = 
                  [{'m11', c_float},{'m12', c_float},{'m13', c_float},{'m14', c_float},
                  {'m21', c_float},{'m22', c_float},{'m23', c_float},{'m24', c_float},
                  {'m31', c_float},{'m32', c_float},{'m33', c_float},{'m34', c_float}]
          
          
          CATransform3DMakeTranslation = c.CATransform3DMakeTranslation 
          CATransform3DMakeTranslation.argtypes = [c_float, c_float, c_float]
          CATransform3DMakeTranslation.restype = CATransform
          
          CATransform3DRotate = c.CATransform3DRotate
          CATransform3DRotate.restype = CATransform3D
          CATransform3DRotate.argtypes = [CATransform c_float, c_float, c_float, c_float]
          
          identity= CATransform3DMakeTranslation(0,0,0)
          identity.m34 = -1/500
          rot = CATransform3DRotate(identity,  math.radians(45), 0,1,0)
          
          v=ui.View(frame=(0,0,800,800))
          sv=ui.View(frame = (50,5,300,300))
          sv.bgcolor = 'blue'
          v.add_subview(sv)
          v.present()
          
          SV = ObjCInstance(sv)
          layer = SV.layer()
          layer.transform = rot
          

          Pythonista interface with ctypes structures returned from ObjC methods can be kind of screwed up, so we might have to modify the argtypes of the setLayer_ method manually to get things to work in the last statement.
          I can give it a try tonight

          1 Reply Last reply Reply Quote 0
          • madivad
            madivad @cvp last edited by

            @cvp said in 3D rotations in a UImage:

            you can always add a scene as a subview of an ui.View

            I'm sorry, but this is exactly the sort of "stupid" (from me) that drives me nuts. I did not think of that, I made an assumption that it was ui or scenekit, I didn't realise you could mix and match like that. That's awesome and I'll definitely play with that idea tonight.

            @JonB excellent, that a great place for me to start. I'll definitely give that a go too, hopefully report back tonight.

            Thanks guys, cheers.

            cvp 1 Reply Last reply Reply Quote 0
            • cvp
              cvp @madivad last edited by cvp

              @madivad In @JonB code, some little typing errors:

              • not objc_util.import (no ".")
              • class CATransform instead of def CATransform
              • not [{'m11', c_float} but [('m11', c_float)
              • CATransform3DRotate.argtypes = [CATransform, c_float, c_float, c_float, c_float]
              • CATransform structure goes up to m44
              madivad 1 Reply Last reply Reply Quote 0
              • madivad
                madivad last edited by

                @JonB said in 3D rotations in a UImage:

                CATransform3DMakeTranslation = c.CATransform3DMakeTranslation
                CATransform3DMakeTranslation.argtypes = [c_float, c_float, c_float]
                CATransform3DMakeTranslation.restype = CATransform

                CATransform3DRotate = c.CATransform3DRotate
                CATransform3DRotate.restype = CATransform3D
                CATransform3DRotate.argtypes = [CATransform c_float, c_float, c_float, c_float]

                identity= CATransform3DMakeTranslation(0,0,0)
                identity.m34 = -1/500
                rot = CATransform3DRotate(identity, math.radians(45), 0,1,0)

                I'm looking through the above code and I want to understand it. This is what was going wrong when I looked at the "wrapper how to" page, it makes little sense to me.

                For starters:

                1. the 1st and 4th lines above:

                CATransform3DMakeTranslation = c.CATransform3DMakeTranslation
                CATransform3DRotate = c.CATransform3DRotate
                where does the "c." come from?

                1. How are the inputs of CATransform3DRotate and CATransform3DMakeTranslation defined? Are those functions a part of the struct? I was thinking I'd need wrappers for each and every function but I'm starting to see (but not yet understand) some of it.
                2. I am getting an error at ctypes.Structure with the carrot pointed to the dot in between. invalid syntax. weird, probably a conflict with other code in the module? import wrong? I'll post full code tonight.
                3. this line:

                CATransform3DRotate.argtypes = [CATransform c_float, c_float, c_float, c_float]
                should there by a comma between the first arguments, or is the first argument not required?
                I am getting an error about argtypes item 1 not having a from type.
                Looking through what you've done there and cursory look at the docs, I've become confused re CATransform and CATransform3D and the relationship between them all. I'll focus on it tonight. Thanks for your inputs gents. cheers.

                cvp 3 Replies Last reply Reply Quote 0
                • cvp
                  cvp @madivad last edited by cvp

                  @madivad said

                  I am getting an error at ctypes.Structure with the carrot pointed to the dot in between. invalid syntax. weird, probably a conflict with other code in the module? import wrong? I'll post full code tonight.

                  Try

                  class CATransform (Structure):
                      _fields_ = [('m11', c_float),('m12', c_float),('m13', c_float),('m14', c_float),('m21', c_float),('m22', c_float),('m23', c_float),('m24', c_float), ('m31', c_float),('m32', c_float),('m33', c_float),('m34', c_float), ('m41', c_float),('m42', c_float),('m43', c_float),('m44', c_float)]
                  
                  1 Reply Last reply Reply Quote 0
                  • cvp
                    cvp @madivad last edited by

                    @madivad said

                    How are the inputs of CATransform3DRotate and CATransform3DMakeTranslation defined? Are those functions a part of the struct? I was thinking I'd need wrappers for each and every function but I'm starting to see (but not yet understand) some of it.

                    See Apple doc, like https://developer.apple.com/documentation/quartzcore/1436524-catransform3drotate?language=objc

                    1 Reply Last reply Reply Quote 0
                    • cvp
                      cvp @madivad last edited by cvp

                      @madivad I run this code without syntax error but it crashes

                      from objc_util import *
                      from ctypes import *
                      import math
                      
                      '''we need to wrap the CATransform structure '''
                      class CATransform (Structure):
                          _fields_ = [('m11', c_float),('m12', c_float),('m13', c_float),('m14', c_float),('m21', c_float),('m22', c_float),('m23', c_float),('m24', c_float), ('m31', c_float),('m32', c_float),('m33', c_float),('m34', c_float), ('m41', c_float),('m42', c_float),('m43', c_float),('m44', c_float)]
                      
                      
                      CATransform3DMakeTranslation = c.CATransform3DMakeTranslation 
                      CATransform3DMakeTranslation.argtypes = [c_float, c_float, c_float]
                      CATransform3DMakeTranslation.restype = CATransform
                      
                      CATransform3DRotate = c.CATransform3DRotate
                      CATransform3DRotate.restype = CATransform
                      CATransform3DRotate.argtypes = [CATransform, c_float, c_float, c_float, c_float]
                      
                      
                      identity= CATransform3DMakeTranslation(0,0,0)
                      identity.m34 = -1/500
                      rot = CATransform3DRotate(identity,  math.radians(45), 0,1,0)
                      
                      v=ui.View(frame=(0,0,800,800))
                      sv=ui.View(frame = (50,5,300,300))
                      sv.bgcolor = 'blue'
                      v.add_subview(sv)
                      v.present()
                      
                      SV = ObjCInstance(sv)
                      layer = SV.layer()
                      layer.transform = rot
                      

                      Error

                      Fatal Python error: Segmentation fault
                      
                      Current thread 0x000000016be37000 (most recent call first):
                        File "/private/var/mobile/Containers/Shared/AppGroup/1B829014-77B3-4446-9B65-034BDDC46F49/Pythonista3/Documents/a8.py", line 20 in <module>
                      
                      Extension modules: pykit_io, _ui, _appex, _frameworksimporter, console, _debugger_ui (total: 6)
                      
                      1 Reply Last reply Reply Quote 0
                      • madivad
                        madivad @cvp last edited by madivad

                        TLDR; I have been researching this for hours and in checking out the image.transform options I can see that I think the transforms I need are already within pythonista, but I had chosen the wrong one initially and had never thought to look at the others. I feel quite stupid about now and am going back to the drawing board. I'm going to leave this post here because I've thought about and learned a lot and this has kind of formed my notes and I want to take the time to thank @JonB and @cvp for their time. I might revisit this code as an ObjC wrapper if I'm wrong, because I feel like I'm close to nailing it, but there is no NEED to reinvent the wheel.

                        I have to revisit the sample code I pasted at the origin because that was just plucked from SE and now I want to work out how I want to apply it. What I pasted isn't actually the code, I'm working on that part now. This is more about the learning for me at the moment and I am trying to get my head around the terminology for the transforms and matrixes. The maths is really beyond me and I'm starting to confuse a lot of these terms.

                        Looking at what @jonb has put together is based on some mistakes I think I have misstated.

                        This is where I'm thinking atm:

                        The code I pasted in the OT was from a SE site and not exactly right for my application. In learning the terms (mostly tonight) I need to MAKE the translation. At least, that's what I think I want.

                        I want the 3D equiv to CGAffineTransform, which doesn't take in the original instance (hence why I think I need the create/make version).

                        CATransform3DMakeRotation vs
                        CATransform3DRotate vs
                        CGAffineTransform <-- the 2D version
                        I think I need to use first one over the second.

                        I'm still confused by the c.CATrans.... where does the c come from in JonB's code?

                        Swapping out ctypes.Structure for just Structure worked

                        CATransform3DMakeTranslation declaration:

                        CATransform3D CATransform3DMakeTranslation(CGFloat tx, CGFloat ty, CGFloat tz);
                        

                        CATranform3DMakeRotation declaration

                           CATransform3D CATransform3DMakeRotation(CGFloat angle, CGFloat x, CGFloat y, CGFloat z);
                        

                        They are both of type: CATransform3D, declared here:

                        typedef struct CATransform3D {
                            ...
                        } CATransform3D;
                        

                        Based on the code you guys have posted, I believe this is going to translate to:

                        class CATransform3D (Structure): ### NOTE 3D
                            _fields_ = [('m11', c_float),('m12', c_float),('m13', c_float),('m14', c_float),('m21', c_float),('m22', c_float),('m23', c_float),('m24', c_float), ('m31', c_float),('m32', c_float),('m33', c_float),('m34', c_float), ('m41', c_float),('m42', c_float),('m43', c_float),('m44', c_float)]
                        

                        Now I have got it crashing too, and I think it's to do with the returned 3DTransform, I need to convert it back to Affine... which means I think I would need to take the original image and trying to 3DRotate it. It's the image from a ui.ImageView which is..... it was at this point he knew he'd stuffed up

                        I'll post more when I recover.

                        I'll be back. :)

                        cvp 2 Replies Last reply Reply Quote 0
                        • cvp
                          cvp @madivad last edited by

                          @madivad said

                          where does the c come from in JonB's code?

                          Part of Pythonista...(try "print(c) in a script which does not import anything)

                          ccc 1 Reply Last reply Reply Quote 0
                          • ccc
                            ccc @cvp last edited by ccc

                            Now add from objc_util.import * and try print(c) again.

                            This is exactly why PEP8 advises against wildcard imports.

                            1 Reply Last reply Reply Quote 0
                            • cvp
                              cvp @madivad last edited by

                              @madivad Could you try this script https://github.com/cvpe/Pythonista-scripts/blob/master/Gists/a4.py.

                              It will ask you first to select a photo from your camera roll, for example an iPhone face, but any photo would be ok.
                              Then, the script displays an empty full screen ui.View with a SceneKit subview.
                              This SceneKit shows a box representing an iPhone with your selected photo as a face.
                              You can use two fingers to rotate or scale the box.
                              And if you move or rotate your device, you will see the box doing the same.
                              There is still a limitation (which I can't actually remove), the box comes back at its initial orientation.
                              But anyway, you can see what Pythonista offers to you.

                              Feedback would be appreciated

                              JonB 1 Reply Last reply Reply Quote 0
                              • JonB
                                JonB @cvp last edited by JonB

                                @cvp
                                Looks like @cvp came to the rescue with my untested code. But here is the original, marked up with the intent. You can do google search for iOS headers CATransform3D, and get the c header files describing the various structure and functions, then just have to manually translate. The way CFUNCTION wrappers work, the doll contains the symbol, but you have to manually set return type (restype) and argument types argtypes.

                                The low level stuff is slightly trickier to work it’s compared to actual ObjC. When converting objc code, you usually don’t need to mess with any of that, because the method encodings define everything, and are built into the runtime. But when passing structures instead of objects, you often have to override restypes. Rubicon handles this cleaner than the current objc_util…

                                
                                from objc_util import *
                                from ctypes import *
                                import ui
                                import math, copy
                                
                                '''we need to wrap the CATransform structure
                                 struct CATransform3D
                                {
                                  CGFloat m11, m12, m13, m14;
                                  CGFloat m21, m22, m23, m24;
                                  CGFloat m31, m32, m33, m34;
                                  CGFloat m41, m42, m43, m44;
                                };
                                this could also have been done with a pack
                                 '''
                                 
                                class CATransform3D(Structure):
                                    _fields_ =   [('m11', CGFloat),('m12', CGFloat),('m13', CGFloat),('m14', CGFloat),
                                        ('m21', CGFloat),('m22', CGFloat),('m23', CGFloat),('m24', CGFloat),
                                        ('m31', CGFloat),('m32', CGFloat),('m33', CGFloat),('m34', CGFloat),
                                        ('m41', CGFloat),('m42', CGFloat),('m43', CGFloat),('m44', CGFloat),
                                        ]
                                
                                ''' 
                                note, c is a variable imported in objc_util, which is the cDLL for the static library.
                                '''
                                
                                '''
                                /* Returns a transform that translates by '(tx, ty, tz)':
                                 * t' =  [1 0 0 0; 0 1 0 0; 0 0 1 0; tx ty tz 1]. */
                                
                                CA_EXTERN CATransform3D CATransform3DMakeTranslation (CGFloat tx,
                                    CGFloat ty, CGFloat tz)
                                    '''
                                CATransform3DMakeTranslation = c.CATransform3DMakeTranslation 
                                CATransform3DMakeTranslation.argtypes = [CGFloat, CGFloat, CGFloat]
                                CATransform3DMakeTranslation.restype = CATransform3D
                                
                                '''
                                /* Rotate 't' by 'angle' radians about the vector '(x, y, z)' and return
                                 * the result. If the vector has zero length the behavior is undefined:
                                 * t' = rotation(angle, x, y, z) * t. */
                                
                                CA_EXTERN CATransform3D CATransform3DRotate (CATransform3D t, CGFloat angle,
                                    CGFloat x, CGFloat y, CGFloat z)
                                    '''
                                CATransform3DRotate = c.CATransform3DRotate
                                CATransform3DRotate.restype = CATransform3D
                                CATransform3DRotate.argtypes = [CATransform3D, CGFloat, CGFloat, CGFloat, CGFloat]
                                
                                ''' named const can often be retrieved using in_dll.  however, we need a mutable copy, so we dont wat to use this as identity'''
                                CATransform3DIdentity=CATransform3D.in_dll(c,'CATransform3DIdentity')
                                identity=copy.deepcopy(CATransform3DIdentity)
                                
                                
                                '''other ways to get identity:
                                
                                identity = CATransform3D()
                                identity.m11=1
                                identity.m22=1
                                identity.m33=1
                                identity.m44=1
                                
                                or
                                
                                identity=CATransform3DMakeTranslation(0,0,0)
                                
                                note, we cant use CATransformMake because that is a c macro, not an exported function
                                '''
                                
                                
                                
                                v=ui.View(frame=(0,0,500,500))
                                v.bg_color='white'
                                sv=ui.ImageView()
                                
                                sv.image=ui.Image.named('card:Clubs3')
                                sv.size_to_fit()
                                v.add_subview(sv)
                                v.present('sheet')
                                
                                layer = sv.objc_instance.layer() 
                                
                                #another way to get current transrorm
                                #identity=layer.transform(argtypes=[],restype=CATransform3D)
                                
                                identity.m34 = -1/500 #sets perspective
                                #rotate 
                                rot = CATransform3DRotate(identity,  math.radians(55), 0,1,0)
                                layer.setTransform_(rot,argtypes=[CATransform3D],restype=None)
                                
                                cvp 1 Reply Last reply Reply Quote 0
                                • cvp
                                  cvp @JonB last edited by

                                  @JonB Marvelous, as usual... Sometimes, I ask me why I try to do something here.
                                  Thanks for him and for me for your clear explanations.

                                  JonB 1 Reply Last reply Reply Quote 0
                                  • JonB
                                    JonB @cvp last edited by

                                    @cvp by the way, the crash was because I was using c_float instead of CGFloat. objc_util defines CGFloat, along with some other useful iOS c types. CGFloat is double on 64 bit (all modern version of iOS) and float on 32 bit machines (ipad 3 or maybe 4 and earlier). Also, as you guys found, I mistakenly forgot the last row of the transform.

                                    There are ways to animate the rotations.
                                    But I suspect SceneKit is ultimately easier to work with.

                                    cvp 1 Reply Last reply Reply Quote 0
                                    • cvp
                                      cvp @JonB last edited by

                                      @JonB Thanks. Hoping that @madivad will find an answer to his request.

                                      1 Reply Last reply Reply Quote 0
                                      • pulbrich
                                        pulbrich last edited by

                                        Have a look at this wrapper for sceneKit. There are plenty of examples, too. Though I made no regression tests lately it should still work, or at least help with understanding the bridging.

                                        link to the GitHub repo

                                        1 Reply Last reply Reply Quote 0
                                        • First post
                                          Last post
                                        Powered by NodeBB Forums | Contributors