face_detector.py strange behavior
Other way of obtaining video camera frames continuously is welcome.
For data processing or just displaying as a preview?
@JonB For data processing.
ok, i do see some issues..
first, we need to implement the didDropSampleBuffer delegate method, and display the reason for a late frame.
second, we need to set the minFrameDuration, so that the delegate is never called more than you need it. on older versions, i believe this was in output.minFrameDuration. in newer iOS versions, i believe you set on the connection, output.connections.videoMinFrameDuration
third, there is an issue regarding what dispatch queue things are called on. or, the delegate needs to always return as fast as possible, calling heavy lifting on another thread, and dropping data otherwise. i think what is happening is that your delegate gets called on a background thread, and just keeps falling farther and farther behind. it might be better to ensure the delegate is called on the main thread.
ill post a gist later with some improvements.
pavlinb last edited by pavlinb
I have a version based of Cethric's. I'll clean it up and post later. I've implemented the drop frame callback so you can see what the issues are.
One problem I have found is that the various ways to set minFrameDuration don't work. That means the callback gets called at a high rate.
It seems that setMinFrameDuration is in camera.captureOutput object ('AVCaptureVideoDataOutput').
But there is also setActiveVideoMinFrameDuration in camera.captureDevice ('AVCaptureDevice').
I couldn't make them working too both.
increase FRAME_PROC_INTERVAL until you stop seeing FrameLate constantly. or set to 1 to go as fast as possible. you will probably see FrameLate, but if you start seeing OutOfBuffers or something like that, it means you are doing some processing and forgetting to release the capture buffer.
there are many hoops to jump through to get the actual min frame interval set, i think i have all the steps correct, though am on an older device. see if it works for you. I have a DESIRED_FPS that you can change to see if you can get <30 fps..
does this one hang? if so, what message do you get on the first frame when it comes back?
This one works: link
from ctypes import * from objc_util import * class CMTime(Structure): _fields_ = [('value', c_int64), ('timescale', c_int32), ('flags', c_uint32), ('epoch', c_int64)] AVCaptureDevice = ObjCClass('AVCaptureDevice') _device = AVCaptureDevice.defaultDeviceWithDeviceType_mediaType_position_( 'AVCaptureDeviceTypeBuiltInWideAngleCamera', 'vide', 1) dur = _device.activeVideoMinFrameDuration(restype=CMTime, argtypes=) dur.value *= 3 _device.lockForConfiguration_(None) _device.setActiveVideoMinFrameDuration_(dur, restype=None, argtypes=[CMTime]) _device.unlockForConfiguration()
@JonB 350 passes and still works. I'll play with parameters to see how it will go.