# THE UNIVERSITY OF WESTERN AUSTRALIA ## :pushpin: Preview * Optical Flow and Motion Field * Tracking * Applications * Optical Flow Constraints * Lucas Kanade Tracking Algorithm * Mean Shift Algorithms ## :stars: OPTICAL FLOW AND MOTION FIELD * Motion Field: * Projection of 3D relative velocity vectors onto the image plane. Optic Flow: * Observed 2D displacement of pixels (brightness patterns) in the image. {% hint style="success" %} Most of the time, motion field is what we want to find. But optic flow is what we actually measure from the image. {% endhint %} {% hint style="success" %} $$Optic Flow \neq Motion Field$$ {% endhint %} ### :barber: Barber's pole illusion ![](https://2797135393-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-M99TkJnCKaTvw9u9gQJ%2F-MAoS8lIG9ucEeSjSn-t%2F-MAoSjqD3wVMfd1bvpxv%2Fbarber%2Bpole%2Billusion.jpg?alt=media\&token=0e2b9e3b-35cf-4404-9232-21cb33308e75) ### Aperture Problem * We can only measure the component of optic flow in the direction of intensity gradient. * We can not measure the component that is tangent to the intensity gradient. ![](https://2797135393-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-M99TkJnCKaTvw9u9gQJ%2F-MAoVbGJZP77EwdDFxCv%2F-MAoVoJb3uVDShBryOXt%2FAperture.png?alt=media\&token=365ebb50-8df6-4921-b2f3-5bf4c3e8add9) ### The Optical Flow Constraints ![](https://2797135393-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-M99TkJnCKaTvw9u9gQJ%2F-MAoVbGJZP77EwdDFxCv%2F-MAoX-KY9e25I4KY_dNy%2FOpticalFlowConstraints.PNG?alt=media\&token=6db8a44f-833a-4307-97b1-036363674807) ![](https://2797135393-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-M99TkJnCKaTvw9u9gQJ%2F-MAoX2M-zHJMlSRAqq_z%2F-MAoYS87BkXInAKpN7mz%2FOpticalFlowConstraint2.PNG?alt=media\&token=3d8afa08-6964-41e0-bb1f-32be5d6198e8) ![](https://2797135393-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-M99TkJnCKaTvw9u9gQJ%2F-MAoYrIr7SisHjLQtC_4%2F-MAoZZ0rYbEQr8zSufXG%2FOpticalFlowConstraint3.PNG?alt=media\&token=7abc9966-9f6e-4672-bbbe-394a0d344a77) ![](https://2797135393-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-M99TkJnCKaTvw9u9gQJ%2F-MAoYrIr7SisHjLQtC_4%2F-MAo_AqT0BByXuTGpY8s%2FOpticalFlowConstraint4.PNG?alt=media\&token=aa5d08b2-7b27-4025-b045-c9e6a7919d42) ### Techniques for Computing Optical Flow ![](https://2797135393-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-M99TkJnCKaTvw9u9gQJ%2F-MAoYrIr7SisHjLQtC_4%2F-MAo_z2cOjCgXb7Kgths%2FTechniquesForComputingOpticalFlow.PNG?alt=media\&token=a5748cb8-e160-470f-9dab-2a9ec6c5f11d) ### Differential Technique: Lucas Kanade Motion ![](https://2797135393-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-M99TkJnCKaTvw9u9gQJ%2F-MAogQ9xSA56MAIHyprT%2F-MAogfT7r32Q-L23c3_E%2FDifferentialTechnique1.PNG?alt=media\&token=aa4cf92f-8c0e-4372-ab16-7557e732d1bd) ### Solving The Aperture Problem ![](https://2797135393-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-M99TkJnCKaTvw9u9gQJ%2F-MAogQ9xSA56MAIHyprT%2F-MAoh7CUcssWLpqalnso%2FSolvingTheAperturePRoblem.PNG?alt=media\&token=451f6cc9-ce34-45c4-9ac3-d7fc65972a75) ![](https://2797135393-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-M99TkJnCKaTvw9u9gQJ%2F-MAogQ9xSA56MAIHyprT%2F-MAohtG_L3XrgQJJFUx9%2FSolvingTheAperturePRoblem2.PNG?alt=media\&token=839ffba0-7a44-4aa5-aef7-0b34d9ff3d2c) ### Lucas Canade Optical Flow ![](https://2797135393-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-M99TkJnCKaTvw9u9gQJ%2F-MAogQ9xSA56MAIHyprT%2F-MAoigM2dc0NnyWRKEBX%2FLucasKanadeOpticalFlow.PNG?alt=media\&token=b76de890-cbdc-4f17-b26e-1d56d8f218e3) ![](https://2797135393-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-M99TkJnCKaTvw9u9gQJ%2F-MAojZD-AQARTRfqlkur%2F-MAojtmFI3TGC1eRkeeS%2FLucasKanadeOpticalFlow2.PNG?alt=media\&token=baa1148f-f4ca-4c5c-94d6-cec8c3bcb03c) ### Implications ![](https://2797135393-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-M99TkJnCKaTvw9u9gQJ%2F-MAogQ9xSA56MAIHyprT%2F-MAojPMLTSLTVZbhF_vi%2FImplication.PNG?alt=media\&token=d07e35dc-2048-4661-a43f-148329e5649f) ### Farneback's Two Frame Optical Flow ![](https://2797135393-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-M99TkJnCKaTvw9u9gQJ%2F-MAojZD-AQARTRfqlkur%2F-MAokQPxS7zvf2Eo3nVX%2FFarneback'sTowFrameOpticalFlow.PNG?alt=media\&token=f339561a-33d2-484f-925f-4211be71d00b) ![](https://2797135393-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-M99TkJnCKaTvw9u9gQJ%2F-MAojZD-AQARTRfqlkur%2F-MAolXvT9vEIXfm3yJep%2FFarneback'sTowFrameOpticalFlow2.PNG?alt=media\&token=e0f9875b-da8b-4378-8578-877e8f014af4) ## TRACKING * Using optical flow, we can track pixels or corners over multiple frames * Sometimes we don't want to track every pixel or every corner * We mat want to track a specific object such as * A tennis ball * Pedestrians * Cars * Or simple blobs ![](https://2797135393-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-M99TkJnCKaTvw9u9gQJ%2F-MAomEuECnUluB3R_Ih_%2F-MAooizNFPlemGUWnDEi%2FTracking.PNG?alt=media\&token=edc7bcd1-6816-487c-901f-7ba163068861) ### Lucas-Kanade Tracking ![](https://2797135393-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-M99TkJnCKaTvw9u9gQJ%2F-MAomEuECnUluB3R_Ih_%2F-MAopZmMrwlwXUr8N6fo%2FLucas-KanadeTracking.PNG?alt=media\&token=4c755071-4b89-4b08-95a0-293bffad014c) ### Tracking with Mean-Shift ![](https://2797135393-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-M99TkJnCKaTvw9u9gQJ%2F-MAomEuECnUluB3R_Ih_%2F-MAoqZDu1ZsXN1qjT6WS%2FTrackingWithMeanShift.PNG?alt=media\&token=834e194e-37e5-4318-9c1a-331aac2440ae) ![](https://2797135393-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-M99TkJnCKaTvw9u9gQJ%2F-MAomEuECnUluB3R_Ih_%2F-MAor2VFKcC9VKJWLUlT%2FTrackingWithMeanShift2.PNG?alt=media\&token=5c2fff9d-b607-4b32-a4b3-47fd90fb7a0d) ![](https://2797135393-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-M99TkJnCKaTvw9u9gQJ%2F-MAomEuECnUluB3R_Ih_%2F-MAorJ7Y8O4gJIdU4umf%2FTrackingWithMeanShift3.PNG?alt=media\&token=627863c8-148c-4033-bf73-a6a00733332a) ### Mean Shift and Colour Models ![](https://2797135393-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-M99TkJnCKaTvw9u9gQJ%2F-MAomEuECnUluB3R_Ih_%2F-MAorxkquFEIQSE92JeJ%2FTrackingWithMeanShift4.PNG?alt=media\&token=d539340b-97ab-4761-96ef-0442c71ab50a) ### Mean Shift On Weight Images ![](https://2797135393-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-M99TkJnCKaTvw9u9gQJ%2F-MAomEuECnUluB3R_Ih_%2F-MAosk0GAS141t9WejSB%2FTrackingWithMeanShift5.PNG?alt=media\&token=14ab4316-0fab-49df-a239-7df7c6718620) ![](https://2797135393-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-M99TkJnCKaTvw9u9gQJ%2F-MAomEuECnUluB3R_Ih_%2F-MAosrpraBDviZSUkJ1-%2FTrackingWithMeanShift6.PNG?alt=media\&token=eb4e4b73-d3e8-4360-ac32-c87aae31044b) ### CamShift (Continuously Adaptive Mean Shift) ![](https://2797135393-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-M99TkJnCKaTvw9u9gQJ%2F-MAot6HQlaDztf19BJOG%2F-MAotqheYsIele4oP5ce%2FCamShift.PNG?alt=media\&token=45f8fa32-6820-4342-b520-14f15fa154fd)