Optical Flow Node

This node analyzes a clip connected to its input using an Optical Flow algorithm. Think of optical flow as a per-pixel motion vector that matches up features over several frames.

The computed optical flow is stored within the Vector and Back Vector aux channels of the output. These channels can be used in other nodes like the Vector Motion Blur or Vector Distort. However, Optical Flow must render twice when connecting it to a Time Stretcher or Time Speed node. These nodes require the channels A. FwdVec and B. BackVec in that order, but Optical Flow generates A. BackVec and A. FwdVec when it processes.

If you find that optical flow is too slow, consider rendering it out into OpenEXR files using a Saver node.

Optical Flow Node Inputs

The Optical Flow node includes a single orange image input.

• Input: The orange background input accepts a 2D image. This is the sequence of frames for which you want to compute optical flow. The output of the Optical Flow node includes the image and vector channels. The vector channels can be displayed by right-clicking in the viewer and choosing Channel > Vectors and then Options > Normalize Color Range.

Optical Flow Node Setup

The Optical Flow node analyzes the frames connected to the background input. Trimming a Loader or MediaIn to only the range you need prevents analyzing unnecessary frames. The output of the node can then be connected to any node that takes advantage of vector channels, such as a Time Stretcher.

Alternatively, if you find the Optical Flow node too slow to analyze the frames, consider rendering it out to an OpenEXR format using a Saver node. Then import the rendered EXR file as your new image with embedded vector channels.

Optical Flow Node Controls Tab (Advanced)

When you add an Optical Flow, Repair Frame, or Tween node to a Comp, a Method drop-down menu in the Inspector allows you to choose between an Advanced GPU-based algorithm or a Classic CPU-based algorithm. This Advanced method is the same Optical Flow algorithm used in other DaVinci Resolve pages.

Warp Count
Decreasing this slider makes the optical flow computations faster. To understand what this option does, you must understand that the optical flow algorithm progressively warps one image until it matches with the other image. After some point, convergence is reached, and additional warps become a waste of computational time. You can tweak this value to speed up the computations, but it is good to watch what the optical flow is doing at the same time.

Iteration Count
Decreasing this slider makes the computations faster. In particular, just like adjusting the Warp Count, adjusting this option higher will eventually yield diminishing returns and not produce significantly better results. By default, this value is set to something that should converge for all possible shots and can be tweaked lower fairly often without reducing the disparity’s quality.

Smoothness
This controls the smoothness of the optical flow. Higher smoothness helps deal with noise, while lower smoothness brings out more detail.

Half Resolution
The Half Resolution checkbox is used purely to speed up the calculation of the optical flow. The input images are resized down and tracked to produce the optical flow.

Optical Flow Node Controls Tab (Classic)

By choosing Classic from the Method drop-down menu in the Inspector, you can use the older CPUbased algorithm to maintain compatibility with Comps created in previous versions. This method may also be better suited for some Stereo3D processing.

When using the Classic method, a single slider at the top of the Inspector improves performance by generating proxies. The remaining Advanced section parameters tune the Optical Flow vector calculations. The default settings serve as a good standard. In most cases, tweaking of the advanced settings is not needed. Many deliver small or diminishing returns. However, depending on the settings, rendering time can easily vary by 10x. If you’re interested in reducing process time, it is best to start by experimenting with the Proxy, Number of Iterations, and Number of Warps sliders and changing the filtering to Bilinear.

Proxy (for Tracking)
The Proxy slider is used purely to speed up the calculation of the optical flow. The input images are resized down by the proxy scale and tracked to produce the optical flow. The computational time is roughly proportional to the number of pixels in the image. This means a proxy scale of 2 will give a 4x speedup, and a proxy scale of 3 will give a 9x speedup.

Smoothness
This controls the smoothness of the optical flow. Higher smoothness helps deal with noise, while lower smoothness brings out more detail.

Edges
This slider is another control for smoothness but applies it based on the color channel. It tends to have the effect of determining how edges in the flow follow edges in the color images. When it is set to a low value, the optical flow becomes smoother and tends to overshoot edges. When it is set to a high value, details from the color images start to slip into the optical flow, which is not desirable. Edges in the flow end up more tightly aligning with the edges in the color images. This can result in streaked-out edges when the optical flow is used for interpolation. As a rough guideline, if you are using the disparity to produce a Z-channel for post effects like Depth of Field, then set it lower in value. If you are using the disparity to perform interpolation, you might want it to be higher in value.

Match Weight
This control sets a threshold for how neighboring groups of foreground/background pixels are matched over several frames. When set to a low value, large structural color features are matched. When set to higher values, small sharp variations in the color are matched. Typically, a good value for this slider is in the [0.7, 0.9] range. When dealing with stereo 3D, setting this option higher tends to improve the matching results in the presence of differences due to smoothly varying shadows or local lighting variations between the left and right images. The user should still perform a color match or deflickering on the initial images, if necessary, so they are as similar as possible. This option also helps with local variations like lighting differences due to light passing through a mirror rig.

Mismatch Penalty
This option controls how the penalty for mismatched regions grows as they become more dissimilar. The slider provides a choice between a balance of Quadratic and Linear penalties. Quadratic strongly penalizes large dissimilarities, while Linear is more robust to dissimilar matches. Moving this slider toward Quadratic tends to give a disparity with more small random variations in it, while Linear produces smoother, more visually pleasing results.

Warp Count
Decreasing this slider makes the optical flow computations faster. In particular, the computational time depends linearly upon this option. To understand what this option does, you must understand that the optical flow algorithm progressively warps one image until it matches with the other image. After some point, convergence is reached, and additional warps become a waste of computational time. The default value in Fusion is set high enough that convergence should always be reached. You can tweak this value to speed up the computations, but it is good to watch what the optical flow is doing at the same time.

Iteration Count
Decreasing this slider makes the computations faster. In particular, the computational time depends linearly upon this option. Just like adjusting the Warp Count, adjusting this option higher will eventually yield diminishing returns and not produce significantly better results. By default, this value is set to something that should converge for all possible shots and can be tweaked lower fairly often without reducing the disparity’s quality.

Filtering
This option controls filtering operations used during flow generation. Catmull-Rom filtering will produce better results, but at the same time, turning on Catmull-Rom will increase the computation time steeply

Optical Flow Node Settings Tab

The Settings tab in the Inspector is also duplicated in other Optical Flow nodes. These common controls are described in detail in the Optical Flow Nodes Settings Tab.

About the Author

Justin Robinson is a Certified DaVinci Resolve, Fusion & Fairlight instructor who is known for simplifying concepts and techniques for anyone looking to learn any aspect of the video post-production workflow. Justin is the founder of JayAreTV, a training and premade asset website offering affordable and accessible video post-production education. You can follow Justin on Twitter at @JayAreTV YouTube at JayAreTV or Facebook at MrJayAreTV