Slide shows with Ken Burns effect and other linear transform

I have raised this topic already in the kdenlive-devel mailing list. Hoping that someone reads this here, I would like to propose an effect that allows linear transformation of both videos and still images. The idea came from the famous Ken Burns effect used on some screen savers and commercial slide show and video editing software where you can animate a still image by zooming into it an panning inside of it.

I have tried several other OSS programs under Linux and failed for various reasons:
- Blender ruins image quality because it downsamples every image to video output size BEFORE performing the transformation.
- Cinelerra doesn't work and is badly supported
- Jahshaka doesn't work and is badly supported

The only tool that doesn't fail is DVD-Slideshow. The math is done perfectly right but it is slow because of some parts being inefficiently implemented and the slide show having to be coded in a text file instead of a GUI editor. There exists a GUI frontend for DVD-Slideshow which is however practically unusable (and also unmaintainable because of some Visual Basic lookalike being used as programming language instead of standard stuff like Perl, Python, TCL/TK etc.)

Now Kdenlive just falls short of providing a solution as it nicely and comfortably handles still images and transitions. The only way to achive zooming and panning are however PIP transitions which produce a horrible (jittering, flickering) output, probably due to low integer accuracy. The transition is also very counter-intuitive to use and handling it slows down the work.

My proposed effect would use a linear affine transformation which is a generalized mathematical term for
- zooming
- panning
- rotation
- shear
- change aspect ratio
- perspective
The mathematical operation would just consist of a matrix multiplication of the pixel coordinates and a re-sampling of the transformed image.

The previously proposed perspective effect could be easily made a part of this function.

The editor window for the effect would, besides the parameters for the transformation, show the first and the last image of the transformed sequence. All this is not difficult to implement, just two things need to be taken care of, very seriously:

- Use FLOAT, not integer; everything else creates a flickering, jittering, pixelated mess
- Make sure that the effect works on the ORIGINAL image, not on the image after having been downsampled to output resolution. Otherwise, we would end at the same problem that Blender has.

A few more remarks:

1. I would very much like to work on this myself if just someone could give me a hint where (probably in MLT) I can find the programming interface that enables me to

- retrieve the pixels from the frame
- write transformed pixels to the new frame
- retrieve keyframe parameters for controlling the effect
- attach the filter editor to the Kdenlive GUI
- tell the filter whether to perform a fast/ugly preview or a
precision (floating point) rendering.

2. I have given up looking elsewhere for a solution. Please no comments like "try tool xyz", I have tried too many and they didn't work. This was wasted time that I could have used for developing a solution by myself.

3. Just before someone points me at tools running under Windows: I know that there exist some free (as in beer) programs. I do not have Windows and hopefully never ever have to install it on my machine. I want something that runs under Linux or is cross-platform.

Hello.
Agree that this is very important and agree with these experiences. Kdenlive i mos promising of all.
Ken Burns equivalents are the one thing missing from any mid-range (or any?) NLE in Linux.

I am a Windows-to-Linux convert and I must applaud the effort thus far. Excellent and well done. Keep up wonderful progress.
There was a program in Windows, of a similar level, called Video Studio by Ulead. Worth emulating at this level for the KBs.
Video Studio did a very nice job in the video effect process to overlay Ken Burns effects on a video. Simple and easier than Premiere, another routine I used in that other world.
Emulating even one effect would be great, 3 would be fantastic.
Best wishes.
Gwt

Very good idea.
I recently try Kdenlive, and I think that that Koen Burns effect realy miss to include stills photos in a video.

 

Actually Cinelerra does the Ken Burns effect extremely well, in just about any resolution. I've done it in 720p and 1080p. It has a pretty powerful compositing engine based on a camera-projector model with keyframes, multi-image overlays, perspective etc. Results look quite good. Of course, you have to get it running and figure out how to format your footage and rendered so Cinelerra can cope, among other things . . . I had it running pretty well on Ubuntu 8.04  64 bit.

If you're just doing still images, you could just use Cinelerra for the still sequences and then render that and import it into Kdenlive.

I'm not using Cinelerra anymore for various reasons, but, hey, for what it's worth. :)

I remember seing some commits in MLT allowing this kind of effects, but I cannot be sure of it. I may be wrong.

I added static pan and zoom, but not yet animated, which is needed for Ken Burns effect. I need to do more work on the sub-pixel/interpolated rendering to give smooth animation of pan and zoom.

I was curious if there was any progress on this effect?

You can utilize the Composite transition to get a Ken-Burns style effect.
Keyframeable zoom, pan and opacity is available.

But your milage will vary due to a nasty downside:
Interpolation between keyframes is linear (no splines!)
Consequently, multi-keyframe motion is pretty jerky and the overall result poor, sometimes unacceptable.

This is one of the key issue causing me switching between cinelerra and kdenlive on a project by project base.

Toby

Yes. Works well in 0.7.5.

You just have to keep in mind to select "black" for the second track to get it work.
The default "Auto" didn't work for me (there's no second track to overlay).

en blender si se puede hacer Burns effect

Hopefully Kdenlive's ability to do a Ken Burns effect of pan and zoom on still images will improve soon. Until then, you can get high quality and smooth pan and zoom with a FOSS app. The latest Cinelerra packages I've tried on Ubuntu Jaunty 64 bit pretty much 'just work' and are stable enough in my experience for this usage. The concepts and procedure are a little unfamiliar in Cinelerra but are easy enough once you figure them out.
1) Open up Cinelerra and set the project parameters through Settings | Parameters or by opening a new project. For instance, I will be doing a 1080p video at 1920x1080 with a frame rate of 29.9700.
2) Under Settings | Preferences | Recording Tab - at the bottom, set import of still images to desired length for your clip. You can easily adjust the clip length by dragging its begin or end frame on the timeline - but if you don't check off this box and set a time you will get a still image of only one frame or something tiny that will be hard to even locate on the timeline. I start out with a clip of 10 sec.
3) File | Load - select the still image (hotkey is 'O'). Cinelerra will do common files like PNG, JPG, and even some Raw files, I have used Canon raw occasionally. The added images will appear in the Resources window in the Media folder.
4) Paste image to a video track on timeline: select it and position the timeline cursor to the desired frame insertion point and hit 'V'; or right click on it and hit paste. Drag it to where you want it and set its duration to exactly how long you want your clip to be.
5) Make sure the Compositor window is shown. Adjust it's viewer zoom level so you can see the entire frame shown by the green rectangle with an x from corner to corner. Try a 50% zoom on HD material or a 100% zoom on DV. The zoom control is on the bottom control bar to the right of the transport control arrows. Make sure you can see the controls, which is an icon bar on the left edge. If not, right click in the viewer area and hit 'show controls'.
6) Now you need to see the camera controls that control the zoom and pan position. The icon for this is a crossed arrow with a tiny right angle in the upper right corner, 4th from the top. Hover over it, it says "Adjust camera automation" I wont go into it, but Cinelerra works with neat model for projecting and positioning images using the concepts of 'camera' and 'projector'. We will just be using the 'camera'.
6) The camera automation is a smallish window with values for x and y - 'pan' positioning of image - and z which is the zoom or size of the image.
7) Keyframes will control the pan and zoom effect. You need at least two. Position the cursor on the timeline at your first keyframe. Plan ahead and make sure you figure out exactly where you want your keyframes because it's cumbersome to move all the keyframes later. The important thing to remember is that you need to always set a keyframe for ALL 3 AXES, X, Y, and Z, or things probably won't look right.
8) To set a keyframe move to the keyframe frame and turn on the 'key' icon to activate keyframing. There is one on the compositor window and one on the timeline window. Either one will do. It glows when it's on. Now adjust the image by altering the X Y and Z values. You must set each value - even if you don't want to change it from the existing value, move it's arrow up and then back or something. The only way to make sure it is set is by moving it. You can drag the image around manually but I find that it's completely reversed dragging down moves the image up, left is right. This is because in the 'camera' model, you are moving the camera or pointing it. It's weird, so I usually end up just changing the numeric values or hitting the up-down arrows to change the values.
9) When the first zoom and pan position is set, move on to the next keyframe. Again, the keyframing mode must be on. If you are only panning, remember to joggle the zoom Z key anyway to make sure the zoom keyframe is also set, or it will go back to the default of 1. Likewise, if you are only zooming, joggle the X and Y keys to set their values. Etc.
10) You can do multipoint camera movements, like slowly zoom in one area, pause there, then pan to the right quickly and zoom back slightly at the same time, pause there, then zoom back out centered on yet another area. There is no limit to the combinations you can get. Just add more keyframes. Again, it's good to plan this out carefully including total clip length etc. Moving keyframes can be a pain because it's hard to move their position in time without screwing up their exact +/- values as well.
11) To actually see the keyframe nodes on the time line, in the multitrack window hit View | Camera X etc. You will see colored lines with squares denoting the nodes.
12) Now render the finished clip. Unfortunately Cinelerra does not have the greatest output formats. I find that the MOV format Quicktime for Linux is the most dependable. I also tend to use MJPEG as the codec, though mpeg4 also works. MJPEG looks pretty good to me and is easily imported and edited by Kdenlive. Oddly, if I play these clips straight in Movie Player or Mplayer, the aspect ratio is stretched way out as are their thumbnails. But they play and edit fine in Kdenlive or Cinelerra.

Until Kdenlive proposes a feature to perform the Ken Burns effect, here is a nice software that does the job pretty nicely:

http://www.photofilmstrip.org/home.html

It's also open-source, and the result can easily be imported into Kdenlive. The interface is particularly nice, maybe the Kdenlive developers will want to take some inspiration there.