I just tried those exact two codes and still got a brightly coloured glitchy video. Gonna investigate ffmpeg problems.
I'm not a linux nut, but I just prefer to use native applications, but in this situation looks like virtualdub is the only good option!
Would you be able to send me your output video so I can compare?
I've used the Deshaker plugin, but there are a few caveats unless things have changed in recent times. Those are:
Deshaker plugin works in RGB colourspace only, no YCbCr, YV12 or YUY2.
VirtualDub will scale the levels when it converts video colour spaces to RGB which is far from desirable, so best to use AVISynth to do a full range conversion, preferably via 3D LUT.
It is also (for me anyway) not desirable to do multiple transcoding and poor colour space conversions through the tool chain just to try and deshake a video that I really should have used either in built camera IS or tripod or rig. :-)
Finally Deshaker works best with video shot at high shutter speeds at and above 1/200th/sec which is total nonsense with regard to sticking to 180deg shutter rule, ie if your shooting 25fps then the shutter speed would be 1/50th/sec, if shooting at 60fps (for smooth slow motion as example) the shutter speed would be 125th/sec. I never get anywhere near 1/200th, thats nuts.
Most important though to me anyway is to first convert to uncompressed image sequences and a proxy at the same time, using AVISynth to do the colour space conversion to RGB, full range levels, then deshake in VDub, then continue with image sequences right through to final encode back to delivery codec like h264, whatever.
I've had mixed success, minor stabilisation is ok, rolling shutter jelly cam stays just that. :-(
> I've used the Deshaker plugin, but there are a few caveats unless things have changed in recent times
Interesting Caveats. IMHO those are applicable to very advanced users.
Having compared nominal videos (taken hand held, and also using a monopod, with a Canon HF S10 Leigra digital video camcorder) and compared the output before and after applying the wine/VirtualDub deshaker plugin, I have a firm opinion that in many cases the improvement using deshaker, despite your caveats, is significant. ie I won't compose a video project again without checking for a deshaker improvement (and in 90% of the cases the improvement from deshaker in the videos that I take is massive).
Now I would like to see a comparable (for quality) Linux package that is as easy to use (and by no means is deshaker that easy, but it is functional). Note I am a total clutz with MS-Windows apps, having left Windows for Linux in 1998, but I am aware of no multiple transcoding being necessary with deshaker, unless one is referring to 2 passes. One can also specify a very high bit rate, such that such two pass transcoding has IMHO less of an effect, as most computers can not handle the high bit rate anyway (so nothing really noticeable is lost)
I also note that every Linux suggestion for stabilization I've tried has left me completely puzzled and unable to apply Linux stabilization as the authors/packagers giving advice as to how to use the packages assume a level of knowledge well beyond anything I'll ever take the time to learn.
I do hope all those working on Linux packages succeed, but please, when you get it to a level of reasonable functionality, please keep in mind that not all of us want to learn the difference between RGB, YCbCr, YV12 or YUY2.
Anyway, its an interesting thread.
A two pass stabilizer is better, because it can react to "future" events - but because of the two passes, it can not be implemented as a simple pipeline type plugin. Therefore, the best stabilization will never be as simple as a brightness adjust...
BTW, the two passes do not increase the number if image conversions, since the image is only transformed on the second pass.
A single pass stabilizer is possible, the mjpegtools stabilizer and the (lost?) centipede's frei0r plugin are of this type.
A short summary of linux video stabilization possibilites that I know, would be:
--------------------------------------------------------------------------------
MJPEGTOOLS:
command line tool, basic single pass, no de-rotation, no empty edge fill, no rolling shutter compensation
CENTIPEDE:
GUI Frei0r tool based on openCV, currently of unknown availability, basic single pass, no de-rotation, no empty edge fill, no rolling shutter compensation.
TRANSCODE:
command line tool, two pass, does de-rotation, does some edge fill (the webpage says it can also use previous frames for edge fill, but it did not look like that to me - checked the source code and saw that it is just the unmoved current frame), no rolling shutter compensation.
VIRTUAL DUB UNDER WINE:
GUI, two pass, does de-rotation, does edge fill using previous and future frames, has some kind of rolling shutter compensation. (the stabilization plugin is "external" to virtual dub, and as far as I know, not open source.)
Of course, this is not a native linux application, and it is a bit of work to get it to run under wine.
My experience (not that extensive...) with them is the following:
In most cases, I use Transcode, which does a pretty good job. My only gripe is the primitive edge compensation. If I can find the time, I plan to make a patch for that and send it to the author... I have also stabilized some HV30 stuff, and did not miss the rolling shutter compensation very much (some people say it is questionable how much can be done about that at all...).
I am quite satisfied with the results, got good results in most cases, where I shot the video handheld while standing still.
Also does a good job of taking out the remaining shake from walking shots with my manfrotto modosteady.
I am no adrenaline guy myself and have no experience with that, but I can understand that with extremely jerky helmet-cam action video (like the above bycicle video) there are limits to what post-festum stabilization can do. But still, while the results probably won't be perfect, the "watchability" of the video will certainly be increased.
@oldpc "but please, when you get it to a level of reasonable functionality, please keep in mind that not all of us want to learn the difference between RGB, YCbCr, YV12 or YUY2.", no problem, but to 'learn' the differences between the colourspaces would take a book, however 'appreciating' the differences and the sequence of events from when we import video to when we encode out is a benefit to maintaining some sort of quality control, especially with regard to colour correction/grading and problem solving.
I understand your comment but please don't assume just because many are amateurs or using kdenlive out of interest in a private capacity that we shouldn't strive for the best we can achieve. :-)
The reference to mulitple encoding / transcoding was simply that when import video into Vdub in one format we have to get the 'deshaked' video back out and if we are then intending putting it back into an NLE, how does that happen? Do we frameserve into kdenlive? Not possible. It has to be reencoded again.
It's all well and good if we deshake our final movie then encode out from Vdub but that's a bit daft if our 'shaky' video element is a fraction of the whole movie, or do we choose the frame sequences from our final movie and deshake them then reencode our edited colour corrected movie back out to final again?
@Marko, I shot some handheld, holding my cam as still as I could manage. Deshaked it all on best settings throughout, set rolling shutter % as per 550D and still got jelly cam out. :-)
But good to hear others having more success. :-)
Marco, thanks for the summary !
Yellow, your points noted.
I hope some day to learn enough to be as capable of doing this video processing as the two of you. For me its simply difficult (currently not possible) to use anything other than the deshaker/virtual dub method, as all other methods assume a level of video understanding that I for one do not have (and hence the stabilization efforts with anything other than virtualdub consistently fail).
I note almost 6 months have gone by since the last post on this stabilization subject ? Has there been any progress in making this more user friendly for the average user?
At the moment I am still using wine/VirtualDub/deshaker-plugin as my stabilization method, applying it to clips that need stabilizing. Needless to say an integrated method in Linux if functionally superior would be preferable.
Even a basic guide as to a Linux "only" stabilization method would be a helpful Linux only start.
As far as I know, there is nothing new in this department....
One method that I forgot to mention above, is a possibility to stabilize video using cinelerra. (Just google "cinelerra stabilization").
It is single pass, does de-rotation, no empty edge fill, no rolling shutter compensation.
In contrast to the other possibilities above, it uses only a single area tracking, and is very "manual".
I tried various method here and I report my experiences :
- VirtualDubMod threw wine wasn't really productive for me, it need to much specific input files / large output / no multithread for my usage. May be I'm wrong using it and don't know how to setup it perfectly, but I spent 2 or 3 hours to make it works without a good result (in my opinion).
- Transcode + vid.stab were my last test, and it's fast and do the job. Thanks to this tutorial : http://314bits.com/blog/2010/09/stabilize-video-in-ubuntu-linux/ , it took only few minute to setup it and do my first conversion on ubuntu. Why I didn't found this link before ? That's why I'm posting it to this famous thread on Google when searching about stabilisation/deshaking video on Linux/Ubuntu.
Deevad, thanks for the http://314bits.com/blog/2010/09/stabilize-video-in-ubuntu-linux/ . I followed that guide, spent an hour trying different combinations of the transcode command and could not get one to work. I had the entire gammet of errors , and the best i could get was a green pattern of lines flashing across an output.
Its simply not user friendly enough, and IMHO needs a wrapper script for anyone who does not want to dive into the details of transcode to get this to work. I don't have the patience to spend more than an hour on anything that is as unproductive as that was for me. For example, in contrast, the wine method with VirtualDub only took me 15 minutes or so to get running and stabilizing videos.
So I'll go back to the wine method with virtual dub and deshaker which works well for me. My guide for that is still here: http://forums.opensuse.org/forums/english/get-technical-help-here/how-fa...
But I appreciate the post, and I hope others have better success than I. Its good to see a pure linux approach, even if it is totally useless to me at this stage of its development.
Maybe someday someone will create a wrapper script or a gui around it (with appropriate prompts/checks to ensure wrong arguments/values are not used).
@oldcpu
Did you try the "--mplayer_probe" flag with transcode? It made it work for me.
Here is a blog that describes the use of the "--mplayer_probe" flag: http://isenmann.wordpress.com/2011/03/22/deshaking-videos-with-linux/
I just started with transcode, and I am trying to find the optimal settings. It seems to not work on the beginning part of the file, and then it kicks-in suddenly. I am going to experiment with different files.
h-munster, no I confess I had not read of the "--mplayer_probe" flag.
Following your advice, and using the link you referenced, I gave it a try. Indeed that makes a world of difference, and the first time I tried stabilizing with transcode using that flag it worked.
I'm both impressed and grateful.
Thankyou.
To my surprise video stabilization using transcode now works out of the box on Debian unstable with transcode from debian-multimedia.org. At least MJPEG and MPEG4 clips got a bit more stable even without the probe tricks. Edge compensation, if I that's the correct term, is visible in shaky frames as the edges seem to be moving around more than the center of screen and skiing clips have some trouble finding enough contrast but this seems like great start.
This function is in MLT now and hopefully to Kdenlive by the end of the year.
http://sourceforge.net/news/?group_id=96039&id=302819
Clicking "Play" on the linked demo video says
"This video has been removed by the user" :-(
Marko, do you mean link on sf "MLT gets video stabilization" http://www.youtube.com/watch?v=8vFTy90U2SQ ??
It displays fine here :-/
Dan, the videostab plugin for MLT is great news. Thanks.
How do you think kdenlive is gonna integrate it?
I guess not as a timeline filter since it's far from real-time.
Maybe a proxy like approach would be more applicable.
Let us know if we can help bringing this feature in kdenlive.
I encountered the problem with the one here: http://vstab.sourceforge.net/
but the one on youtube works now.
I tried searching and found the mlt/melt info. Ran a stabilization on one of my clips. Anybody have a link to command line options for melt?
I copied from ddennedy's amd marco's examples. and entered:
melt -verbose -profile quarter_ntsc_wide 100_0032.MP4 out=1000 -filter videostab -consumer xml:100_0032-vstab.mlt all=1 real_time=-2
melt 100_0032-vstab.mlt
melt 100_0032-vstab.mlt -verbose -consumer avformat:100_0032-vstab.webm properties=webm real_time=-2 threads=3
(my video was 100_0032.MP4)
How do I find out about different options, formats and properties? Anyone have a link to more information on videostab filter? I've messed with transcode and melt/videostab - but am just googling and guessing.
melt --help will give the options, funny I've too being looking at melt on the CLI for this but at the moment have to encode out was hoping dragging the source clip and analyised mlt file into kdenlive would surfice but it crashes kdenlive, so waiting for integration into kdenlive interface. :-)
I'm on a integration in kdenlive for the stabilization. i will commit a brach with that in later. First this will simple call the simple commandline. Later all adjustable parameter will be changeable.
Parameter to use can be viewn with
melt -query filter=videostab
melt -query filter=videostab2
I'm looking forward to the integration, but that might be real tricky! Thanks for the hints. I shot video on a little Kodak Playsport ZI10 in 720P at 30FPS (really 29.97). I'm giving the following a shot:
melt -verbose -profile atsc_720p_2997 100_0032.MP4 -filter videostab -consumer xml:100_0032-vstab.mlt all=1 real_time=-2
melt 100_0032-vstab.mlt
melt 100_0032-vstab.mlt -verbose -consumer avformat:100_0032-vstab.mp4 properties=mp4 real_time=-2 threads=3
I already ran the original clip through transcode, so I'll take a look to compare transcode versus mlt/videostab. But - wow!!!!! This is brutally slow on my tiny Netbook. I'm thinking any app that calls it might need a word of warning - get a cup of coffee, take a nap, head off to the office and check when you get back. I know I'm going at it underpowered, but it takes some time.
Original Clip: http://www.youtube.com/watch?v=mfXTzxMYjXA
Stabilized with transcode (just default settings): http://www.youtube.com/watch?v=vWBQdk0U1xg
Stabilized with melt videostab filter: http://www.youtube.com/watch?v=SZNvEm_32eI
Stabilized with MELT using videostab2 filter: http://youtu.be/a88TuT5d8PA
(edited - I messed something up. The original clip is much larger in size than the transcoded and melt'ed versions. I must have done something wrong. I went from about 22 megs down to 5-6 megs)
In all reality, I checked out a small, lightweight tripod with the legs shock-corded together. For pan and sweeps standing in one location, I'm going to use a tripod (still need to figure out the best way to handle the speed of the pan and sweep though). I'll still try to learn more about video stabilization, as I want to take clips while hiking or biking and those will be pretty unsteady. I didn't want a tripod or a monopod, as I picked up a pocket camcorder specifically so it would fit in my shirt pocket and I could carry it anywhere. However, I need to do something to clean up the motion in my clips.
I have one of those Gorilla pods with the bendy legs and a bean bag type thing both are great when travelling light to gain a bit of support on a rock, fence, tree whatever.
Also with the Gorilla pod I bend the legs right round upwards each side of the camera and use them like handle bars spacing my hands further apart from the camera and a little offset front to back it helps stop some of the rotation of the camera. Not ideal but can help.
Yellow - thanks for the tip!! When I bought the little Kodak ZI10, I also bought a Sony external clip on mic and a small desktop tripod. I looked at the Gorilla pod, but decided on a small plastic Ultra-pod II. It hadn't even occurred to me to mount the camera on the pod to hold the camera steady. Sure - I figured using it on a desktop would be cool for a mini-interview. So, I mounted the little camcorder on the ultra-pod, held two of the legs of the pod, rather than the camera itself - and wandered through the house shooting at 720P, 60FPS. Tried viewing it on my Netbook, no go. My wife is out for the day so I hooked up to the new Mac Book Pro we bought a few days ago. She may not be getting that Mac book Pro back!
Anyway, thanks for the tip. That made a huge difference, and I might be able to use software to filter out the remaining jitter and shake. Not about to reshoot my last project (it was a one hour round-trip, and a four hour clean-up whittled down to 24 short clips further whittled down to 2-3 minutes of video). But will use that method the next time out.
So - pay attention to shooting methods first to cut down on shake. Then use image stabilization on digital video to clean it up.
Absolutely, shooting methods and practice, including minimizing panning :-) is most important, there's no magic button to fix after. Even hardware assisted GPU stabilization solutions take time away from editing and are not necessarily a one hit fixes all. :-) But we all have occasions a stabilize tool is inevitable.
My personal preference with regard to panning is not to :-), but to take 3 or 4 shots of various points of interest in the view rather than a pan l/r or r/l and cut between them or dolly a couple of them with a small slider. Panning can feel uncomfortable, difficult to get steady unless using a fluid head tripod and I think it takes far too long in the length of the video to go across the pan and keep it comfortable, compared to a more snappy cut between a few short clips of interest.
All personal choices though, no right and wrong way. :-)
marco, not wishing to jump the gun but I see the Stabilize options in kdenlives Project menu in latest svn but when I try applying it to a clip in the Project Tree or for a clip on the Time line nothing appears to happen. I guess this is work in progress?
I do have the stabilize options in MLT on the CLI.
the new code that works is in the stabilize branch of kdenlive.
if someone would help testing ( it should work, but it is yet not possible, to change the default parameters ATM)
if is seems to work for you i could merge this into master
marco, unfortunately I'm not building from source but instead using sunabs svn PPA on Ubuntu, I'll try building the stabilize branch to test, many thanks.
Hi Yellow, I'm using svn in Ubuntu 11.10 in Gnome2 environment. I've tested the stabilize but I see no changes.
I've used a clip 1920x1080 50i and the same converted at 1280x720 25p but I don't see any processing starting. (i've transcode already installed).
What's missing?
you dont need transcode for that, the code is in mlt too.
it takes a (maybe very long) time to see the progressbar changing.
after that you have a myvideo.mp4.mlt file from myvideo.mp4.
When you clip "add to project" the resulting .mlt file will be added to project also.
so could you test this with a bit more time ;)
i'll commit some changes to adjust all the parameters for the filters
I don't see any interface. Again, no progress bar is present and CPU activity is normal.
hmm, did you update some minutes ago ?
i pushed some changes into stabilize branch.
I've seen now. This morning were not available. However nothing is changed for Stabilize. Or first I choose to stabilize AND AFTER I MUST transcode in another format?
Marco, when running from the command line, I saw that I needed to run melt first, it generated an mlt file (that I could watch) then you ran melt again to end up with a stabilized video. On my under-powered netbook it took a significant amount of time, and I named the output clip with a new name. (it took me at least 30-45 minutes to stablize a small clip, but I have a slower netbook)
As integrated into KDenlive, do you still do a multi-step process? Does it make a new clip? Or does the .mlt file go into the timeline for viewing, then you get the stabilized clip on final rendering? I was just wondering how it worked so that when stabilize hits the sunab ppa I'd know what to expect.
@betaversion:
you must not convert anything before. Just add clip to project, right click, stabilize , stabilize(transcode) (which mean with the code from transcode , no transcode itself needed)
@callmebruce
yes, like the command line options for that, the stabilize procedure like written before will make exact this. When you click "start", it will add the file to a playlist, attaches the videostab or videostab2 filter and render this to the video.mp4.mlt file. This File is the same type as the command line xml version. When you clicked "add to project" this .mlt file will be added to project. You can view this new mlt file or use this in timeline. The original file will not be changed or modified. Means: you have 2 files after stabilize 1 file.
I'd like to build the stabilize Branch, but can't install a recent MLT version due to compatibility issues with my AV Linux system. Because of that I use to build Kdenlive with the build script. Is there a way to modify the build script to build the stabilize instead of the master branch?
after a first short loo you could change
KDENLIVE_HEAD=1
KDENLIVE_REVISION=
to
KDENLIVE_HEAD=0
KDENLIVE_REVISION=stabilize
If so, I just see no effect calling stabilize function.
Hmm, I have compiled the Stabilize-Branch via Build-Script now, but the stabilizing entries in project menu still don't seem to have a function for me.
but it should. i'v commited some changes now to stabilize branch and would merge to master if some test where done.
right click on a clip should show a menu stabililze -> videostab(transcode). After that you should see a dialog and you could start stabiizing.
I also made some changes to videostab2. It has now some SSE2 optimizing, which work 10 times faster in stabilizing (when you have sse2 enabled during mlt compile). I'll push this changes to github, and ask wo merge this in mlt-master these days. i hope you could see now some changes, else it will be later in master branch.
BTW: you need an mlt version with vdieostab filters in, else you will not see the menu
what can i do, to get mlt vdieostab filter?
latest git version of mlt should have it in
i'll test this now under linux , and if it works for the "main" platform, i'll merge this to master
Hi Marco.
I just tried the videostab branch. For my tests, I downloaded a shaky video from the net:
http://www.youtube.com/watch?v=nxRzpFuSxbQ (there is a download link below the video).
Tried to deshake it in Kdenlive using videostab(transcode) leaving all parameters to the default values, and the resulting .mlt clip crashes Kdenlive and MLT when trying to play it...
By the way, the UI for the videostab(transcode) filter would really need work... I think sliders should have a numeric field beside so that user knows the value for later use, and angle values could use a dial widget (http://doc.qt.nokia.com/latest/widgets-sliders.html), some documentation (tooltip or similar) would also be necessary...
The UI stuff can be done later, but we cannot afford to have a filter that results in Kdenlive committing suicide... maybe you can try to reproduce with the clip I mentionned.
Hi
i tried your video (on a mac, tested on command line and kdenlive) it creates the mlt file without segfault. the .mlt file is playable.
kdenlive is having problems creating the thumb and proxy. (but it is a playable simple mlt file)
did you update kdenlive(stabilize) before test.(some timer issues and some more where in earlier today).
Yes the ui is a mess. it is first only for having something to adjust.
Best would be to reuse the effectsliders. mlt paramters could be read direct from the mlt repository.
But for this whe need to save the pointer to Mlt::Repository and give this to the classes.
but it should not crash with the latest code of mlt(videostab) and kdenlive(stabilize)
i'll test the file on linix with latest versions:
it did crash
too low contrast! No field remains.
(no translations are detected in frame 872)too low contrast! No field remains.
the problem will be (i think) the videostab is for videos that are uncut, else it would be difficult to find the next frame shake, when it is complete different image)
i think it i a bug in the filter, which occours when using multiple threads (pointer is deleted in one, use in another thread).
No changes yet. (What's wrong?)
as j-b-m said, there issues with that.
maybe i should not call mlt direct as lib, but else start a nnew process, so that a segfault will not kill kdenlive itself ?
i dont want to make kdenlive instable with this feature.
It's a nonsense. The other transcode functions works good, Why this code must be unstable?
i'd feel better, if someone could test this branch too and tell if it works or not.
Should we do a bigger rewrite to reuse the code from effectstack in this filter ?
i'd tried to find one bug in videostab2 when replying the stabilized video, but this is only in some rare cases.
i'll take a look next days, or some idea from others ?
I'm agree. A single (not KDE) user can't give a true idea if it's really working or not.
Someone else?
I just found this discussion. September 2011 I started working on my father's 8mm home movies. I have about 8000 feet transferred in 1920x1080 M-JPEG (lossless). I discovered deshaker and got it running using virtualdub under Wine.
I came up with this work flow.
In virtualdub 2 passes are required for deshaker
1) Drop all the even frames because I couldn't convince the guy who did the transfer to leave the frame rate alone and just record each image only once. Deshaker doesn't work with the extra duplicate frames. Follow this filter with deshaker pass one.
2) Drop the even frames again to match pass one. Follow this with deshaker pass two. Since I was working with old film I followed with the old film color restoration filter and a dirt removal filter on the stabilized image. The output of pass 2 was saved to disk using a lossless compression codec.
The result of this is a stable, better color reel of film as a smaller file that works well with kdenlive instead of the massive original M-JPEG. All NLE is done on the stabilzed file. I don't transcode until all editing, additional image manipulation, color grading, and effects are done. Then I transcode and make a DVD to give relatives.
I discovered that the original video from the film transfer was still on the server of the guy who did it for me, so I had him transfer it to another hard drive. This data is very useful because the movie camera actually put a larger image on the film and the transfer process actually captured that image. There are a couple of white squares in the left side corners for sprocket holes in the film, but there is additional image to fill in part of the frame as it moves around. Even on a tripod the old 8mm movie film has a lot of mechanical slop and less than perfect registration as it moved through the camera. Deshaker is provides some viewer comfort when watching old film and it looks much better.
