Memory problems

21 posts / 0 new

Last post

Wed, 06/27/2012 - 15:54

normcross

Offline

Joined: 04/02/2011

Posts:

Memory problems

Has anyone else noticed any memory problems with 12.04 64b?

When I first installed 12.04, just using Unity and system monitor with nothing else, it was using up about 1gb an hour doing nothing so I dumped Unity (would have anyway) for Kde and Gnome(no effects)

Just had a nice little holiday on The Rhine, so I threw a whole folder full of clips at Kdenlive, about 350 actually. Kdenlive worked it's way right through the whole lot perfectly but then things got sluggish. Checking my memory, it had used up about 7.4Gb of ram plus 2Gb swap and I had to reboot to get it back.

Today I was putting together a 2.5 minute clip, all the usual moving clips about and a few transitions and effects etc. Took about 2 hours. Then I noticed my slowmo clips starting to judder so I checked my memory. All used up, had to reboot.

So, not sure where the problem is.

Average:

Your rating: None

Wed, 08/08/2012 - 19:07

orbspider

Offline

Joined: 07/26/2011

Posts:

Re: Memory problems

Hi I'm using kdenlive 0.9.2 and my RAM slowly gets sucked up while working on a project until 3900Mb of my 4gb is used and then the desktop inevitably freezes n very hard to kill kdenlive if task man not already open cos can't get to xkill by Alt-F2 or launcher button...

debian wheezy with LXDE

what the problem is, I dono yet either.

edit:
this is actually becoming a pain, because after making a 5min project working with AVCHD, my ram reached 3400Mb and I thought it better to quit kdenlive and restart it to render the project -as one time the rendering got to 95% and the desktop froze with no more available ram.

Thu, 08/09/2012 - 12:03

normcross

Offline

Joined: 04/02/2011

Posts:

Re: Memory problems

Tell me about it!!

I put in an extra 4GB giving me 8GB. Working on a Project, memory usage creeks up and up and when I want to Render, if it has reached halfway I have to save and re-start Kdenlive.

The sad part is, I'm treating this as the norm, now.

I must admit, this started sometime after upgrading to Ubuntu 64 bit , but I can't say for certain if it was immediate.

Kdenlive sunab svn.
Ubuntu 12.04 64 bit
Gnome (no frills)

Thu, 08/09/2012 - 12:24

hvdwolf

Offline

Joined: 07/08/2009

Posts:

Re: Memory problems

It has obivously something to with memory leaks from (I think) either MLT or kdenlive.
Hopefully someone with debugging skills will take a look. If not the best thing to do is create a bug ticket describing this behavior.

Thu, 08/09/2012 - 13:58

normcross

Offline

Joined: 04/02/2011

Posts:

Re: Memory problems

Hi hvdwolf,

You said :- If not the best thing to do is create a bug ticket describing this behaviour.

And I did, on the 29th June last.

I just work round bugs now. Adds a little more excitement to the Project :-)

Sun, 08/12/2012 - 20:03

OpenThinking

Offline

Joined: 02/20/2011

Posts:

Re: Memory problems

It works for me in Kubuntu 64-bits with kdenlive build with the build script. I haven't edited very large projects lately though. kdenlive seems to release all memory when I closes it.

Sun, 08/12/2012 - 22:42

yellow

Offline

Joined: 09/09/2009

Posts:

Re: Memory problems

I think the issue is the desktop grinding to a halt and freezing whilst kdenlive is open rather than whether it releases memory when it closes, I recently had instances of kdenlive being open with a reasonable sized project but not actively editing and had the desktop grind to a halt and kdenlive crash outright all of it's own accord after perhaps 20mins. :-) Ubuntu 12.04 x64

Mon, 08/13/2012 - 06:07

ddennedy

Offline

Joined: 06/26/2007

Posts:

Re: Memory problems

A big memory leak in MLT was fixed recently:
http://sourceforge.net/tracker/?func=detail&aid=3551459&group_id=96039&a...

Mon, 08/13/2012 - 07:31

yellow

Offline

Joined: 09/09/2009

Posts:

Re: Memory problems

Excellent. Many thanks.

Sun, 08/19/2012 - 12:09

#10

orbspider

Offline

Joined: 07/26/2011

Posts:

Re: Memory problems

I c the mem problem is fixed in the latest MLT from git.. that's good

but I can never get git to work on my wireless connection, with fw disabled as well.

Cloning into 'mlt'...
fatal: unable to connect to github.com:
github.com[0: 207.97.227.239]: errno=Connection timed out

Sun, 08/19/2012 - 11:37

#11

normcross

Offline

Joined: 04/02/2011

Posts:

Re: Memory problems

On kdenlive-svn +MLT-git, it says MLT has been waiting to build for 2 days now.

Mon, 08/20/2012 - 15:57

#12

cryophallion

Offline

Joined: 02/20/2008

Posts:

Re: Memory problems

I have been suffering from this as well. My 4gb of ram is used up in minutes. When rendering down HDv to a dvd, it runs out of memory around 50 megs into the final file. The fact that the bug report above mentions "it is not flat" makes me really nervous. I mean, I've got that much leaking that fast, it doesn't exactly make me hopeful. I have a project that I need to finish, but this is obviously not working.

I tried sunab's svn, but I still have the same issue.

I'm trying to track it down a bit though. I made a liveusb of some kubuntu images to test. Both 32 and 64 bit versions of 12.04 are affected. I tried a baseline render on 11.10, but the vob wasn't creating properly. However, a render to another hdv video DID work with no memory leak, which means it may not be there. I will try it on the usb drive with the newest version and see if it is a ubuntu conflict or not. May also try mint to see if I get lucky there.

Either way, this isn't a memory leak, it's a memory flood.

EDIT: Ok, so, while I can't seem to create a vob in 11.10, it WILL render an avi with no leak using kdenlive 8 (which is from the base repos). Upgrading to the newest version has the leak. Tried mint maya, and it had the same issue (not surprising, considering that it uses the same repos).

So, I hope he fixed the right one, because this qualifies as a showstopper to me.

Mon, 08/20/2012 - 20:06

#13

cryophallion

Offline

Joined: 02/20/2008

Posts:

Re: Memory problems

I noticed this too. I just tried building it myself, but I get a build error in sox. Working to figure out if I am missing dependencies...
factory.c:81:45: error: missing binary operator before token "("
filter_sox.c:41:44: error: missing binary operator before token "("

EDIT: Got it to compile, but kdenlive crashes. I'll have to investigate further.

Mon, 08/20/2012 - 20:31

#14

cryophallion

Offline

Joined: 02/20/2008

Posts:

Re: Memory problems

Ok, I got kdenlive to run. I ran the project with the git version, and I will admit it's better... but certainly not fixed. Melt lasts about twice as long, but that is only about 100mb into a dvd conversion. The memory leak is still definitely there. Let me know what you need from me for a good bug report (like how to make the png graphs), and I'll be happy to provide it.

Wed, 08/22/2012 - 01:13

#15

cryophallion

Offline

Joined: 02/20/2008

Posts:

Re: Memory problems

Alright, so I decided to test on Openshot to see if there was a chance that would work. Shockingly, it will render fine in openshot. However, I loathe the user interface and usability of the program (I first edited on premiere 6.5, kdenlive follows this design rather well). Openshot does everything in seconds instead of the ususal hour:min:sec:frames interface.

Either way, it rendered fine in openshot, but I'm not sure how they interfaced with melt (esp since the ps is named openshot for the render, and on kdenlive it is actually melt). And in case anyone would like to see how the render was supposed to go, here is the script, in which I just tried it with one video:

#! /bin/sh

SOURCE="file:///home/david/Projects/Video/kdenlive/scripts/script001.sh.mlt"
TARGET="file:///home/david/Projects/Video/kdenlive/title.vob"
RENDERER="/usr/bin/kdenlive_render"
MELT="/usr/bin/melt"
PARAMETERS="-pid:3936 in=45204 out=3136 $MELT atsc_1080p_2997 avformat - consumer:$SOURCE $TARGET f=dvd vcodec=mpeg2video acodec=ac3 s=720x480 vb=6000k maxrate=9000k minrate=0 bufsize=1835008 packetsize=2048 muxrate=10080000 ab=192k ar=48000 g=18 me_range=63 trellis=1 mlt_profile=dv_ntsc_wide pass=1 threads=2 real_time=-1"
$RENDERER $PARAMETERS

The mlt file is attached.

Attachment	Size
script001.sh_.mlt	21.36 KB

Wed, 08/22/2012 - 15:15

#16

orbspider

Offline

Joined: 07/26/2011

Posts:

Re: Memory problems

My renders are often getting up to 95% ram use! If they finish about there then they make it. I'm not sure if the ram use stops increasing there but I had a 13min h264 render that failed at 96% ram use.

so, right now, I make sure my h264 are shorter than 7 or 8 min, which is fine for mp4 video.. but vob needs to be longer so it doesn't halt on the dvd while playing, so I just make the short vob's and concatenate them afterwards (with an easy Thunar right click custom action actually: cat > NEW.vob %F that joins the selected files)

my h264 renders are hitting 450mb @12Mbit/s (master copy later put thru handbrake)

I normally get ram back by restarting kdenlive, but now a few times that hasn't been enough, and I have to log-out.

and I need to get ram back down to 200mb mark to make renders. But rendering from scripts is vital to success, with no project loaded, and sometimes exiting other app's, especially chrome!

sure hope this bug will be fixed.

Wed, 08/22/2012 - 17:25

#17

normcross

Offline

Joined: 04/02/2011

Posts:

Re: Memory problems

Using the latest sunab ppa, I did a little test.

I rendered a 1080 HD project which was using 39 clips and lasted 3mins 25secs.

Memory used after boot :- 419 MiB
Load Kdenlive:- 570 MiB
Load Project:- 794 MiB
Rendering peeked at :- 2.0 GiB

So that's about 1.2 GiB to render 3min 25secs. The trouble is, I can't remember what it used to use because I never had to worry before.

Fri, 08/24/2012 - 04:26

#18

orbspider

Offline

Joined: 07/26/2011

Posts:

Re: Memory problems

I switched back to rendering to mpeg2 10Mbits and it's saving me a lot of time and also melt isn't hitting the RAM ceiling

my start up ram use is between 190-220mb! lxde :)

btw, concatenating the vob's gave problems in bombono, like the new vob was shown as length of one of the concat'd vobs.. also can't generate dvd

Sat, 08/25/2012 - 13:04

#19

hvdwolf

Offline

Joined: 07/08/2009

Posts:

Re: Memory problems

Normcross said:
"So that's about 1.2 GiB to render 3min 25secs. The trouble is, I can't remember what it used to use because I never had to worry before."

What if you uninstall the 0.9.2 (or 0.9.3 sunab ppa), temporarily remove/deactivate the sunab ppas, and reinstall the stable 0.8.x version from your Ubuntu and rerun the same project (if it is compatible). What memory consumption do you get with that version?

- EDIT: and another thing: Do you use multiple threads?
I have a quad-core i7 with a possibility for 8 threads. So I use 2 threads for proxies and 3-5 threads for encoding (experimenting a little) as this really speeds up things.
Encoding h264 (mpeg4 AVC) uses (much) more memory (and processing power) then mpeg-4 ASP or mepg-2. I can imagine that when using multiple threads for encoding h264 the memory consumption will increase with equal amounts compared to mpeg4 ASP or mpeg-2. I haven't checked b.t.w.

Thu, 09/06/2012 - 19:03

#20

cryophallion

Offline

Joined: 02/20/2008

Posts:

Re: Memory problems

Some new info. So, I tried rendering my 1080i video again with the new version down to a vob. Sadly, memory leak still happens all too soon. But then I tried the bluray profile posted on another thread. That one never got above 30%, (it did use more cpu though). While the video didn't render the audio properly, which I am checking on (too slowly, as it is over two hours, and it takes 5 hours to render on my laptop), it;s interesting that it may be a melt issue within a certain profile.

Thu, 09/06/2012 - 19:09

#21

DiveO

Offline

Joined: 09/26/2011

Posts:

Re: Memory problems

Sorry to ask: but what amount of "memory" are you measuring exactly? With a virtual memory manager in place on the os side and many things going on behind the scene I always found it to be rather difficult in my system architect career so far to measure memory consumption on standard operating systems ... the only things were I could reasonable measurements are certain embedded system with fixed memory mapping and a single process/multithread setup, where everything is clear and mapped only once.

Memory problems

Online users