Render Farms

I am more than willing to try out some code to create a render farm of some type. this would be great for small production houses that could utilize some old machines to crunch the numbers and help editors get their productions out faster.

You can use melt to render kdenlive projects (that is what kdenlive itself uses) and incrond to make watch folders. You will need to mindful of file paths in the .kdenlive XML files. You might need to use common file paths between systems or transform the XML to adjust the paths.

Hi,

I'm going to try and give this a go. I have a dual processor PowerMac G5 as a file server here and using melt on it to render projects out seems ideal. I've had kdenlive running on it, but things are a little flaky with output compared to the x86 version on my main workstation.

Is there a particular section of the melt docs to look at with regards to rendering a project that I've saved, I've had a quick look at the melt man page but I can't see anything obvious and I won't be able to sit down properly for a few days.

Any pointers would be most appreciated.

Thanks.

A render farm in Kdenlive is realizable. I've taken a look at the scripts that can be generated to render from. You're able to set the in- and out-points (i.e. frame number) and then you can distribute them to other computers however you like. It'd be nice to see this built-in to Kdenlive in the future so I didn't have to go through and manually do it. One way of knowing where to set the render points is to use Guides on the timeline. This is what I currently do.

One complaint I have as an editor is the thumbnails. (I'll start a new post - this is just an aside). The audio and video thumbnails take way too long to be generated. Specifically, I've never seen audio thumbnails that take so long to be generated and that look so bad. The audio waveforms SHOULD look more like those in Audacity (for example) and should be more useful for lining up clips. My editing is seriously hindered by the amount of time I have to wait to get the audio thumbnail and then it's not easy to visually inspect them and see where clips line up without a lot of searching back and forth.

>kmauser, why not do this with a simple bash script? No need to do it manually.
Create a renderscript to start with in Kdenlive.
Open the renderscript.sh file and you'll see the in and out points in plain text;
"MELT="in=13"
PARAMETERS="out=26 /usr/bin/melt atsc_1080p_25 avformat..."

Create a bash script that takes this file, changes in and out points with a given span, save out as a new file, repeat until the end.

Yeah, that's what I do currently. I just generate the scripts from within Kdenlive, edit them in plain text, and then run them on the command-line. I have a few machines and use NFS and ssh into them and execute everything, so I'm able to use a renderfarm with Kdenlive. What I meant was that I use the Guides to help me determine what my in- and out- points should be so that I can send jobs to different computers. I've experienced that if I don't overlap the rendering by at least 10 frames (i.e. in=0, out=1000 on computer 1 and in=990, out=2000 on computer 2) that the final file doesn't play smoothly. I just use cat to stitch all the parts together. Maybe someone knows the reason for this, I don't and haven't looked into it.

Anyway, It'd be nice to see Kdenlive have some of this built-in though. For instance, helping to setup the NFS shares, making sure that Kdenlive and all dependencies are installed on the rendering (client) machines, etc. It'd be best to have this feature within the Render options of Kdenlive. It could also get fancy and determine the number of cores and/or CPU frequency on the rendering computers and then "optimize" the sections it decides to give to certain computers.

I've used Cinelerra in the past for several reasons - one of them being that it has a renderfarm. You're able to define the rendering machines and the number of jobs you want to create among those hosts.

I've started writing some code to accomplish my ideas and make it a bit easy on me in the long run. I'm just writing it in something called Gambas because it is easy for me. If others are interested, I can share what I have once I've polished it up a little bit.

Lastly, I'm new to the forums here so I'm not sure where to post new ideas. I apologize if this isn't the correct place. I tried searching the other day but didn't find any place definitive.

I second the wish for a render farm to be added to Kdenlive. After all, isn't that what Kdenlive is all about, making video editing easy by being a GUI to MLT. So why not an interface to a render farm? I have four networked dual core Linux machines here in the building, and I would love to put them all to use on a render project.
Imagine, video rendered in 1/4 of the time. 8)

+1 on that, definitely. I've got access to a cluster of servers in a datacenter that aren't doing much of anything during the night, and I'd love to be able to send render jobs to it.

Yeh, +1 too.

Problem with render scripts is that the paths to the clips are absolute in the .mlt file. I have most clips on an external USB hdd which I don't want to plug into the server I want to render the movie on every time. A client/server version of melt would be awesome, or perhaps kdenlive as a network client to a melt/libmlt service on another machine.

Are there any plans regarding this?

> Problem with render scripts is that the paths to the clips are absolute in the .mlt file.

Could you not just use NFS shares? Keeping the path absolute shouldn't be that big of a deal.

I just started looking into this tonight.. I bought a new HD video cam and thought it would be a lot easier to just do my edits on my little x120e ThinkPad and push the rendering off to the three Atom 330 machines I have running throughout the house. So far it looks like information on setting up a renderfarm is pretty sparse. Anyone have any suggestions for more detailed info?

(first post btw)