0002556: No support for multiple audio streams

(0007953)
j-b-m (administrator)
2012-04-02 20:19

Well we will need some more infos about that file format. The first step is to check if FFmpeg supports it. If it does, then check if MLT can handle that and then we can make it work in Kdenlive.

Can you test the FFmpeg side, and if it works, attach a small sample clip here... so we can make further tests.

(0007955)
FishB8 (reporter)
2012-04-02 22:03

FFmpeg does handle it, provided it's recent enough to support ProRes. MLT also seems to handle it fine given that kdenlive imports it without any issues. It just sees the audio as a mono audio source since it ignores all but the first audio stream. (Did you have any specific test for MLC you would like done?)

There can be up to 8 audio streams if the audio source is from audio embedded in the SDI signal.

I will try to record a short clip for you that I can send in. All the files I have now are in the 50GB - 100GB range. I don't want to chop them up with ffmpeg since it might alter some things.

Here's the output from ffprobe, which should give a pretty good idea of what is in the streams.

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'SC1ATK18.mov':
  Metadata:
    major_brand : qt
    minor_version : 537199360
    compatible_brands: qt
    creation_time : 2012-03-23 06:42:45
  Duration: 01:02:06.96, start: 0.000000, bitrate: 136658 kb/s
    Stream #0:0(eng): Audio: pcm_s24le (in24 / 0x34326E69), 48000 Hz, 1 channels, s32, 1152 kb/s
    Metadata:
      creation_time : 2012-03-23 06:42:45
      handler_name : ?Linux Alias Data Handler
    Stream #0:1(eng): Audio: pcm_s24le (in24 / 0x34326E69), 48000 Hz, 1 channels, s32, 1152 kb/s
    Metadata:
      creation_time : 2012-03-23 06:42:45
      handler_name : ?Linux Alias Data Handler
    Stream #0:2(eng): Video: prores (apcn / 0x6E637061), yuv422p10le, 1920x1080, 134351 kb/s, 29.97 fps, 29.97 tbr, 2997 tbn, 2997 tbc
    Metadata:
      creation_time : 2012-03-23 06:42:45
      handler_name : ?Linux Alias Data Handler
    Stream #0:3(eng): Data: none (tmcd / 0x64636D74)
    Metadata:
      creation_time : 2012-03-23 06:42:45
      handler_name : $Libquicktime Time Code Media Handler
      timecode : 01:14:57:07
Unsupported codec with id 0 for input stream 3
[STREAM]
index=0
codec_name=pcm_s24le
codec_long_name=PCM signed 24-bit little-endian
codec_type=audio
codec_time_base=1/48000
codec_tag_string=in24
codec_tag=0x34326e69
sample_fmt=s32
sample_rate=48000
channels=1
bits_per_sample=24
id=N/A
r_frame_rate=0/0
avg_frame_rate=0/0
time_base=1/48000
start_time=0.000000
duration=3726.960292
nb_frames=178893916
TAG:creation_time=2012-03-23 06:42:45
TAG:language=eng
TAG:handler_name=?Linux Alias Data Handler
[/STREAM]
[STREAM]
index=1
codec_name=pcm_s24le
codec_long_name=PCM signed 24-bit little-endian
codec_type=audio
codec_time_base=1/48000
codec_tag_string=in24
codec_tag=0x34326e69
sample_fmt=s32
sample_rate=48000
channels=1
bits_per_sample=24
id=N/A
r_frame_rate=0/0
avg_frame_rate=0/0
time_base=1/48000
start_time=0.000000
duration=3726.960292
nb_frames=178893916
TAG:creation_time=2012-03-23 06:42:45
TAG:language=eng
TAG:handler_name=?Linux Alias Data Handler
[/STREAM]
[STREAM]
index=2
codec_name=prores
codec_long_name=ProRes
codec_type=video
codec_time_base=1/2997
codec_tag_string=apcn
codec_tag=0x6e637061
width=1920
height=1080
has_b_frames=0
sample_aspect_ratio=N/A
display_aspect_ratio=N/A
pix_fmt=yuv422p10le
level=-99
timecode=N/A
id=N/A
r_frame_rate=2997/100
avg_frame_rate=2997/100
time_base=1/2997
start_time=0.000000
duration=3726.960294
nb_frames=111697
TAG:creation_time=2012-03-23 06:42:45
TAG:language=eng
TAG:handler_name=?Linux Alias Data Handler
[/STREAM]
[STREAM]
index=3
codec_name=unknown
codec_long_name=unknown
codec_type=data
codec_time_base=1/30
codec_tag_string=tmcd
codec_tag=0x64636d74
id=N/A
r_frame_rate=0/0
avg_frame_rate=0/0
time_base=1/2997
start_time=0.000000
duration=3726.960294
nb_frames=1
TAG:creation_time=2012-03-23 06:42:45
TAG:language=eng
TAG:handler_name=$Libquicktime Time Code Media Handler
TAG:timecode=01:14:57:07
[/STREAM]
[FORMAT]
filename=SC1ATK18.mov
nb_streams=4
format_name=mov,mp4,m4a,3gp,3g2,mj2
format_long_name=QuickTime/MPEG-4/Motion JPEG 2000 format
start_time=0.000000
duration=3726.960294
size=63665127029
bit_rate=136658557
TAG:major_brand=qt
TAG:minor_version=537199360
TAG:compatible_brands=qt
TAG:creation_time=2012-03-23 06:42:45
[/FORMAT]

(0007956)
j-b-m (administrator)
2012-04-02 22:42

From the MLT's avformat producer doc, the "audio_index" property should allow you to choose the audio stream you want.

For example:

melt myclip.mov audio_index=1

Should play the first audio stream. Setting audio_index to "all" should be played in the audio track:

melt myclip.mov audio_index=all

Is that what you want? In Kdenlive, the clip properties (in the "advanced" tab), you can force the audio index to a specific value, however there is no way to set it to all. This should be easily fixed if that is what you need.

(0007957)
FishB8 (reporter)
2012-04-03 00:25

Yes, that is sort of what I need. Most of the time I need both audio channels. The audio index does let me select either of the two streams, but not both.

What would be nice (but require more work) is audio channel mapping. Since kdenlive is currently limited to stereo audio, it would be nice to be able to map any audio stream->channel to the left and right.

The easiest way to do that, that I can think of is to have in the advanced settings, an "Audio Left Mapping" and "Audio Right Mapping" with two number selections, first to select stream, and second to select channel in the stream.

Default would probably be:

Audio Left Mapping: Stream-0 Channel-0
Audio Right Mapping: Stream-0 Channel-1

But in my case, I would be setting it to:

Audio Left Mapping: Stream-0 Channel-0
Audio Right Mapping: Stream-1 Channel-0

Then in the future when you add support for more advanced audio options, you can make something like an audio matrix-mixer.

I made a short clip that I am loading into drop box. It's only about 10 seconds long, but even that, at 720p, is about 280 meg. I'll add a link as soon as it's loaded.

(0007958)
FishB8 (reporter)
2012-04-03 02:22

OK, Dropbox upload finished. Just me pointing camera at the Ki-pro while recording music in the audio inputs.

http://dl.dropbox.com/u/38727164/DWTS2012_3.mov [^]

Issue History
Date Modified	Username	Field	Change
2012-03-27 21:09	FishB8	New Issue
2012-04-02 20:19	j-b-m	Note Added: 0007953
2012-04-02 20:19	j-b-m	Assigned To	=> j-b-m
2012-04-02 20:19	j-b-m	Status	new => feedback
2012-04-02 22:03	FishB8	Note Added: 0007955
2012-04-02 22:03	FishB8	Status	feedback => assigned
2012-04-02 22:42	j-b-m	Note Added: 0007956
2012-04-03 00:25	FishB8	Note Added: 0007957
2012-04-03 02:22	FishB8	Note Added: 0007958

Notes
(0007953) j-b-m (administrator) 2012-04-02 20:19	Well we will need some more infos about that file format. The first step is to check if FFmpeg supports it. If it does, then check if MLT can handle that and then we can make it work in Kdenlive. Can you test the FFmpeg side, and if it works, attach a small sample clip here... so we can make further tests.

(0007955) FishB8 (reporter) 2012-04-02 22:03	FFmpeg does handle it, provided it's recent enough to support ProRes. MLT also seems to handle it fine given that kdenlive imports it without any issues. It just sees the audio as a mono audio source since it ignores all but the first audio stream. (Did you have any specific test for MLC you would like done?) There can be up to 8 audio streams if the audio source is from audio embedded in the SDI signal. I will try to record a short clip for you that I can send in. All the files I have now are in the 50GB - 100GB range. I don't want to chop them up with ffmpeg since it might alter some things. Here's the output from ffprobe, which should give a pretty good idea of what is in the streams. Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'SC1ATK18.mov': Metadata: major_brand : qt minor_version : 537199360 compatible_brands: qt creation_time : 2012-03-23 06:42:45 Duration: 01:02:06.96, start: 0.000000, bitrate: 136658 kb/s Stream #0:0(eng): Audio: pcm_s24le (in24 / 0x34326E69), 48000 Hz, 1 channels, s32, 1152 kb/s Metadata: creation_time : 2012-03-23 06:42:45 handler_name : ?Linux Alias Data Handler Stream #0:1(eng): Audio: pcm_s24le (in24 / 0x34326E69), 48000 Hz, 1 channels, s32, 1152 kb/s Metadata: creation_time : 2012-03-23 06:42:45 handler_name : ?Linux Alias Data Handler Stream #0:2(eng): Video: prores (apcn / 0x6E637061), yuv422p10le, 1920x1080, 134351 kb/s, 29.97 fps, 29.97 tbr, 2997 tbn, 2997 tbc Metadata: creation_time : 2012-03-23 06:42:45 handler_name : ?Linux Alias Data Handler Stream #0:3(eng): Data: none (tmcd / 0x64636D74) Metadata: creation_time : 2012-03-23 06:42:45 handler_name : $Libquicktime Time Code Media Handler timecode : 01:14:57:07 Unsupported codec with id 0 for input stream 3 [STREAM] index=0 codec_name=pcm_s24le codec_long_name=PCM signed 24-bit little-endian codec_type=audio codec_time_base=1/48000 codec_tag_string=in24 codec_tag=0x34326e69 sample_fmt=s32 sample_rate=48000 channels=1 bits_per_sample=24 id=N/A r_frame_rate=0/0 avg_frame_rate=0/0 time_base=1/48000 start_time=0.000000 duration=3726.960292 nb_frames=178893916 TAG:creation_time=2012-03-23 06:42:45 TAG:language=eng TAG:handler_name=?Linux Alias Data Handler [/STREAM] [STREAM] index=1 codec_name=pcm_s24le codec_long_name=PCM signed 24-bit little-endian codec_type=audio codec_time_base=1/48000 codec_tag_string=in24 codec_tag=0x34326e69 sample_fmt=s32 sample_rate=48000 channels=1 bits_per_sample=24 id=N/A r_frame_rate=0/0 avg_frame_rate=0/0 time_base=1/48000 start_time=0.000000 duration=3726.960292 nb_frames=178893916 TAG:creation_time=2012-03-23 06:42:45 TAG:language=eng TAG:handler_name=?Linux Alias Data Handler [/STREAM] [STREAM] index=2 codec_name=prores codec_long_name=ProRes codec_type=video codec_time_base=1/2997 codec_tag_string=apcn codec_tag=0x6e637061 width=1920 height=1080 has_b_frames=0 sample_aspect_ratio=N/A display_aspect_ratio=N/A pix_fmt=yuv422p10le level=-99 timecode=N/A id=N/A r_frame_rate=2997/100 avg_frame_rate=2997/100 time_base=1/2997 start_time=0.000000 duration=3726.960294 nb_frames=111697 TAG:creation_time=2012-03-23 06:42:45 TAG:language=eng TAG:handler_name=?Linux Alias Data Handler [/STREAM] [STREAM] index=3 codec_name=unknown codec_long_name=unknown codec_type=data codec_time_base=1/30 codec_tag_string=tmcd codec_tag=0x64636d74 id=N/A r_frame_rate=0/0 avg_frame_rate=0/0 time_base=1/2997 start_time=0.000000 duration=3726.960294 nb_frames=1 TAG:creation_time=2012-03-23 06:42:45 TAG:language=eng TAG:handler_name=$Libquicktime Time Code Media Handler TAG:timecode=01:14:57:07 [/STREAM] [FORMAT] filename=SC1ATK18.mov nb_streams=4 format_name=mov,mp4,m4a,3gp,3g2,mj2 format_long_name=QuickTime/MPEG-4/Motion JPEG 2000 format start_time=0.000000 duration=3726.960294 size=63665127029 bit_rate=136658557 TAG:major_brand=qt TAG:minor_version=537199360 TAG:compatible_brands=qt TAG:creation_time=2012-03-23 06:42:45 [/FORMAT]

(0007956) j-b-m (administrator) 2012-04-02 22:42	From the MLT's avformat producer doc, the "audio_index" property should allow you to choose the audio stream you want. For example: melt myclip.mov audio_index=1 Should play the first audio stream. Setting audio_index to "all" should be played in the audio track: melt myclip.mov audio_index=all Is that what you want? In Kdenlive, the clip properties (in the "advanced" tab), you can force the audio index to a specific value, however there is no way to set it to all. This should be easily fixed if that is what you need.

(0007957) FishB8 (reporter) 2012-04-03 00:25	Yes, that is sort of what I need. Most of the time I need both audio channels. The audio index does let me select either of the two streams, but not both. What would be nice (but require more work) is audio channel mapping. Since kdenlive is currently limited to stereo audio, it would be nice to be able to map any audio stream->channel to the left and right. The easiest way to do that, that I can think of is to have in the advanced settings, an "Audio Left Mapping" and "Audio Right Mapping" with two number selections, first to select stream, and second to select channel in the stream. Default would probably be: Audio Left Mapping: Stream-0 Channel-0 Audio Right Mapping: Stream-0 Channel-1 But in my case, I would be setting it to: Audio Left Mapping: Stream-0 Channel-0 Audio Right Mapping: Stream-1 Channel-0 Then in the future when you add support for more advanced audio options, you can make something like an audio matrix-mixer. I made a short clip that I am loading into drop box. It's only about 10 seconds long, but even that, at 720p, is about 280 meg. I'll add a link as soon as it's loaded.

(0007958) FishB8 (reporter) 2012-04-03 02:22	OK, Dropbox upload finished. Just me pointing camera at the Ki-pro while recording music in the audio inputs. http://dl.dropbox.com/u/38727164/DWTS2012_3.mov [^]

Relationships