我正在尝试通过两个 mp4 视频创建画中画视频。
- 第一个输入文件 (0:v) 有一个时间偏移。
- 第二个输入（1：v）文件正在旋转 90 度。
- 我正在使用 apad 来扩展第一个音频 (0:a)。
- 我正在使用 amerge 将两个音频输入合并为一个音频。
错误
我在两台不同的机器上使用相同的命令时遇到了类似的错误。一台机器有 GPU（图形处理单元），另一台没有。（我正在尝试开始使用硬件加速，因此新机器配有 GPU。）它们的配置不同，如果您需要这些信息，请告诉我。
无 GPU 的机器
ffmpeg version 3.0.git Copyright (c) 2000-2016 the FFmpeg developers
Error while decoding stream #1:1: Cannot allocate memory
带有 GPU 的机器
ffmpeg version N-90913-gd176497 Copyright (c) 2000-2018 the FFmpeg developer
Error while filtering: Cannot allocate memory
Failed to inject frame into filter network: Cannot allocate memory
Error while processing the decoded data for stream #1:1
即使出现错误，没有 GPU 的机器也能完成视频转换。带有 GPU 的机器则完全失败，并显示“转换失败！”消息。
命令：
ffmpeg -itsoffset 1.801 -i 2327_segment_0_remote_0.mp4 \
-i 2327_segment_0_local_0.mp4 -filter_complex \
" [1:v]scale=iw/4:-1:flags=lanczos[loc0]; \
[0:v]transpose=1[rotate1]; \
[rotate1][loc0]overlay=main_w-overlay_w-10:main_h-overlay_h-10:eof_action=pass[rem0]; \
[0:a]apad[0a]; [0a][1:a]amerge=inputs=2[a]" \
-map "[rem0]" -map "[a]" -ac 2 -vcodec libx264 -ar 44100 -acodec aac \
2327_segment_0.mp4
ffmpeg version 3.0.git Copyright (c) 2000-2016 the FFmpeg developers
built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.2) 20160609
configuration: --prefix=/home/daryl/ffmpeg_build --pkg-config-flags=--static --extra-cflags=-I/home/daryl/ffmpeg_build/include --extra-ldflags=-L/home/daryl/ffmpeg_build/lib --bindir=/home/daryl/bin --enable-gpl --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-nonfree
libavutil 55. 32.100 / 55. 32.100
libavcodec 57. 63.103 / 57. 63.103
libavformat 57. 52.100 / 57. 52.100
libavdevice 57. 0.102 / 57. 0.102
libavfilter 6. 64.100 / 6. 64.100
libswscale 4. 1.100 / 4. 1.100
libswresample 2. 2.100 / 2. 2.100
libpostproc 54. 0.100 / 54. 0.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '2327_segment_0_remote_0.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf57.52.100
Duration: 00:00:09.12, start: 0.000000, bitrate: 762 kb/s
Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 640x480 [SAR 1:1 DAR 4:3], 688 kb/s, 16.67 fps, 16.67 tbr, 12800 tbn, 33.33 tbc (default)
Metadata:
handler_name : VideoHandler
Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, mono, fltp, 69 kb/s (default)
Metadata:
handler_name : SoundHandler
Input #1, mov,mp4,m4a,3gp,3g2,mj2, from '2327_segment_0_local_0.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf57.52.100
Duration: 00:00:12.24, start: 0.000000, bitrate: 398 kb/s
Stream #1:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 640x480 [SAR 1:1 DAR 4:3], 324 kb/s, 16.67 fps, 16.67 tbr, 12800 tbn, 33.33 tbc (default)
Metadata:
handler_name : VideoHandler
Stream #1:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, mono, fltp, 69 kb/s (default)
Metadata:
handler_name : SoundHandler
[Parsed_amerge_4 @ 0x283a300] No channel layout for input 1
[Parsed_amerge_4 @ 0x283a300] Input channel layouts overlap: output layout will be determined by the number of distinct input channels
[libx264 @ 0x2849b20] using SAR=1/1
[libx264 @ 0x2849b20] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2
[libx264 @ 0x2849b20] profile High, level 2.2
[libx264 @ 0x2849b20] 264 - core 148 r2643 5c65704 - H.264/MPEG-4 AVC codec - Copyleft 2003-2015 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=1 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=16 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to '2327_segment_0.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf57.52.100
Stream #0:0: Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuv420p, 480x640 [SAR 1:1 DAR 3:4], q=-1--1, 16.67 fps, 12800 tbn, 16.67 tbc (default)
Metadata:
encoder : Lavc57.63.103 libx264
Side data:
cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
Stream #0:1: Audio: aac (LC) ([64][0][0][0] / 0x0040), 44100 Hz, stereo, fltp, 128 kb/s (default)
Metadata:
encoder : Lavc57.63.103 aac
Stream mapping:
Stream #0:0 (h264) -> transpose
Stream #0:1 (aac) -> apad
Stream #1:0 (h264) -> scale
Stream #1:1 (aac) -> amerge:in1
overlay -> Stream #0:0 (libx264)
amerge -> Stream #0:1 (aac)
Press [q] to stop, [?] for help
Error while decoding stream #1:1: Cannot allocate memory
Last message repeated 17 times
frame= 182 fps= 29 q=27.0 Lsize= 975kB time=00:00:13.55 bitrate= 589.4kbits/s dup=30 drop=0 speed=2.13x
video:786kB audio:182kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.772906%
[libx264 @ 0x2849b20] frame I:1 Avg QP:15.27 size: 22292
[libx264 @ 0x2849b20] frame P:127 Avg QP:21.91 size: 5290
[libx264 @ 0x2849b20] frame B:54 Avg QP:22.79 size: 2036
[libx264 @ 0x2849b20] consecutive B-frames: 52.2% 20.9% 11.5% 15.4%
[libx264 @ 0x2849b20] mb I I16..4: 6.0% 58.8% 35.2%
[libx264 @ 0x2849b20] mb P I16..4: 2.6% 9.1% 2.0% P16..4: 38.6% 11.5% 2.9% 0.0% 0.0% skip:33.1%
[libx264 @ 0x2849b20] mb B I16..4: 0.3% 0.9% 0.5% B16..8: 30.3% 6.1% 0.7% direct: 0.8% skip:60.4% L0:51.5% L1:44.3% BI: 4.2%
[libx264 @ 0x2849b20] 8x8 transform intra:65.2% inter:69.2%
[libx264 @ 0x2849b20] coded y,uvDC,uvAC intra: 60.3% 63.2% 6.7% inter: 14.4% 10.5% 0.2%
[libx264 @ 0x2849b20] i16 v,h,dc,p: 21% 23% 18% 37%
[libx264 @ 0x2849b20] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 24% 22% 19% 5% 5% 7% 5% 6% 7%
[libx264 @ 0x2849b20] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 30% 19% 17% 5% 5% 7% 4% 6% 4%
[libx264 @ 0x2849b20] i8c dc,h,v,p: 52% 17% 26% 5%
[libx264 @ 0x2849b20] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0x2849b20] ref P L0: 76.5% 11.4% 8.1% 3.9%
[libx264 @ 0x2849b20] ref B L0: 89.9% 9.6% 0.5%
[libx264 @ 0x2849b20] ref B L1: 96.4% 3.6%
[libx264 @ 0x2849b20] kb/s:589.06
[aac @ 0x2838940] Qavg: 1670.505
ffmpeg -itsoffset 1.801 -i 2327_segment_0_remote_0.mp4 \
-i 2327_segment_0_local_0.mp4 -filter_complex \
" [1:v]scale=iw/4:-1:flags=lanczos[loc0]; \
[0:v]transpose=1[rotate1]; \
[rotate1][loc0]overlay=main_w-overlay_w-10:main_h-overlay_h-10:eof_action=pass[rem0]; \
[0:a]apad[0a]; [0a][1:a]amerge=inputs=2[a]" \
-map "[rem0]" -map "[a]" -ac 2 -vcodec libx264 -ar 44100 -acodec aac \
2327_segment_0.mp4
ffmpeg version N-90913-gd176497 Copyright (c) 2000-2018 the FFmpeg developers
built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.9) 20160609
configuration: --as=yasm --bindir=/home/dreedy/bin --cpu=native --extra-cflags=-I/home/dreedy/ffmpeg_build/include --extra-ldflags=-L/home/dreedy/ffmpeg_build/lib --extra-libs=-lpthread --pkg-config-flags=--static --prefix=/home/dreedy/ffmpeg_build --enable-cuda --enable-gpl --enable-ladspa --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libgsm --enable-libmp3lame --enable-libopus --enable-libsmbclient --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libx264 --enable-libx265 --enable-nonfree --enable-nvenc --enable-opengl --enable-pic --enable-static --enable-vaapi --enable-vdpau --enable-version3 --enable-libxvid --enable-omx --enable-openal --enable-openssl
libavutil 56. 18.100 / 56. 18.100
libavcodec 58. 19.100 / 58. 19.100
libavformat 58. 13.100 / 58. 13.100
libavdevice 58. 4.100 / 58. 4.100
libavfilter 7. 21.100 / 7. 21.100
libswscale 5. 2.100 / 5. 2.100
libswresample 3. 2.100 / 3. 2.100
libpostproc 55. 2.100 / 55. 2.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '2327_segment_0_remote_0.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf57.52.100
Duration: 00:00:09.12, start: 0.000000, bitrate: 762 kb/s
Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 640x480 [SAR 1:1 DAR 4:3], 688 kb/s, 16.67 fps, 16.67 tbr, 12800 tbn, 33.33 tbc (default)
Metadata:
handler_name : VideoHandler
Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, mono, fltp, 69 kb/s (default)
Metadata:
handler_name : SoundHandler
Input #1, mov,mp4,m4a,3gp,3g2,mj2, from '2327_segment_0_local_0.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf57.52.100
Duration: 00:00:12.24, start: 0.000000, bitrate: 398 kb/s
Stream #1:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 640x480 [SAR 1:1 DAR 4:3], 324 kb/s, 16.67 fps, 16.67 tbr, 12800 tbn, 33.33 tbc (default)
Metadata:
handler_name : VideoHandler
Stream #1:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, mono, fltp, 69 kb/s (default)
Metadata:
handler_name : SoundHandler
Stream mapping:
Stream #0:0 (h264) -> transpose
Stream #0:1 (aac) -> apad
Stream #1:0 (h264) -> scale
Stream #1:1 (aac) -> amerge:in1
overlay -> Stream #0:0 (libx264)
amerge -> Stream #0:1 (aac)
Press [q] to stop, [?] for help
[Parsed_amerge_4 @ 0x31e9c40] No channel layout for input 1
[Parsed_amerge_4 @ 0x31e9c40] Input channel layouts overlap: output layout will be determined by the number of distinct input channels
[libx264 @ 0x2f2dc80] using SAR=1/1
[libx264 @ 0x2f2dc80] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 AVX2 LZCNT BMI2
[libx264 @ 0x2f2dc80] profile High, level 2.2
[libx264 @ 0x2f2dc80] 264 - core 148 r2643 5c65704 - H.264/MPEG-4 AVC codec - Copyleft 2003-2015 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=16 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to '2327_segment_0.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf58.13.100
Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv420p, 480x640 [SAR 1:1 DAR 3:4], q=-1--1, 16.67 fps, 12800 tbn, 16.67 tbc (default)
Metadata:
encoder : Lavc58.19.100 libx264
Side data:
cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default)
Metadata:
encoder : Lavc58.19.100 aac
Error while filtering: Cannot allocate memory
Failed to inject frame into filter network: Cannot allocate memory
Error while processing the decoded data for stream #1:1
[aac @ 0x2f2eb80] Qavg: 136.254
[aac @ 0x2f2eb80] 2 frames left in the queue on closing
线程
一些谷歌搜索建议指定线程，但这没有帮助：
-threads 1
-filter_threads 1
更新
我发现，当使用另一个视频作为第二个输入文件时，此命令运行良好。以下是存在转换问题的视频的 ffprobe 信息。我最初使用 RecordRTC 从 WebRTC 流中保存的两个 .webm 文件开始。在将这些文件转换为图片叠加视频中的图片之前，我先将这些文件转换为 .mp4 作为中间步骤。
Input #0, matroska,webm, from '2327_segment_0_local_0.webm':
Metadata:
encoder : Chrome
Duration: N/A, start: 0.000000, bitrate: N/A
Stream #0:0(eng): Audio: opus, 48000 Hz, mono, fltp (default)
Stream #0:1(eng): Video: h264 (Baseline), yuv420p, 640x480 [SAR 1:1 DAR 4:3], 16.67 tbr, 1k tbn, 2k tbc (default)
Input #0, matroska,webm, from '2327_segment_0_remote_0.webm':
Metadata:
encoder : Chrome
Duration: N/A, start: 0.000000, bitrate: N/A
Stream #0:0(eng): Audio: opus, 48000 Hz, mono, fltp (default)
Stream #0:1(eng): Video: h264 (Baseline), yuv420p, 640x480 [SAR 1:1 DAR 4:3], 16.67 tbr, 1k tbn, 2k tbc (default)
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '2327_segment_0_local_0.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf57.52.100
Duration: 00:00:12.24, start: 0.000000, bitrate: 398 kb/s
Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 640x480 [SAR 1:1 DAR 4:3], 324 kb/s, 16.67 fps, 16.67 tbr, 12800 tbn, 33.33 tbc (default)
Metadata:
handler_name : VideoHandler
Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, mono, fltp, 69 kb/s (default)
Metadata:
handler_name : SoundHandler
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '2327_segment_0_remote_0.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf57.52.100
Duration: 00:00:09.12, start: 0.000000, bitrate: 762 kb/s
Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 640x480 [SAR 1:1 DAR 4:3], 688 kb/s, 16.67 fps, 16.67 tbr, 12800 tbn, 33.33 tbc (default)
Metadata:
handler_name : VideoHandler
Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, mono, fltp, 69 kb/s (default)
Metadata:
handler_name : SoundHandler
经过大量的研究，我决定将我的例子简化到最低限度。我发现如果我删除偏移量，那么视频转换。但是，背景视频开始得太早，不同步。我做了一些测试，发现它会以不超过 1.200 秒的偏移量进行转换。
背景视频为 9.12 秒。叠加视频为 12.24 秒。背景视频应显示第一帧，直到 1.801 秒过去。背景视频应在 10.921 秒处结束。叠加视频应播放完整的画中画视频的全部 12.24 秒。
答案1
将 .webm 文件转换为 .mp4 时，您正在将音频从 opus 转换为 aac。看来，当您尝试在图片叠加层中创建图片时，转换在解码 aac 流时遇到了问题。
我建议尝试使用不同的音频编解码器将 .webm 文件转换为 .mp4。然后再次尝试覆盖命令。
例子：
ffmpeg -i 2327_segment_0_local_0.webm -c:a ac3_fixed 2327_segment_0_local_0.mp4
ffmpeg -i 2327_segment_0_remote_0.webm -c:a ac3_fixed 2327_segment_0_remote_0.mp4