什么是音视频的分离和合成
分离就是将视频1的声音和图像分别取出来
合成就是将视频1的图像和非视频1的声音组合成一个新的视频
如何进行音视频的分离和合成
安卓提供了两个API来帮助我们完成这个操作
MediaExtractor用于分离视频
MediaMuxer用于合成视频
下面我就来介绍一下这两个API的使用
MediaExtractor
分离音频
1.设置音频源
MediaExtractor audioExtractor = new MediaExtractor();
audioExtractor.setDataSource(audioPath);
MediaExtractor audioExtractor = new MediaExtractor();
audioExtractor.setDataSource(audioPath);2.获取源文件中轨道的数量，并遍历找到我们需要的音频轨
for (int i = 0; i < audioExtractor.getTrackCount(); i++) {
MediaFormat format = audioExtractor.getTrackFormat(i);
if (format.getString(MediaFormat.KEY_MIME).startsWith("audio/")) { srcATrackIndex = i; audioTrackIndex = muxer.addTrack(format); break; } }
for (int i = 0; i < audioExtractor.getTrackCount(); i++) {
MediaFormat format = audioExtractor.getTrackFormat(i);
if (format.getString(MediaFormat.KEY_MIME).startsWith("audio/")) { srcATrackIndex = i; audioTrackIndex = muxer.addTrack(format); break; } }分离视频
1.设置视频源
MediaExtractor videoExtractor = new MediaExtractor();
videoExtractor.setDataSource(videoPath);
MediaExtractor videoExtractor = new MediaExtractor();
videoExtractor.setDataSource(videoPath);2.获取视频源文件中的轨道数，并遍历找到我们所需要的视频轨
for (int i = 0; i < videoExtractor.getTrackCount(); i++) {
MediaFormat format = videoExtractor.getTrackFormat(i);
if (format.getString(MediaFormat.KEY_MIME).startsWith("video/")) { srcVTrackIndex = i; videoTrackIndex = muxer.addTrack(format); break; } }
for (int i = 0; i < videoExtractor.getTrackCount(); i++) {
MediaFormat format = videoExtractor.getTrackFormat(i);
if (format.getString(MediaFormat.KEY_MIME).startsWith("video/")) { srcVTrackIndex = i; videoTrackIndex = muxer.addTrack(format); break; } }音频和视频额的分离方法一模一样，区别在于MediaFormat类型的不同，MediaFormat封装了媒体的数据格式信息
MediaMuxer
如何合成视频？
1.设置合成后视频的路径和格式
MediaMuxer muxer = new MediaMuxer(outPath, MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4);
MediaMuxer muxer = new MediaMuxer(outPath, MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4);2.将MediaExtractor分离出来的音轨和视轨添加到自己的轨道中
audioTrackIndex = muxer.addTrack(format);
videoTrackIndex = muxer.addTrack(format);3.添加完所有轨道后start
muxer.start();
muxer.start();4.采集音频源的音轨样本
audioExtractor.selectTrack(srcATrackIndex); //移动到音频轨上
if (audioTrackIndex != -1) {
MediaCodec.BufferInfo info = new MediaCodec.BufferInfo(); info.presentationTimeUs = 0; ByteBuffer buffer = ByteBuffer.allocate(100 * 1024); while (true) { int sampleSize = audioExtractor.readSampleData(buffer, 0); if (sampleSize < 0) { //没有可获取的样本，退出循环 break; } info.offset = 0; info.size = sampleSize; info.flags = MediaCodec.BUFFER_FLAG_SYNC_FRAME; info.presentationTimeUs = audioExtractor.getSampleTime(); muxer.writeSampleData(audioTrackIndex, buffer, info);//将样本写入新的轨道 audioExtractor.advance(); //进入下一个样本 } }
audioExtractor.selectTrack(srcATrackIndex); //移动到音频轨上
if (audioTrackIndex != -1) {
MediaCodec.BufferInfo info = new MediaCodec.BufferInfo(); info.presentationTimeUs = 0; ByteBuffer buffer = ByteBuffer.allocate(100 * 1024); while (true) { int sampleSize = audioExtractor.readSampleData(buffer, 0); if (sampleSize < 0) { //没有可获取的样本，退出循环 break; } info.offset = 0; info.size = sampleSize; info.flags = MediaCodec.BUFFER_FLAG_SYNC_FRAME; info.presentationTimeUs = audioExtractor.getSampleTime(); muxer.writeSampleData(audioTrackIndex, buffer, info);//将样本写入新的轨道 audioExtractor.advance(); //进入下一个样本 } }5.采集视频源的视频轨样本
videoExtractor.selectTrack(srcVTrackIndex); //移动到视频轨上
if (videoTrackIndex != -1) {
MediaCodec.BufferInfo info = new MediaCodec.BufferInfo(); info.presentationTimeUs = 0; ByteBuffer buffer = ByteBuffer.allocate(100 * 1024); while (true) { int sampleSize = videoExtractor.readSampleData(buffer, 0); if (sampleSize < 0) { //没有可获取的样本，退出循环 break; } info.offset = 0; info.size = sampleSize; info.flags = MediaCodec.BUFFER_FLAG_SYNC_FRAME; info.presentationTimeUs = videoExtractor.getSampleTime(); muxer.writeSampleData(videoTrackIndex, buffer, info);//将样本写入新的轨道 videoExtractor.advance(); //进入下一个样本 } }
videoExtractor.selectTrack(srcVTrackIndex); //移动到视频轨上
if (videoTrackIndex != -1) {
MediaCodec.BufferInfo info = new MediaCodec.BufferInfo(); info.presentationTimeUs = 0; ByteBuffer buffer = ByteBuffer.allocate(100 * 1024); while (true) { int sampleSize = videoExtractor.readSampleData(buffer, 0); if (sampleSize < 0) { //没有可获取的样本，退出循环 break; } info.offset = 0; info.size = sampleSize; info.flags = MediaCodec.BUFFER_FLAG_SYNC_FRAME; info.presentationTimeUs = videoExtractor.getSampleTime(); muxer.writeSampleData(videoTrackIndex, buffer, info);//将样本写入新的轨道 videoExtractor.advance(); //进入下一个样本 } }6.停止并释放资源
muxer.stop();
muxer.release();
muxer.stop();
muxer.release();完整代码
public class MediaUtil { private static final String TAG = "MediaUtil"; /** * @param audioPath 音频文件路劲 * @param videoPath 视频文件路径 * @param outPath 合成之后的保存路径 */ public static void combineVideo(String audioPath, String videoPath, String outPath) { MediaMuxer muxer = null; MediaExtractor audioExtractor = null; MediaExtractor videoExtractor = null; try { muxer = new MediaMuxer(outPath, MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4); //找到音频文件的音频轨 audioExtractor = new MediaExtractor(); audioExtractor.setDataSource(audioPath); int srcATrackIndex = -1; //音频源的音频轨 int audioTrackIndex = -1; //音频轨添加到muxer后返回的新的轨道 //在此循环，目的是找到我们需要的音频轨 for (int i = 0; i < audioExtractor.getTrackCount(); i++) { MediaFormat format = audioExtractor.getTrackFormat(i); if (format.getString(MediaFormat.KEY_MIME).startsWith("audio/")) { srcATrackIndex = i; audioTrackIndex = muxer.addTrack(format); break; } } Log.d(TAG,"音频轨源索引："+srcATrackIndex+" 音频轨新索引:"+audioTrackIndex); //找到视频文件的视频轨 videoExtractor = new MediaExtractor(); videoExtractor.setDataSource(videoPath); int srcVTrackIndex = -1; //视频源的视频轨 int videoTrackIndex = -1; //视频轨添加到muxer后返回的新的轨道 for (int i = 0; i < videoExtractor.getTrackCount(); i++) { MediaFormat format = videoExtractor.getTrackFormat(i); if (format.getString(MediaFormat.KEY_MIME).startsWith("video/")) { srcVTrackIndex = i; videoTrackIndex = muxer.addTrack(format); break; } } Log.d(TAG,"视频轨源索引："+srcVTrackIndex+" 视频频轨新索引:"+videoTrackIndex); //添加完所有轨道后start muxer.start(); Log.d(TAG,"开始合成视频..."); //封装音频track audioExtractor.selectTrack(srcATrackIndex); //移动到音频轨上 if (audioTrackIndex != -1) { MediaCodec.BufferInfo info = new MediaCodec.BufferInfo(); info.presentationTimeUs = 0; ByteBuffer buffer = ByteBuffer.allocate(100 * 1024); while (true) { int sampleSize = audioExtractor.readSampleData(buffer, 0); if (sampleSize < 0) { //没有可获取的样本，退出循环 break; } info.offset = 0; info.size = sampleSize; info.flags = MediaCodec.BUFFER_FLAG_SYNC_FRAME; info.presentationTimeUs = audioExtractor.getSampleTime(); muxer.writeSampleData(audioTrackIndex, buffer, info);//将样本写入新的轨道 audioExtractor.advance(); //进入下一个样本 } } Log.d(TAG,"音频轨样本采集完成"); //封装视频track videoExtractor.selectTrack(srcVTrackIndex); //移动到视频轨上 if (videoTrackIndex != -1) { MediaCodec.BufferInfo info = new MediaCodec.BufferInfo(); info.presentationTimeUs = 0; ByteBuffer buffer = ByteBuffer.allocate(100 * 1024); while (true) { int sampleSize = videoExtractor.readSampleData(buffer, 0); if (sampleSize < 0) { //没有可获取的样本，退出循环 break; } info.offset = 0; info.size = sampleSize; info.flags = MediaCodec.BUFFER_FLAG_SYNC_FRAME; info.presentationTimeUs = videoExtractor.getSampleTime(); muxer.writeSampleData(videoTrackIndex, buffer, info);//将样本写入新的轨道 videoExtractor.advance(); //进入下一个样本 } } muxer.stop(); Log.d(TAG,"视频轨样本采集完成"); Log.d(TAG,"视频合成完毕："+outPath); } catch (IOException e) { e.printStackTrace(); Log.d(TAG,"合成出错:"+e.getMessage()); } finally { //释放资源 if(audioExtractor!=null){ audioExtractor.release(); } if(videoExtractor!=null){ videoExtractor.release(); } if(muxer!=null){ muxer.release(); } } } }
public class MediaUtil { private static final String TAG = "MediaUtil"; /** * @param audioPath 音频文件路劲 * @param videoPath 视频文件路径 * @param outPath 合成之后的保存路径 */ public static void combineVideo(String audioPath, String videoPath, String outPath) { MediaMuxer muxer = null; MediaExtractor audioExtractor = null; MediaExtractor videoExtractor = null; try { muxer = new MediaMuxer(outPath, MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4); //找到音频文件的音频轨 audioExtractor = new MediaExtractor(); audioExtractor.setDataSource(audioPath); int srcATrackIndex = -1; //音频源的音频轨 int audioTrackIndex = -1; //音频轨添加到muxer后返回的新的轨道 //在此循环，目的是找到我们需要的音频轨 for (int i = 0; i < audioExtractor.getTrackCount(); i++) { MediaFormat format = audioExtractor.getTrackFormat(i); if (format.getString(MediaFormat.KEY_MIME).startsWith("audio/")) { srcATrackIndex = i; audioTrackIndex = muxer.addTrack(format); break; } } Log.d(TAG,"音频轨源索引："+srcATrackIndex+" 音频轨新索引:"+audioTrackIndex); //找到视频文件的视频轨 videoExtractor = new MediaExtractor(); videoExtractor.setDataSource(videoPath); int srcVTrackIndex = -1; //视频源的视频轨 int videoTrackIndex = -1; //视频轨添加到muxer后返回的新的轨道 for (int i = 0; i < videoExtractor.getTrackCount(); i++) { MediaFormat format = videoExtractor.getTrackFormat(i); if (format.getString(MediaFormat.KEY_MIME).startsWith("video/")) { srcVTrackIndex = i; videoTrackIndex = muxer.addTrack(format); break; } } Log.d(TAG,"视频轨源索引："+srcVTrackIndex+" 视频频轨新索引:"+videoTrackIndex); //添加完所有轨道后start muxer.start(); Log.d(TAG,"开始合成视频..."); //封装音频track audioExtractor.selectTrack(srcATrackIndex); //移动到音频轨上 if (audioTrackIndex != -1) { MediaCodec.BufferInfo info = new MediaCodec.BufferInfo(); info.presentationTimeUs = 0; ByteBuffer buffer = ByteBuffer.allocate(100 * 1024); while (true) { int sampleSize = audioExtractor.readSampleData(buffer, 0); if (sampleSize < 0) { //没有可获取的样本，退出循环 break; } info.offset = 0; info.size = sampleSize; info.flags = MediaCodec.BUFFER_FLAG_SYNC_FRAME; info.presentationTimeUs = audioExtractor.getSampleTime(); muxer.writeSampleData(audioTrackIndex, buffer, info);//将样本写入新的轨道 audioExtractor.advance(); //进入下一个样本 } } Log.d(TAG,"音频轨样本采集完成"); //封装视频track videoExtractor.selectTrack(srcVTrackIndex); //移动到视频轨上 if (videoTrackIndex != -1) { MediaCodec.BufferInfo info = new MediaCodec.BufferInfo(); info.presentationTimeUs = 0; ByteBuffer buffer = ByteBuffer.allocate(100 * 1024); while (true) { int sampleSize = videoExtractor.readSampleData(buffer, 0); if (sampleSize < 0) { //没有可获取的样本，退出循环 break; } info.offset = 0; info.size = sampleSize; info.flags = MediaCodec.BUFFER_FLAG_SYNC_FRAME; info.presentationTimeUs = videoExtractor.getSampleTime(); muxer.writeSampleData(videoTrackIndex, buffer, info);//将样本写入新的轨道 videoExtractor.advance(); //进入下一个样本 } } muxer.stop(); Log.d(TAG,"视频轨样本采集完成"); Log.d(TAG,"视频合成完毕："+outPath); } catch (IOException e) { e.printStackTrace(); Log.d(TAG,"合成出错:"+e.getMessage()); } finally { //释放资源 if(audioExtractor!=null){ audioExtractor.release(); } if(videoExtractor!=null){ videoExtractor.release(); } if(muxer!=null){ muxer.release(); } } } }踩坑总结
问题1
MPEG4Writer: Unsupported mime 'audio/mpeg'
当音频文件是mp3时会报这个错误 ,需要的格式是AAC，m4a
stackoverflow 我将两个源文件都替换了mp4格式后不再报错了
问题2
WVMExtractor: Failed to open libwvm.so: dlopen failed: library "libwvm.so" not found
小米5 6.0.1机型上出现的问题，意思是缺少so库
由于我就是这款手机，也没测过其他手机有木有这个so库
后续我会用其他手机在测试这个问题
MediaUtil类的封装放置在libplayer下的util包下
方法调用放置在app/demo/media/track下
本文章为转载内容，我们尊重原作者对文章享有的著作权。如有内容错误或侵权问题，欢迎原作者联系我们进行内容更正或删除文章。
