此处直接根据NDK实现来源码分析描述问题
问题描述和原理分析:
在我们直接使用MediaCodec进行解码和渲染时,一般情况下大家可以都习惯性在同一个线程中完成MediaCodec的解码和渲染,其实际我们应该拆分成两部分来处理,将解码和渲染放入不同线程完成,如此就会加快解码和渲染,其实现原理是,同一个线程中,解码和渲染将会被互相影响,而渲染是有一个Fence栅栏Buffer标记,可以简单理解为VSync屏幕刷新周期信号,若是60fps则VSync将会在16.67ms通知屏幕刷新信号一次,因此若在调用MediaCodec的渲染接口【AMediaCodec_releaseOutputBuffer】时(java层MediaCodec同理),将会在buffer处理放入Surface时被Fence栅栏wait。
因此若放在不同线程,那么将不会相互影响,即解码和渲染都将会被加快。
先写结论:
软解码时将会被阻塞渲染,而硬解码时将会是异步直接归还BUffer给Surface,但是本文要讨论的是,解码器分为了输入队列和输出队列,我们应该将输入队列和输出队列分两个线程进行解码,原因是输出队列可能会多次执行即一个输入Buffer可能会在收到多次输出Buffer才结束当前帧解码完成,另外若是硬解码器则可以将输出队列和渲染操作放入同一个线程,如此将会加快解码和渲染。
源码分析:
// [frameworks/av/media/ndk/NdkMediaCodec.cpp]
EXPORT
media_status_t AMediaCodec_releaseOutputBuffer(AMediaCodec *mData, size_t idx, bool render) {
if (render) {
// 请求渲染
return translate_error(mData->mCodec->renderOutputBufferAndRelease(idx));
} else {
return translate_error(mData->mCodec->releaseOutputBuffer(idx));
}
}
renderOutputBufferAndRelease实现:
注意该方法调用是同步调用,即会被阻塞的调用
// [frameworks/av/media/libstagefright/MediaCodec.cpp]
status_t MediaCodec::renderOutputBufferAndRelease(size_t index) {
sp<AMessage> msg = new AMessage(kWhatReleaseOutputBuffer, this);
msg->setSize("index", index);
msg->setInt32("render", true);
sp<AMessage> response;
return PostAndAwaitResponse(msg, &response);
}
接收渲染事件处理
// [frameworks/av/media/libstagefright/MediaCodec.cpp]
void MediaCodec::onMessageReceived(const sp<AMessage> &msg) {
switch (msg->what()) {
case kWhatReleaseOutputBuffer:
{
sp<AReplyToken> replyID;
CHECK(msg->senderAwaitsResponse(&replyID));
if (!isExecuting()) {
PostReplyWithError(replyID, INVALID_OPERATION);
break;
} else if (mFlags & kFlagStickyError) {
PostReplyWithError(replyID, getStickyError());
break;
}
// 处理渲染
status_t err = onReleaseOutputBuffer(msg);
PostReplyWithError(replyID, err);
break;
}
}
}
onReleaseOutputBuffer实现:
// [frameworks/av/media/libstagefright/MediaCodec.cpp]
status_t MediaCodec::onReleaseOutputBuffer(const sp<AMessage> &msg) {
size_t index;
CHECK(msg->findSize("index", &index));
int32_t render;
if (!msg->findInt32("render", &render)) {
render = 0;
}
if (!isExecuting()) {
return -EINVAL;
}
if (index >= mPortBuffers[kPortIndexOutput].size()) {
return -ERANGE;
}
// 获取当前将被渲染的buffer
BufferInfo *info = &mPortBuffers[kPortIndexOutput][index];
if (info->mData == nullptr || !info->mOwnedByClient) {
return -EACCES;
}
// 加锁获取buffer,但这里我们可以认为是不耗时的,非常快
// synchronization boundary for getBufferAndFormat
sp<MediaCodecBuffer> buffer;
{
Mutex::Autolock al(mBufferLock);
info->mOwnedByClient = false;
buffer = info->mData;
info->mData.clear();
}
if (render && buffer->size() != 0) {
// 渲染流程
int64_t mediaTimeUs = -1;
buffer->meta()->findInt64("timeUs", &mediaTimeUs);
int64_t renderTimeNs = 0;
if (!msg->findInt64("timestampNs", &renderTimeNs)) {
// use media timestamp if client did not request a specific render timestamp
ALOGV("using buffer PTS of %lld", (long long)mediaTimeUs);
renderTimeNs = mediaTimeUs * 1000;
}
if (mSoftRenderer != NULL) {
// 软解码时,直接在此处处理了渲染流程
std::list<FrameRenderTracker::Info> doneFrames = mSoftRenderer->render(
buffer->data(), buffer->size(), mediaTimeUs, renderTimeNs,
mPortBuffers[kPortIndexOutput].size(), buffer->format());
// if we are running, notify rendered frames
if (!doneFrames.empty() && mState == STARTED && mOnFrameRenderedNotification != NULL) {
sp<AMessage> notify = mOnFrameRenderedNotification->dup();
sp<AMessage> data = new AMessage;
if (CreateFramesRenderedMessage(doneFrames, data)) {
notify->setMessage("data", data);
notify->post();
}
}
}
// 该流程将不展开分析,在我之前的MediaCodec系列分析文章中已分析过,它不是阻塞调用,是AMessage异步执行的。
// 最终硬解码时将会是异步直接归还BUffer给Surface。
// 但是本文要讨论的是,解码器分为了输入队列和输出队列,我们应该将输入队列和输出队列分两个线程进行解码,
// 另外若是硬解码器则可以将输出队列和渲染操作放入同一个线程,如此将会加快解码和渲染。
mBufferChannel->renderOutputBuffer(buffer, renderTimeNs);
} else {
mBufferChannel->discardBuffer(buffer);
}
return OK;
}
mSoftRenderer->render实现分析:
软解码时,直接在此处处理了渲染流程
// [frameworks/av/media/libstagefright/colorconversion/SoftwareRenderer.cpp]
std::list<FrameRenderTracker::Info> SoftwareRenderer::render(
const void *data, size_t , int64_t mediaTimeUs, nsecs_t renderTimeNs,
size_t numOutputBuffers, const sp<AMessage>& format) {
resetFormatIfChanged(format, numOutputBuffers);
FrameRenderTracker::Info *info = NULL;
ANativeWindowBuffer *buf;
int fenceFd = -1;
// 获取一个Surface的buffer及其buffer的Fence文件描述符
int err = mNativeWindow->dequeueBuffer(mNativeWindow.get(), &buf, &fenceFd);
if (err == 0 && fenceFd >= 0) {
// 获取成功
info = mRenderTracker.updateInfoForDequeuedBuffer(buf, fenceFd, 0);
sp<Fence> fence = new Fence(fenceFd);
// 注意此处处理:将会无限wait
// 见下面的分析
err = fence->waitForever("SoftwareRenderer::render");
}
// 省略其它无关代码
// 成功归还待渲染buffer的数据给Surface去渲染处理
if ((err = mNativeWindow->queueBuffer(mNativeWindow.get(), buf, -1)) != 0) {
ALOGW("Surface::queueBuffer returned error %d", err);
} else {
mRenderTracker.onFrameQueued(mediaTimeUs, (GraphicBuffer *)buf, Fence::NO_FENCE);
}
buf = NULL;
return mRenderTracker.checkFencesAndGetRenderedFrames(info, info != NULL /* dropIncomplete */);
}
fence->waitForever(“SoftwareRenderer::render”)实现分析:
此处理将会可能无限等待
// [frameworks/native/libs/ui/Fence.cpp]
status_t Fence::waitForever(const char* logname) {
ATRACE_CALL();
if (mFenceFd == -1) {
return NO_ERROR;
}
int warningTimeout = 3000;
// 默认先wait等待3秒的VSync信号
int err = sync_wait(mFenceFd, warningTimeout);
if (err < 0 && errno == ETIME) {
ALOGE("%s: fence %d didn't signal in %u ms --Initializing dump",
logname, mFenceFd.get(), warningTimeout);
dump(mFenceFd);
// 失败则将会无限等待
err = sync_wait(mFenceFd, TIMEOUT_NEVER);
}
return err < 0 ? -errno : status_t(NO_ERROR);
}
其实上面的分析过程,在我此前的android MediaCodec源码实现分析系列文章中已有分析过。
本文结束