FLV文件中VideoPacket的详解
众所周知,FLV文件体主要有三种Tag组成:AUDIODATA、VIDEODATA和SCRIPTDATA,其中SCRIPTDATA中包含了MetaData,通过解析这些元数据,可以得到解码器的初始化信息,如音频解码元数据audiocodecid、audiodatarate、audiodelay、audiosamplerate、audiosamplesize、stereo,视频解码元数据videocodecid、framerate、height、width、videodatarate等信息。这些信息里面有的对于初始化解码器非常有用,例如解码器必须知道codecid,才能进行初始化从操作,视频解码器初始化时有的需要知道图像宽高信息,才能对解码重建Buffer进行初始化。
某些FLV文件,在SCRIPTDATA的MetaData中并未给出相应信息,这样就需要对AUDIODATA、VIDEODATA进行解析。Adobe Flash Video File Format Specification中只是简单交代了AVCVIDEOPACKET的格式。对其他的codecID对应的VideoPacket并未详细介绍。本文专门针对视频解码初始化过程中需要的width和height信息的深入解析为例,详细介绍一些VideoPacket。因为FLV的VideoPacket的格式和SWF文件离得VideoPacket格式基本一致,也可以参考SWF File Format Specification。详见http://www.adobe.com/go/swf_file_format。
1、VideoTagBody
IF FrameType == 5
UI8
ELSE (
IF CodecID == 2
H263VIDEOPACKET //Sorensen H263
IF CodecID == 3
SCREENVIDEOPACKET //Screen
IF CodecID == 4
VP6FLVVIDEOPACKET //On2 Vp6
IF CodecID == 5
VP6FLVALPHAVIDEOPACKET //On2 VP6 alpha
IF CodecID == 6
SCREENV2VIDEOPACKET //Screen V2
IF CodecID == 7
AVCVIDEOPACKET //AVC
)
2、H263VIDEOPACKET
这里的H263是Sorenson H263,对ITU-T的H.263进行了订制,所以在picture层和MB层的header的格式发生了改动,且GOB层被删除。其具体格式为:
表格 1 H263VIDEOPACKET
Field
Type
Comment
PictureStartCode
UB[17]
Similar to H.263 5.1.1
0000 0000 0000 0000 1
Version
UB[5]
Video format version
Flash Player 6 supports 0 and 1
TemporalReference
UB[8]
See H.263 5.1.2
PictureSize
UB[3]
000: custom, 1 byte
001: custom, 2 bytes
010: CIF (352x288)
011: QCIF (176x144)
100: SQCIF (128x96)
101: 320x240
110: 160x120
111: reserved
CustomWidth
If PictureSize = 000, UB[8]
If PictureSize = 001, UB[16]
Otherwise absent
Note: UB[16] is not the same
as UI16; there is no byte
swapping.
Width in pixels
CustomHeight
If PictureSize = 000, UB[8]
If PictureSize = 001, UB[16]
Otherwise absent
Note: UB[16] is not the same
as UI16; there is no byte
swapping.
Height in pixels
PictureType
UB[2]
00: intra frame
01: inter frame
10: disposable inter frame
11: reserved
DeblockingFlag
UB[1]
Requests use of deblocking
filter (advisory only, Flash
Player may ignore)
Quantizer
UB[5]
See H.263 5.1.4
ExtraInformationFlag
UB[1]
See H.263 5.1.9
ExtraInformation
If ExtraInformationFlag = 1,
UB[8]
Otherwise absent
See H.263 5.1.10
...
The ExtraInformationFlag-
ExtraInformation sequence
repeats until an
ExtraInformationFlag of 0 is
encountered
Macroblock
MACROBLOCK
See following
PictureStuffing
varies
See H.263 5.1.13
从上表可以得出,不用深入到macroblock层,只需解析头部,就可很简单的得到width和height信息。
2、SCREENVIDEOPACKET和SCREENV2VIDEOPACKET
Screen Video是一个简单的无损压缩视频格式,它利用帧间编码传递位图,其pixel_froamt是BGR24,专用于电脑截屏视频传输。像素数据使用了Zlib开放标准进行了压缩。其具体格式为:
表格 2 SCREENVIDEOPACKET
Field
Type
Comment
BlockWidth
UB[4]
Pixel width of each block in the grid. This value is stored as (actualWidth / 16) - 1, so possible block sizes are a multiple of 16 and not more than 256.
ImageWidth
UB[12]
Pixel width of the full image.
BlockHeight
UB[4]
Pixel height of each block in the grid. This value is stored
as (actualHeight / 16) - 1, so possible block sizes are a
multiple of 16 and not more than 256.
ImageHeight
UB[12]
Pixel height of the full image.
ImageBlocks
IMAGEBLOCK[n]
Blocks of image data. See preceding for details of how
to calculate n. Blocks are ordered from bottom left to
top right, in rows.
表格 3 SCREENV2VIDEOPACKET
Field
Type
Comment
BlockWidth
UB[4]
Pixel width of each block in the grid. This value is stored as (actualWidth / 16) - 1, so possible block sizes are a multiple of 16 and not more than 256.
ImageWidth
UB[12]
Pixel width of the full image.
BlockHeight
UB[4]
Pixel height of each block in the grid. This value is stored as (actualHeight / 16) - 1, so possible block sizes are a multiple of 16 and not more than 256.
ImageHeight
UB[12]
Pixel height of the full image.
Reserved
UB[6]
Must be 0
HasIFrameImage
UB[1]
If 1, has IFrameImage
HasPaletteInfo
UB[1]
If 1, has PaletteInfo
PaletteInfo
If HasPaletteInfo,
IMAGEBLOCK
One block of data to describe the palette.
ImageBlocks
IMAGEBLOCKV2[n]
Blocks of image data. See Block format for details of how to calculate n. Blocks are ordered from bottom left to top right in rows and can be a combination of keyblocks and interblocks.
IFrameImage
If HasIFrameImage,
IMAGEBLOCKV2[n]
Blocks of image data representing interblocks that must be combined with the previous keyblocks to produce the image. See Block format for details of how to calculate n. Blocks are ordered from bottom left to top right in rows.
从上表可以得出,不用深入到macroblock层,只需解析头部,就可很简单的得到width和height信息。
3、VP6FLVVIDEOPACKET和VP6FLVALPHAVIDEOPACKET
VP6FLVVIDEOPACKET和VP6FLVALPHAVIDEOPACKET比较简单,只是简单的将VP6或VP6A的一个视频帧封装到VideoPacket即可。
表格 4 VP6FLVVIDEOPACKET
Field
Type
Comment
HorizontalAdjustment
UB[4]
Number of pixels to subtract from the total width. The resulting width is used on the stage, and the rightmost pixels of the video is cropped.
VerticalAdjustment
UB[4]
Number of pixels to subtract from the total height. The resulting height is used on the stage, and the rightmost pixels of the video is cropped.
Data
UI8[n]
Raw VP6 video stream data
表格 5 VP6FLVALPHAVIDEOPACKET
Field
Type
Comment
HorizontalAdjustment
UB[4]
Number of pixels to subtract from the total width. The resulting width is used on the stage, and the rightmost pixels of the video is cropped.
VerticalAdjustment
UB[4]
Number of pixels to subtract from the total height. The resulting height is used on the stage, and the rightmost pixels of the video is cropped.
OffsetToAlpha
UI[24]
Offset in bytes to the alpha channel video data.
Data
UI8[n]
Raw VP6 video stream data.
AlphaData
UI8[n]
Raw VP6 video stream data representing the alpha channel
这样如果MetaData中没有宽高信息的话,就需要深入到Data层进行header parse,解析得到宽高信息,过程比较麻烦,不过可以参考FFmpeg的avcodec中的vp6_parse_header()函数,亦或是参考VP6或VP6Alpha的codec文档吧,本文暂不做详解。
4、AVCVIDEOPACKET
FLV将AVC的SPS(序列参数集)和PPS(图像参数集)和其他的配置信息封装到AVCDecoderConfigurationRecord,其具体格式为:
AVCDecoderConfigurationRecord {
unsigned int(8) ConfigurationVersion = 1;
unsigned int(8) AVCProfileIndication;
unsigned int(8) ProfileCompatibility;
unsigned int(8) AVCLevelIndication;
bit(6) reserved = ‘111111’b;
unsigned int(2) LengthSizeMinusOne;
bit(3) reserved = ‘111’b;
unsigned int(5) numOfSequenceParameterSets;
for (i=0; i< numOfSequenceParameterSets; i++) {
unsigned int(16) sequenceParameterSetLength ;
bit(8*sequenceParameterSetLength) sequenceParameterSetNALUnit; //SPS
}
unsigned int(8) numOfPictureParameterSets;
for (i=0; i< numOfPictureParameterSets; i++) {
unsigned int(16) pictureParameterSetLength;
bit(8*pictureParameterSetLength) pictureParameterSetNALUnit; //PPS
}
}
图像的宽高信息在SPS中,所以需要得到宽高信息,需要解析H.264的SPS,详见H.264的文档,当然也可以参考FFmpeg的avcodec中的ff_h264_decode_seq_parameter_set()函数。
5、一点点个人想法
流媒体播放中,多媒体音视频包经过网络到达终端,脱去一层层网络打包外衣,而后容器的外衣、再次是codec的header,经过艰辛的历程到达解码器。最终的目的就是节省网络带宽。随着互联网免费服务潮的到来,那些效果不错,但是收费高昂的标准或技术,遇到了很大的挑战。Google主推的VP8,较之ITU-T的H264,虽在编码效果上稍微逊色,但是其最大的杀手锏——免费,抓住了H264专利费高昂的七寸。Google强大的号召力或许真的会使VPx系列的codec最后胜出。一般公司玩技术、牛公司玩标准、超牛公司玩潮流,不得不佩服美帝的技术帝国。何种这些风潮才能Made in China?
---------------------
作者:xietao_live_cn
版权声明:本文为博主原创文章,转载请附上博文链接!