FFmpeg RTSP交互实现以及问题解决

原创

fengyuzaitu 2017-07-29 18:07:37 博主文章分类：视音频 ©著作权

©著作权归作者所有：来自51CTO博客作者fengyuzaitu的原创作品，请联系作者获取转载授权，否则将追究法律责任

知识准备

RFC文档地址：http://www.networksorcery.com/enp/rfc/rfc3550.txt

RTSP标准文档规范https://tools.ietf.org/html/rfc7826#page-94

默认情况下，wireshark并没有分析数据包的内容，从而判断是否是rtsp数据包，它是根据端口，默认端口是554，认为是进行rtsp协议会话，所以会在捕获界面显示数据包的Protocol协议，如果知道哪些端口也是进行rtsp会话的情况下，可以在菜单栏中选择分析，点击编码为，在字段中选择tcp port 值填写指定的端口，然后在当前的协议中，选择RTSP。另外，可以通过鼠标右键选择追踪流，点击其中的TCP，查看RTSP的交互过程

OPTIONS

Public: OPTIONS, DESCRIBE, PLAY, PAUSE, SETUP, TEARDOWN, SET_PARAMETER, GET_PARAMETER

返回rtsp服务器支持的请求方法

2）GET_PARAMETER rtsp://192.168.58.173:554/channel=1/trackID=video RTSP/1.0

根据规范，GET_PARAMETER服务器和客户端可以选择实现。

目前有的VLC采用TCP进行rtsp播放视频流，如果不响应GET_PARAMETER会产生断流的情况

在暂停流媒体播放，定期发送GET_PARAMETER作为心跳包维持连接

LIVE555针对该指令请求的回复代码如下，简单回复下当前的版本号

void RTSPServer::RTSPClientSession
::handleCmd_GET_PARAMETER(RTSPServer::RTSPClientConnection* ourClientConnection,
ServerMediaSubsession* /*subsession*/, char const* /*fullRequestStr*/) {
// By default, we implement "GET_PARAMETER" just as a 'keep alive', and send back a dummy response.
// (If you want to handle "GET_PARAMETER" properly, you can do so by defining a subclass of "RTSPServer"
// and "RTSPServer::RTSPClientSession", and then reimplement this virtual function in your subclass.)
setRTSPResponse(ourClientConnection, "200 OK", fOurSessionId, LIVEMEDIA_LIBRARY_VERSION_STRING);
}

问题以及解决方案

1)405 Method Not Allowed
主要是在进行OPTIONS指令包封装的时候，通过wireshark抓包进行编写，报文内容如下：

Request: OPTIONS rtsp:://192.168.1.88 RTSP/1.0\r\n
Method: OPTIONS
URL: rtsp:://192.168.1.88
以为信息的开头是Request，实际上这是解析的语句
错误：
request_stream << "REQUEST: " <<"OPTIONS " << "rtsp://192.168.0.114 " << "RTSP/1.0\r\n";
request_stream << "CSeq: " << "2\r\n";
request_stream << "User-Agent: " << "LibVLC/2.1.5 (Live555 Streaming Media v2014.0)\r\n\r\n";

正确：
request_stream << "OPTIONS " << "rtsp://192.168.0.114 " << "RTSP/1.0\r\n";
request_stream << "CSeq: " << "2\r\n";
request_stream << "User-Agent: " << "LibVLC/2.1.5 (Live555 Streaming Media v2014.0)\r\n\r\n";

2)404 Stream Not Found
主要是在进行DESCRIBE的时候没有填写获取的视频流信息
错误:
request_stream << "DESCRIBE " << "rtsp://192.168.0.114 " << "RTSP/1.0\r\n";
正确:
request_stream << "DESCRIBE " << "rtsp://192.168.0.114/smoke.264 " << "RTSP/1.0\r\n";//error

3) 451 Parameter Not Understood

主要是url后面没有指定trackid,例如指定/trackID=0

错误：

SETUP rtsp://192.168.18.201:554/cam/realmonitor?channel=1&subtype=0 RTSP/1.0
CSeq: 4
User-Agent: LibVLC/2.2.8 (LIVE555 Streaming Media v2017.08.22)
Authorization: Digest username="admin", realm="Login to 4M01111PAJB50A1", nonce="3344152e2d9f717b3c3f29792f31e125", uri="rtsp://192.168.18.201:554/cam/realmonitor?channel=1&subtype=0", response="d69d677d06e9a5471327ff977cc09d9e"
Transport: RTP/AVP;unicast;client_port=45056-45057
RTSP/1.0 451 Parameter Not Understood
CSeq: 4
Session: 1029986489118

正确：

SETUP rtsp://192.168.18.201:554/cam/realmonitor?channel=1&subtype=0/trackID=0 RTSP/1.0
CSeq: 5
User-Agent: LibVLC/2.2.8 (LIVE555 Streaming Media v2017.08.22)
Authorization: Digest username="admin", realm="Login to 4M01111PAJB50A1", nonce="ba0a98388d61508258fc859081a62fb0", uri="rtsp://192.168.18.201:554/cam/realmonitor?channel=1&subtype=0", response="c72d514d419ee7383c4cc1f94a0b785d"
Session: 1039947334568
Transport: RTP/AVP;unicast;client_port=45058-45059

测试代码
#include <iostream>
#include <fstream>
#include <string>
#include <sstream>
#include <boost/asio.hpp>
#include "socket.h"

using namespace std;
using namespace boost::asio;

const char pszRtspServerIP[32] = "192.168.0.114";
short sRtspServerPort = 8554;

void WriteFile(char* buf);
{
ofstream ofs;
ofs.open("rtspoption.txt");
ofs << buf << endl;
ofs.close();
}

int HandleOptionCommand(ip::tcp::socket &sock)
{
boost::system::error_code ec;
boost::asio::streambuf request;
std::ostream request_stream(&request);
request_stream << "OPTIONS " << "rtsp://192.168.0.114 " << "RTSP/1.0\r\n";
request_stream << "CSeq: " << "2\r\n";
request_stream << "User-Agent: " << "LibVLC/2.1.5 (Live555 Streaming Media v2014.0)\r\n\r\n";

boost::asio::write(sock, request);

char buf[1024] = { 0 };
size_t len = sock.read_some(buffer(buf), ec);
return 0;
}

int HanleDescribeCommand(ip::tcp::socket &sock)
{
boost::system::error_code ec;
boost::asio::streambuf request;
std::ostream request_stream(&request);
request_stream << "DESCRIBE " << "rtsp://192.168.0.114/smoke.264 " << "RTSP/1.0\r\n";
request_stream << "CSeq: " << "3\r\n";
request_stream << "Accept: " << "application/sdp\r\n";
request_stream << "User-Agent: " << "LibVLC/2.1.5 (Live555 Streaming Media v2014.0)\r\n\r\n";

boost::asio::write(sock, request);

char buf[1024] = { 0 };
size_t len = sock.read_some(buffer(buf), ec);
//a=control:track1
return 0;
}

int HandleSetupCommand(ip::tcp::socket &sock)
{
boost::system::error_code ec;
boost::asio::streambuf request;
std::ostream request_stream(&request);
request_stream << "SETUP " << "rtsp://192.168.0.114/smoke.264 " << "RTSP/1.0\r\n";
request_stream << "CSeq: " << "3\r\n";
request_stream << "Transport: " << "RTP/AVP/TCP;unicast;interleaved=0-1\r\n";
request_stream << "User-Agent: " << "LibVLC/2.1.5 (Live555 Streaming Media v2014.0)\r\n\r\n";

boost::asio::write(sock, request);

char buf[1024] = { 0 };
size_t len = sock.read_some(buffer(buf), ec);
return 0;

}

int main(int argc, char* argv[])
{
io_service iosev;
ip::tcp::socket socket(iosev);
ip::tcp::endpoint ep(ip::address_v4::from_string(pszRtspServerIP), sRtspServerPort);
boost::system::error_code ec;
socket.connect(ep, ec);
if (ec) return -1;

HandleOptionCommand(socket);
HanleDescribeCommand(socket);
HandleSetupCommand(socket);

return 0;
}

RTSP解复用器

AVInputFormat ff_rtsp_demuxer = {
.name = "rtsp",
.long_name = NULL_IF_CONFIG_SMALL("RTSP input"),
.priv_data_size = sizeof(RTSPState),
.read_probe = rtsp_probe,
.read_header = rtsp_read_header,
.read_packet = rtsp_read_packet,
.read_close = rtsp_read_close,
.read_seek = rtsp_read_seek,
.flags = AVFMT_NOFILE,
.read_play = rtsp_read_play,
.read_pause = rtsp_read_pause,
.priv_class = &rtsp_demuxer_class,
};

代码堆栈剖析：
ffplayd.exe!sdp_parse_line(AVFormatContext * s, SDPParseState * s1, int letter, const char * buf) 行 576 C
ffplayd.exe!ff_sdp_parse(AVFormatContext * s, const char * content) 行 721 C
ffplayd.exe!ff_rtsp_setup_input_streams(AVFormatContext * s, RTSPMessageHeader * reply) 行 622 C
ffplayd.exe!ff_rtsp_connect(AVFormatContext * s) 行 1897 C
> ffplayd.exe!rtsp_read_header(AVFormatContext * s) 行 726 C
ffplayd.exe!avformat_open_input(AVFormatContext * * ps, const char * filename, AVInputFormat * fmt, AVDictionary * * options) 行 631 C
ffplayd.exe!read_thread(void * arg) 行 2780 C
ffplayd.exe!SDL_RunThread(void * data) 行 283 C
ffplayd.exe!RunThread(void * data) 行 91 C
ffplayd.exe!RunThreadViaBeginThreadEx(void * data) 行 106 C
[外部代码]

探测格式，主要是根据URL起始字符的匹配

static int rtsp_probe(const AVProbeData *p)
{
if (
#if CONFIG_TLS_PROTOCOL
av_strstart(p->filename, "rtsps:", NULL) ||
#endif
av_strstart(p->filename, "rtsp:", NULL))
return AVPROBE_SCORE_MAX;
return 0;
}

ff_rtsp_setup_input_streams创建rtsp交互连接

ff_rtsp_send_cmd优先发送DESCRIBE指令，海康和大华RTSP指令交互的区别在于，大华在OPTIONS阶段就会请求认证信息RTSP/1.0 401 Unauthorized

int ff_sdp_parse(AVFormatContext *s, const char *content) 解析SDK中的内容，实际上这里就已经完全知道码流的数据格式以及所有的流信息，根本不需要探测码流格式

sdp_parse_rtpmap函数分析出h264码流格式，保存在AVFormatContext中的stream流中

/* parse the rtpmap description: <codec_name>/<clock_rate>[/<other params>] */
static int sdp_parse_rtpmap(AVFormatContext *s,
AVStream *st, RTSPStream *rtsp_st,
int payload_type, const char *p)
{
//*p = H264/90000
//最终st中保存探测的码流格式
AVCodecParameters *par = st->codecpar;
char buf[256];
int i;
const AVCodecDescriptor *desc;
const char *c_name;

/* See if we can handle this kind of payload.
* The space should normally not be there but some Real streams or
* particular servers ("RealServer Version 6.1.3.970", see issue 1658)
* have a trailing space. */
//关键，buf获取到H264
get_word_sep(buf, sizeof(buf), "/ ", &p);
if (payload_type < RTP_PT_PRIVATE) {
/* We are in a standard case
* (from http://www.iana.org/assignments/rtp-parameters). */
par->codec_id = ff_rtp_codec_id(buf, par->codec_type);
}

if (par->codec_id == AV_CODEC_ID_NONE) {
const RTPDynamicProtocolHandler *handler =
ff_rtp_handler_find_by_name(buf, par->codec_type);
init_rtp_handler(handler, rtsp_st, st);
/* If no dynamic handler was found, check with the list of standard
* allocated types, if such a stream for some reason happens to
* use a private payload type. This isn't handled in rtpdec.c, since
* the format name from the rtpmap line never is passed into rtpdec. */
if (!rtsp_st->dynamic_handler)
par->codec_id = ff_rtp_codec_id(buf, par->codec_type);
}

desc = avcodec_descriptor_get(par->codec_id);
if (desc && desc->name)
c_name = desc->name;
else
c_name = "(null)";

get_word_sep(buf, sizeof(buf), "/", &p);
i = atoi(buf);
switch (par->codec_type) {
case AVMEDIA_TYPE_AUDIO:
av_log(s, AV_LOG_DEBUG, "audio codec set to: %s\n", c_name);
par->sample_rate = RTSP_DEFAULT_AUDIO_SAMPLERATE;
par->channels = RTSP_DEFAULT_NB_AUDIO_CHANNELS;
if (i > 0) {
par->sample_rate = i;
avpriv_set_pts_info(st, 32, 1, par->sample_rate);
get_word_sep(buf, sizeof(buf), "/", &p);
i = atoi(buf);
if (i > 0)
par->channels = i;
}
av_log(s, AV_LOG_DEBUG, "audio samplerate set to: %i\n",
par->sample_rate);
av_log(s, AV_LOG_DEBUG, "audio channels set to: %i\n",
par->channels);
break;
case AVMEDIA_TYPE_VIDEO:
av_log(s, AV_LOG_DEBUG, "video codec set to: %s\n", c_name);
if (i > 0)
avpriv_set_pts_info(st, 32, 1, i);
break;
default:
break;
}
finalize_rtp_handler_init(s, rtsp_st, st);
return 0;
}

附上SDK数据
v=0
o=- 1625669797472718 1625669797472718 IN IP4 192.168.18.204
s=Media Presentation
e=NONE
b=AS:5050
t=0 0
a=control:rtsp://192.168.18.204:554/h264/ch1/main/av_stream/
m=video 0 RTP/AVP 96
b=AS:5000
a=control:rtsp://192.168.18.204:554/h264/ch1/main/av_stream/trackID=1
a=rtpmap:96 H264/90000
a=fmtp:96 profile-level-id=420029; packetization-mode=1; sprop-parameter-sets=Z0IAKpY1QPAET8s3AQEBAg==,aM48gA==
a=Media_header:MEDIAINFO=494D4B48010100000400000100000000000000000000000000000000000000000000000000000000;
a=appversion:1.0

码流格式探测分析

> ffplayd.exe!rtp_read(URLContext * h, unsigned char * buf, int size) 行 377 C

(调用recvfrom读取RTP传输数据,transfer_func指向了rtp_read的实现)

ffplayd.exe!retry_transfer_wrapper(URLContext * h, unsigned char * buf, int size, int size_min, int (URLContext *, unsigned char *, int) * transfer_func) 行 376 C

ffplayd.exe!ffurl_read(URLContext * h, unsigned char * buf, int size) 行 412 C

ffplayd.exe!udp_read_packet(AVFormatContext * s, RTSPStream * * prtsp_st, unsigned char * buf, int buf_size, __int64 wait_end) 行 2033 C

ffplayd.exe!read_packet(AVFormatContext * s, RTSPStream * * rtsp_st, RTSPStream * first_queue_st, __int64 wait_end) 行 2116 C

ffplayd.exe!ff_rtsp_fetch_packet(AVFormatContext * s, AVPacket * pkt) 行 2202 C

ffplayd.exe!rtsp_read_packet(AVFormatContext * s, AVPacket * pkt) 行 879 C

ffplayd.exe!ff_read_packet(AVFormatContext * s, AVPacket * pkt) 行 856 C

ffplayd.exe!read_frame_internal(AVFormatContext * s, AVPacket * pkt) 行 1582 C

ffplayd.exe!avformat_find_stream_info(AVFormatContext * ic, AVDictionary * * options) 行 3772 C

ffplayd.exe!read_thread(void * arg) 行 2805 C

ffplayd.exe!SDL_RunThread(void * data) 行 283 C

ffplayd.exe!RunThread(void * data) 行 91 C

ffplayd.exe!RunThreadViaBeginThreadEx(void * data) 行 106

重点分析函数ff_rtsp_fetch_packet，该函数调用read_packet获取到RTP数据，调用ff_rtp_parse_packet分析RTP数据，去掉RTP包头，添加起始码，然后封装成AVPacket，但是封装的AVPacket并不是完整的NAL单元的视频流，对于FU-A分包的数据，仍然需要对多个AVPacket进行重新组装

> ffplayd.exe!h264_handle_packet_fu_a(AVFormatContext * ctx, PayloadContext * data, AVPacket * pkt, const unsigned char * buf, int len, int * nal_counters, int nal_mask) 行 291 C

(对FU-A分包的RTP格式数据，会根据是否是第一个包添加起始码，关键是start_bit = fu_header >> 7;)

ffplayd.exe!h264_handle_packet(AVFormatContext * ctx, PayloadContext * data, AVStream * st, AVPacket * pkt, unsigned int * timestamp, const unsigned char * buf, int len, unsigned short seq, int flags) 行 359 C

ffplayd.exe!rtp_parse_packet_internal(RTPDemuxContext * s, AVPacket * pkt, const unsigned char * buf, int len) 行 692 C

ffplayd.exe!rtp_parse_one_packet(RTPDemuxContext * s, AVPacket * pkt, unsigned char * * bufptr, int len) 行 841 C

ffplayd.exe!ff_rtp_parse_packet(RTPDemuxContext * s, AVPacket * pkt, unsigned char * * bufptr, int len) 行 875 C

ffplayd.exe!ff_rtsp_fetch_packet(AVFormatContext * s, AVPacket * pkt) 行 2217 C

ffplayd.exe!rtsp_read_packet(AVFormatContext * s, AVPacket * pkt) 行 879 C

ffplayd.exe!ff_read_packet(AVFormatContext * s, AVPacket * pkt) 行 856 C

ffplayd.exe!read_frame_internal(AVFormatContext * s, AVPacket * pkt) 行 1582 C

ffplayd.exe!av_read_frame(AVFormatContext * s, AVPacket * pkt) 行 1776 C

ffplayd.exe!read_thread(void * arg) 行 3008 C

ffplayd.exe!SDL_RunThread(void * data) 行 283 C

ffplayd.exe!RunThread(void * data) 行 91 C

ffplayd.exe!RunThreadViaBeginThreadEx(void * data) 行 106 C

av_read_frame分包代码剖析

ffplayd.exe!ff_combine_frame(ParseContext * pc, int next, const unsigned char * * buf, int * buf_size) 行 265 C

> ffplayd.exe!h264_parse(AVCodecParserContext * s, AVCodecContext * avctx, const unsigned char * * poutbuf, int * poutbuf_size, const unsigned char * buf, int buf_size) 行 595 C

ffplayd.exe!av_parser_parse2(AVCodecParserContext * s, AVCodecContext * avctx, unsigned char * * poutbuf, int * poutbuf_size, const unsigned char * buf, int buf_size, __int64 pts, __int64 dts, __int64 pos) 行 166 C

ffplayd.exe!parse_packet(AVFormatContext * s, AVPacket * pkt, int stream_index) 行 1461 C

ffplayd.exe!read_frame_internal(AVFormatContext * s, AVPacket * pkt) 行 1675 C

ffplayd.exe!av_read_frame(AVFormatContext * s, AVPacket * pkt) 行 1776 C

ffplayd.exe!read_thread(void * arg) 行 3008 C

ffplayd.exe!SDL_RunThread(void * data) 行 283 C

ffplayd.exe!RunThread(void * data) 行 91 C

ffplayd.exe!RunThreadViaBeginThreadEx(void * data) 行 106 C

[外部代码]

HEVC码流探测剖析

例如:"rtsp://admin:admin12345@192.168.28.136:554/h265/ch1/main/av_stream"

请求海康摄像机H265码流

SDP报文如下:
v=0
o=- 1566124110963848 1566124110963848 IN IP4 192.168.28.136
s=Media Presentation
e=NONE
b=AS:5100
t=0 0
a=control:rtsp://192.168.28.136:554/h265/ch1/main/av_stream/
m=video 0 RTP/AVP 96
c=IN IP4 0.0.0.0
b=AS:5000
a=recvonly
a=x-dimensions:1920,1080
a=control:rtsp://192.168.28.136:554/h265/ch1/main/av_stream/trackID=1
a=rtpmap:96 H265/90000
a=fmtp:96 sprop-sps=QgEBAWAAAAMAsAAAAwAAAwB7oAPAgBDlja5JMvTcBAQEAg==; sprop-pps=RAHA8vA8kAA=
m=audio 0 RTP/AVP 8
c=IN IP4 0.0.0.0
b=AS:50
a=recvonly
a=control:rtsp://192.168.28.136:554/h265/ch1/main/av_stream/trackID=2
a=rtpmap:8 PCMA/8000
a=Media_header:MEDIAINFO=494D4B48010200000400050011710110401F000000FA000000000000000000000000000000000000;
a=appversion:1.0

static int sdp_parse_rtpmap(AVFormatContext *s,
AVStream *st, RTSPStream *rtsp_st,
int payload_type, const char *p)

通过sdp_parse_rtpmap函数分析SDP中的a=rtpmap:96 H265/90000
在调用函数 const RTPDynamicProtocolHandler *handler =
ff_rtp_handler_find_by_name(buf, par->codec_type);中获取到h265的处理句柄

rtsp 分包NALU

       if (codec_id == AV_CODEC_ID_HEVC)
            ret = hevc_parse_nal_header(nal, logctx);
        else
            ret = h264_parse_nal_header(nal, logctx);

static void parse_fmtp(AVFormatContext *s, RTSPState *rt,
int payload_type, const char *line)
通过h265的句柄调用
static av_cold int hevc_parse_sdp_line(AVFormatContext *ctx, int st_index,
PayloadContext *hevc_data, const char *line)
该函数将SPS/PPS的内容保存在(AVFormatContext结构体中的AVStream流中的extradata变量

创建视频流和音频流
update_stream_avctx函数只是将avcodec_parameters_to_context(st->internal->avctx, st->codecpar);
并没有做其他的操作

上一篇：boost thread 类应用以及资源耗尽异常

下一篇：软考特殊的测试阶段及任务

提问和评论都可以，用心的回复会被更多人看到评论

发布评论

相关文章

官方博客	全部文章	热门标签	班级博客
了解我们	网站地图	意见反馈

鸿蒙开发者社区	51CTO学堂
51CTO	软考资讯