(目录)
filter，可以翻译为过滤器，滤镜。在FFmpeg中有多种多样的滤镜，你可以把他们当成一个个小工具，专门用于处理视频和音频数据，以便实现一定的目的。如overlay这个滤镜，可以将一个图画覆盖到另一个图画上transport这个滤镜可以将图画做旋转等等。
一、概述
FFMPEG 除了具有强大的封装/解封装，编/解码功能之外，还包含了一个非常强大的组件，滤镜avfilter。 avfilter 组件常用于多媒体处理与编辑，ffmpeg 包含多种滤镜，比如旋转，加水印，多宫格等等。
一个filter的输出可以作为另一个filter的输入，因此多个filter可以组织成为一个网状的filter graph，从而实现更加复杂或者综合的任务。
关于 filter 的概念和用法可以去查找 FFmpeg 的其他资料，这里不再赘述。
libavfilter 库实现了 filter 相关的功能，使得我们可以利用 FFmpeg 提供的诸多 filter 来实现自己所需的功能。具体的使用方法见官方示例：filter_audio.c，filtering_audio.c，filtering_video.c。
二、重要结构体
在使用FFmpeg开发时，使用AVFilter的流程较为复杂，涉及到的数据结构和函数也比较多，那么使用FFmpeg AVFilter的整体流程是什么样，在其执行过程中都有哪些步骤，需要注意哪些细节？这些都是需要我们整理和总结的。 首先，我们需要引入三个概念结构体：AVFilterGraph 、AVFilterContext、AVFilter。
2.1、AVFilterGraph
AVFilterGraph表示一个filter graph，当然它也包含了filter chain的概念，对filters系统的整体管理。 graph 包含了诸多 filter context实例，并负责它们之间的link，graph会负责创建、保存、释放 这些相关的 filter context和link，一般不需要用户进行管理。除此之外，它还有线程特性和最大线程数量的字段和filter context类似。 graph的操作有：分配一个graph，往graph中添加一个filter context，添加一个 filter graph，对 filter 进行 link 操作，检查内部的link和format是否有效，释放graph等。
typedef struct AVFilterGraph {
const AVClass *av_class;
AVFilterContext **filters;
unsigned nb_filters;
char *scale_sws_opts; ///< sws options to use for the auto-inserted scale filters
/**
* Type of multithreading allowed for filters in this graph. A combination
* of AVFILTER_THREAD_* flags.
*
* May be set by the caller at any point, the setting will apply to all
* filters initialized after that. The default is allowing everything.
*
* When a filter in this graph is initialized, this field is combined using
* bit AND with AVFilterContext.thread_type to get the final mask used for
* determining allowed threading types. I.e. a threading type needs to be
* set in both to be allowed.
*/
int thread_type;
/**
* Maximum number of threads used by filters in this graph. May be set by
* the caller before adding any filters to the filtergraph. Zero (the
* default) means that the number of threads is determined automatically.
*/
int nb_threads;
/**
* Opaque object for libavfilter internal use.
*/
AVFilterGraphInternal *internal;
/**
* Opaque user data. May be set by the caller to an arbitrary value, e.g. to
* be used from callbacks like @ref AVFilterGraph.execute.
* Libavfilter will not touch this field in any way.
*/
void *opaque;
/**
* This callback may be set by the caller immediately after allocating the
* graph and before adding any filters to it, to provide a custom
* multithreading implementation.
*
* If set, filters with slice threading capability will call this callback
* to execute multiple jobs in parallel.
*
* If this field is left unset, libavfilter will use its internal
* implementation, which may or may not be multithreaded depending on the
* platform and build options.
*/
avfilter_execute_func *execute;
char *aresample_swr_opts; ///< swr options to use for the auto-inserted aresample filters, Access ONLY through AVOptions
/**
* Private fields
*
* The following fields are for internal use only.
* Their type, offset, number and semantic can change without notice.
*/
AVFilterLink **sink_links;
int sink_links_count;
unsigned disable_auto_convert;
} AVFilterGraph;
我们简单点看，我们在调用的时候我们真正能触及到的一般主要是const AVClass av_class;AVFilterContext ** filters; 这两个对象 av_class在初始化AVFilterGraph 的时候就已经一起初始化了，它就是存储滤镜一些相关配置的，在初始化时就已经有一些默认的配置被赋值。 filters需要后续滤镜生成后再进行赋值绑定的，它是滤镜上下文指针类型的指针。
2.2、AVFilter
滤镜结构体，是用来存储具体滤镜信息和一些函数指针，函数指针能让我们外部的进行重写逻辑，作为回调等等。但对于我们实现基本滤镜来说我们根本不需要了解，我们只要关注一些基本信息，例如滤镜名字，滤镜参数等。
typedef struct AVFilter {
/**
* Filter name. Must be non-NULL and unique among filters.
*/
const char *name;
/**
* A description of the filter. May be NULL.
*
* You should use the NULL_IF_CONFIG_SMALL() macro to define it.
*/
const char *description;
/**
* List of static inputs.
*
* NULL if there are no (static) inputs. Instances of filters with
* AVFILTER_FLAG_DYNAMIC_INPUTS set may have more inputs than present in
* this list.
*/
const AVFilterPad *inputs;
/**
* List of static outputs.
*
* NULL if there are no (static) outputs. Instances of filters with
* AVFILTER_FLAG_DYNAMIC_OUTPUTS set may have more outputs than present in
* this list.
*/
const AVFilterPad *outputs;
/**
* A class for the private data, used to declare filter private AVOptions.
* This field is NULL for filters that do not declare any options.
*
* If this field is non-NULL, the first member of the filter private data
* must be a pointer to AVClass, which will be set by libavfilter generic
* code to this class.
*/
const AVClass *priv_class;
/**
* A combination of AVFILTER_FLAG_*
*/
int flags;
/*****************************************************************
* All fields below this line are not part of the public API. They
* may not be used outside of libavfilter and can be changed and
* removed at will.
* New public fields should be added right above.
*****************************************************************
*/
/**
* The number of entries in the list of inputs.
*/
uint8_t nb_inputs;
/**
* The number of entries in the list of outputs.
*/
uint8_t nb_outputs;
/**
* This field determines the state of the formats union.
* It is an enum FilterFormatsState value.
*/
uint8_t formats_state;
/**
* Filter pre-initialization function
*
* This callback will be called immediately after the filter context is
* allocated, to allow allocating and initing sub-objects.
*
* If this callback is not NULL, the uninit callback will be called on
* allocation failure.
*
* @return 0 on success,
* AVERROR code on failure (but the code will be
* dropped and treated as ENOMEM by the calling code)
*/
int (*preinit)(AVFilterContext *ctx);
/**
* Filter initialization function.
*
* This callback will be called only once during the filter lifetime, after
* all the options have been set, but before links between filters are
* established and format negotiation is done.
*
* Basic filter initialization should be done here. Filters with dynamic
* inputs and/or outputs should create those inputs/outputs here based on
* provided options. No more changes to this filter's inputs/outputs can be
* done after this callback.
*
* This callback must not assume that the filter links exist or frame
* parameters are known.
*
* @ref AVFilter.uninit "uninit" is guaranteed to be called even if
* initialization fails, so this callback does not have to clean up on
* failure.
*
* @return 0 on success, a negative AVERROR on failure
*/
int (*init)(AVFilterContext *ctx);
/**
* Filter uninitialization function.
*
* Called only once right before the filter is freed. Should deallocate any
* memory held by the filter, release any buffer references, etc. It does
* not need to deallocate the AVFilterContext.priv memory itself.
*
* This callback may be called even if @ref AVFilter.init "init" was not
* called or failed, so it must be prepared to handle such a situation.
*/
void (*uninit)(AVFilterContext *ctx);
/**
* The state of the following union is determined by formats_state.
* See the documentation of enum FilterFormatsState in internal.h.
*/
union {
/**
* Query formats supported by the filter on its inputs and outputs.
*
* This callback is called after the filter is initialized (so the inputs
* and outputs are fixed), shortly before the format negotiation. This
* callback may be called more than once.
*
* This callback must set ::AVFilterLink's
* @ref AVFilterFormatsConfig.formats "outcfg.formats"
* on every input link and
* @ref AVFilterFormatsConfig.formats "incfg.formats"
* on every output link to a list of pixel/sample formats that the filter
* supports on that link.
* For audio links, this filter must also set
* @ref AVFilterFormatsConfig.samplerates "incfg.samplerates"
* /
* @ref AVFilterFormatsConfig.samplerates "outcfg.samplerates"
* and @ref AVFilterFormatsConfig.channel_layouts "incfg.channel_layouts"
* /
* @ref AVFilterFormatsConfig.channel_layouts "outcfg.channel_layouts"
* analogously.
*
* This callback must never be NULL if the union is in this state.
*
* @return zero on success, a negative value corresponding to an
* AVERROR code otherwise
*/
int (*query_func)(AVFilterContext *);
/**
* A pointer to an array of admissible pixel formats delimited
* by AV_PIX_FMT_NONE. The generic code will use this list
* to indicate that this filter supports each of these pixel formats,
* provided that all inputs and outputs use the same pixel format.
*
* This list must never be NULL if the union is in this state.
* The type of all inputs and outputs of filters using this must
* be AVMEDIA_TYPE_VIDEO.
*/
const enum AVPixelFormat *pixels_list;
/**
* Analogous to pixels, but delimited by AV_SAMPLE_FMT_NONE
* and restricted to filters that only have AVMEDIA_TYPE_AUDIO
* inputs and outputs.
*
* In addition to that the generic code will mark all inputs
* and all outputs as supporting all sample rates and every
* channel count and channel layout, as long as all inputs
* and outputs use the same sample rate and channel count/layout.
*/
const enum AVSampleFormat *samples_list;
/**
* Equivalent to { pix_fmt, AV_PIX_FMT_NONE } as pixels_list.
*/
enum AVPixelFormat pix_fmt;
/**
* Equivalent to { sample_fmt, AV_SAMPLE_FMT_NONE } as samples_list.
*/
enum AVSampleFormat sample_fmt;
} formats;
int priv_size; ///< size of private data to allocate for the filter
int flags_internal; ///< Additional flags for avfilter internal use only.
/**
* Make the filter instance process a command.
*
* @param cmd the command to process, for handling simplicity all commands must be alphanumeric only
* @param arg the argument for the command
* @param res a buffer with size res_size where the filter(s) can return a response. This must not change when the command is not supported.
* @param flags if AVFILTER_CMD_FLAG_FAST is set and the command would be
* time consuming then a filter should treat it like an unsupported command
*
* @returns >=0 on success otherwise an error code.
* AVERROR(ENOSYS) on unsupported commands
*/
int (*process_command)(AVFilterContext *, const char *cmd, const char *arg, char *res, int res_len, int flags);
/**
* Filter activation function.
*
* Called when any processing is needed from the filter, instead of any
* filter_frame and request_frame on pads.
*
* The function must examine inlinks and outlinks and perform a single
* step of processing. If there is nothing to do, the function must do
* nothing and not return an error. If more steps are or may be
* possible, it must use ff_filter_set_ready() to schedule another
* activation.
*/
int (*activate)(AVFilterContext *ctx);
} AVFilter;
AVFilter表示一个被注册的filter类型，如果仅是为了使用它，我们只需知道它的部分特性就可以了。 1、AVFilter.name 和 AVFilter.description：filter 的名称和描述。 2. AVFilter.inputs 和 AVFilter.outputs：AVFilterPad 的数组，描述了filter的输入和输出。 3. AVFilter.flags：filter的特性。 首先，针对 filter 的输入和输出数组，有以下三个帮助函数：
/**
* 获取 AVFilterPad 数组的数量，该数组以NULL结尾，一般传入AVFilter.inputs和AVFilter.outputs
*/
unsigned avfilter_filter_pad_count(const AVFilter *filter, int is_output)
{
return is_output ? filter->nb_outputs : filter->nb_inputs;
}
/**
* Get the name of an AVFilterPad.
*
* @param pads an array of AVFilterPads
* @param pad_idx index of the pad in the array; it is the caller's
* responsibility to ensure the index is valid
*
* @return name of the pad_idx'th pad in pads
*/
const char *avfilter_pad_get_name(const AVFilterPad *pads, int pad_idx)
{
return pads[pad_idx].name;
}
/**
* Get the type of an AVFilterPad.
*
* @param pads an array of AVFilterPads
* @param pad_idx index of the pad in the array; it is the caller's
* responsibility to ensure the index is valid
*
* @return type of the pad_idx'th pad in pads
*/
// 获取数组中指向下标的 pad 所需的数据类型，调用者应确保传入的下标有效
enum AVMediaType avfilter_pad_get_type(const AVFilterPad *pads, int pad_idx);
然后，就是 AVFilter 的 flag，其可取值如下：
/**
* The number of the filter inputs is not determined just by AVFilter.inputs.
* The filter might add additional inputs during initialization depending on the
* options supplied to it.
*/
#define AVFILTER_FLAG_DYNAMIC_INPUTS (1 << 0)
/**
* The number of the filter outputs is not determined just by AVFilter.outputs.
* The filter might add additional outputs during initialization depending on
* the options supplied to it.
*/
#define AVFILTER_FLAG_DYNAMIC_OUTPUTS (1 << 1)
/**
* The filter supports multithreading by splitting frames into multiple parts
* and processing them concurrently.
*/
#define AVFILTER_FLAG_SLICE_THREADS (1 << 2)
/**
* The filter is a "metadata" filter - it does not modify the frame data in any
* way. It may only affect the metadata (i.e. those fields copied by
* av_frame_copy_props()).
*
* More precisely, this means:
* - video: the data of any frame output by the filter must be exactly equal to
* some frame that is received on one of its inputs. Furthermore, all frames
* produced on a given output must correspond to frames received on the same
* input and their order must be unchanged. Note that the filter may still
* drop or duplicate the frames.
* - audio: the data produced by the filter on any of its outputs (viewed e.g.
* as an array of interleaved samples) must be exactly equal to the data
* received by the filter on one of its inputs.
*/
#define AVFILTER_FLAG_METADATA_ONLY (1 << 3)
/**
* The filter can create hardware frames using AVFilterContext.hw_device_ctx.
*/
#define AVFILTER_FLAG_HWDEVICE (1 << 4)
/**
* Some filters support a generic "enable" expression option that can be used
* to enable or disable a filter in the timeline. Filters supporting this
* option have this flag set. When the enable expression is false, the default
* no-op filter_frame() function is called in place of the filter_frame()
* callback defined on each input pad, thus the frame is passed unchanged to
* the next filters.
*/
#define AVFILTER_FLAG_SUPPORT_TIMELINE_GENERIC (1 << 16)
/**
* Same as AVFILTER_FLAG_SUPPORT_TIMELINE_GENERIC, except that the filter will
* have its filter_frame() callback(s) called as usual even when the enable
* expression is false. The filter will disable filtering within the
* filter_frame() callback(s) itself, for example executing code depending on
* the AVFilterContext->is_disabled value.
*/
#define AVFILTER_FLAG_SUPPORT_TIMELINE_INTERNAL (1 << 17)
/**
* Handy mask to test whether the filter supports or no the timeline feature
* (internally or generically).
*/
#define AVFILTER_FLAG_SUPPORT_TIMELINE (AVFILTER_FLAG_SUPPORT_TIMELINE_GENERIC | AVFILTER_FLAG_SUPPORT_TIMELINE_INTERNAL)
2.3、AVFilterContext
AVFilterContext表示一个AVFilter的实例，我们在实际使用filter时，就是使用这个类型。 AVFilterContext在被使用前，它必须是ready状态的，也就是被初始化的。说白了，就是需要对filter进行一些选项上的设置，然后通过初始化告诉FFmpeg我们已经做了相关的配置。
/** An instance of a filter */
struct AVFilterContext {
const AVClass *av_class; ///< needed for av_log() and filters common options
const AVFilter *filter; ///< the AVFilter of which this is an instance
char *name; ///< name of this filter instance
AVFilterPad *input_pads; ///< array of input pads
AVFilterLink **inputs; ///< array of pointers to input links
unsigned nb_inputs; ///< number of input pads
AVFilterPad *output_pads; ///< array of output pads
AVFilterLink **outputs; ///< array of pointers to output links
unsigned nb_outputs; ///< number of output pads
void *priv; ///< private data for use by the filter
struct AVFilterGraph *graph; ///< filtergraph this filter belongs to
/**
* Type of multithreading being allowed/used. A combination of
* AVFILTER_THREAD_* flags.
*
* May be set by the caller before initializing the filter to forbid some
* or all kinds of multithreading for this filter. The default is allowing
* everything.
*
* When the filter is initialized, this field is combined using bit AND with
* AVFilterGraph.thread_type to get the final mask used for determining
* allowed threading types. I.e. a threading type needs to be set in both
* to be allowed.
*
* After the filter is initialized, libavfilter sets this field to the
* threading type that is actually used (0 for no multithreading).
*/
int thread_type;
/**
* An opaque struct for libavfilter internal use.
*/
AVFilterInternal *internal;
struct AVFilterCommand *command_queue;
char *enable_str; ///< enable expression string
void *enable; ///< parsed expression (AVExpr*)
double *var_values; ///< variable values for the enable expression
int is_disabled; ///< the enabled state from the last expression evaluation
/**
* For filters which will create hardware frames, sets the device the
* filter should create them in. All other filters will ignore this field:
* in particular, a filter which consumes or processes hardware frames will
* instead use the hw_frames_ctx field in AVFilterLink to carry the
* hardware context information.
*
* May be set by the caller on filters flagged with AVFILTER_FLAG_HWDEVICE
* before initializing the filter with avfilter_init_str() or
* avfilter_init_dict().
*/
AVBufferRef *hw_device_ctx;
/**
* Max number of threads allowed in this filter instance.
* If <= 0, its value is ignored.
* Overrides global number of threads set per filter graph.
*/
int nb_threads;
/**
* Ready status of the filter.
* A non-0 value means that the filter needs activating;
* a higher value suggests a more urgent activation.
*/
unsigned ready;
/**
* Sets the number of extra hardware frames which the filter will
* allocate on its output links for use in following filters or by
* the caller.
*
* Some hardware filters require all frames that they will use for
* output to be defined in advance before filtering starts. For such
* filters, any hardware frame pools used for output must therefore be
* of fixed size. The extra frames set here are on top of any number
* that the filter needs internally in order to operate normally.
*
* This field must be set before the graph containing this filter is
* configured.
*/
int extra_hw_frames;
};
AVFilterContext 表示 AVFilter 的一个实例。其重要字段如下： 1. AVFilterContext .filter：这个 filter context 是那种 filter 的实例。 2. AVFilterContext .name：这个 filter context 的名称。 3. AVFilterContext.graph：这个 filter context 属于哪个 graph。 4. AVFilterContext .input_pads，AVFilterContext.inputs ，AVFilterContext.nb_inputs：分别描述了input pads数组，input links数组，以及input 数量。 5. AVFilterContext .output_pads，AVFilterContext.outputs ，AVFilterContext.nb_outputs：分别描述了output pads数组，output links数组，以及 output 数量。 6. AVFilterContext.thread_type，AVFilterContext.nb_threads：线程特性和可开启的最大线程数量。 7. AVFilterContext.is_disabled：filter 关于 timeline 特征的开启标志。 8. AVFilterContext.ready：表示 filter 是否被初始化，非 0 值表示 filter 需要被初始化，且值越大表示越急切。 filter context初始化的方法有三种：avfilter_init_str()和avfilter_init_dict()，以及通过AVOption API直接操作filter context。 注意初始化的 avfilter_init_str() 和 avfilter_init_dict() 在行为上的差别：当传入的 options 中有不被 filter 所支持的参数时，这两个函数的行为是不同，avfilter_init_str() 调用会失败，而 avfilter_init_dict() 函数则不会失败，它会将不能应用于指定 filter 的 option 通过参数返回，然后继续执行任务。
/*
* 使用提供的参数初始化 filter。
*
* 参数args：表示用于初始化 filter 的 options。该字符串必须使用 ":" 来分割各个键值对，
* 而键值对的形式为 'key=value'。如果不需要设置选项，args为空。
*
* 除了这种方式设置选项之外，还可以利用 AVOptions API 直接对 filter 设置选项。
*
* 返回值：成功返回0，失败返回一个负的错误值
*/
int avfilter_init_str(AVFilterContext *ctx, const char *args);
/*
* 使用提供的参数初始化filter。
*
* 参数 options：以 dict 形式提供的 options。
*
* 返回值：成功返回0，失败返回一个负的错误值
*
* 注意：这个函数和 avfilter_init_str 函数的功能是一样的，只不过传递的参数形式不同。
* 但是当传入的 options 中有不被 filter 所支持的参数时，这两个函数的行为是不同：
* avfilter_init_str 调用会失败，而这个函数则不会失败，它会将不能应用于指定 filter 的 option 通过参数 options 返回，然后继续执行任务。
*/
int avfilter_init_dict(AVFilterContext *ctx, AVDictionary **options);
/*
* 释放 filter context。
*/
void avfilter_free(AVFilterContext *filter);
三、流程梳理
3.1、FFmpeg AVFilter 使用整体流程
下图就是FFmpeg AVFilter在使用过程中的流程图：
![[图片]](https://s2.51cto.com/images/blog/202401/15181944_65a506c00e2cb44729.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=)
我们对上图先做下说明，理解下图中每个步骤的关系，然后，才从代码的角度来给出其使用的步骤。
1. 最顶端的AVFilterGraph，这个结构前面介绍过，主要管理加入的过滤器，其中加入的过滤器就是通过函数avfilter_graph_create_filter来创建并加入，这个函数返回是AVFilterContext（其封装了AVFilter的详细参数信息）。
2. buffer和buffersink这两个过滤器是FFMpeg为我们实现好的，buffer表示源，用来向后面的过滤器提供数据输入（其实就是原始的AVFrame）；buffersink过滤器是最终输出的（经过过滤器链处理后的数据AVFrame），其它的诸如filter 1 等过滤器是由avfilter_graph_parse_ptr函数解析外部传入的过滤器描述字符串自动生成的，内部也是通过avfilter_graph_create_filter来创建过滤器的。
3. 上面的buffer、filter 1、filter 2、filter n、buffersink之间是通过avfilter_link函数来进行关联的（通过AVFilterLink结构），这样子过滤器和过滤器之间就通过AVFilterLink进行关联上了，前一个过滤器的输出就是下一个过滤器的输入，注意，除了源和接收过滤器之外，其它的过滤器至少有一个输入和输出，这很好理解，中间的过滤器处理完AVFrame后，得到新的处理后的AVFrame数据，然后把新的AVFrame数据作为下一个过滤器的输入。
4. 过滤器建立完成后，首先我们通过av_buffersrc_add_frame把最原始的AVFrame（没有经过任何过滤器处理的）加入到buffer过滤器的fifo队列。
5. 然后调用buffersink过滤器的av_buffersink_get_frame_flags来获取处理完后的数据帧（这个最终放入buffersink过滤器的AVFrame是通过之前创建的一系列过滤器处理后的数据）。
6. 使用流程图就介绍到这里，下面结合上面的使用流程图详细说下FFMpeg中使用过滤器的步骤，这个过程我们分为三个部分：过滤器构建、数据加工、资源释放。
初始的流程大概可以总结为,创建filtergraph, 创建source filter 和sink filter, 将 filter连接起来添加到filtergraph, 配置filtergraph，可参考下图：
3.2、过滤器构建流程
3.2.1、分配AVFilterGraph
AVFilterGraph* graph = avfilter_graph_alloc();
3.2.2、创建过滤器源
char srcArgs[256] = {0};
AVFilterContext *srcFilterCtx;
AVFilter* srcFilter = avfilter_get_by_name("buffer");
avfilter_graph_create_filter(&srcFilterCtx, srcFilter ,"out_buffer", srcArgs, NULL, graph);
3.2.3、创建接收过滤器
AVFilterContext *sinkFilterCtx;
AVFilter* sinkFilter = avfilter_get_by_name("buffersink");
avfilter_graph_create_filter(&sinkFilterCtx, sinkFilter,"in_buffersink", NULL, NULL, graph);
3.2.4、生成源和接收过滤器的输入输出
这里主要是把源和接收过滤器封装给AVFilterInOut结构，使用这个中间结构来把过滤器字符串解析并链接进graph，主要代码如下：
AVFilterInOut *inputs = avfilter_inout_alloc();
AVFilterInOut *outputs = avfilter_inout_alloc();
outputs->name = av_strdup("in");
outputs->filter_ctx = srcFilterCtx;
outputs->pad_idx = 0;
outputs->next = NULL;
inputs->name = av_strdup("out");
inputs->filter_ctx = sinkFilterCtx;
inputs->pad_idx = 0;
inputs->next = NULL;
这里源对应的AVFilterInOut的name最好定义为in，接收对应的name为out，因为FFMpeg源码里默认会通过这样个name来对默认的输出和输入进行查找。
3.2.5、通过解析过滤器字符串添加过滤器
const *char filtergraph = "[in1]过滤器名称=参数1:参数2[out1]";
int ret = avfilter_graph_parse_ptr(graph, filtergraph, &inputs, &outputs, NULL);
3.2.6、检查过滤器的完整性
avfilter_graph_config(graph, NULL);
3.3、数据加工
3.3.1、向源过滤器加入AVFrame
AVFrame* frame; // 这是解码后获取的数据帧
int ret = av_buffersrc_add_frame(srcFilterCtx, frame);
3.3.2、从buffersink接收处理后的AVFrame
int ret = av_buffersink_get_frame_flags(sinkFilterCtx, frame, 0);
现在我们就可以使用处理后的AVFrame，比如显示或播放出来。
3.4、资源释放
使用结束后，调用avfilter_graph_free(&graph);释放掉AVFilterGraph类型的graph。
四、实例
extern "C" {
#include <libavcodec/avcodec.h>
#include <libavformat/avformat.h>
#include <libavfilter/avfiltergraph.h>
#include <libavfilter/buffersink.h>
#include <libavfilter/buffersrc.h>
#include <libavutil/opt.h>
#include <libavutil/imgutils.h>
}
int main(int argc, char* argv)
{
int ret = 0;
// input yuv
FILE* inFile = NULL;
const char* inFileName = "sintel_480x272_yuv420p.yuv";
fopen_s(&inFile, inFileName, "rb+");
if (!inFile) {
printf("Fail to open file\n");
return -1;
}
int in_width = 480;
int in_height = 272;
// output yuv
FILE* outFile = NULL;
const char* outFileName = "out_crop_vfilter.yuv";
fopen_s(&outFile, outFileName, "wb");
if (!outFile) {
printf("Fail to create file for output\n");
return -1;
}
avfilter_register_all();
AVFilterGraph* filter_graph = avfilter_graph_alloc();
if (!filter_graph) {
printf("Fail to create filter graph!\n");
return -1;
}
// source filter
char args[512];
_snprintf_s(args, sizeof(args),
"video_size=%dx%d:pix_fmt=%d:time_base=%d/%d:pixel_aspect=%d/%d",
in_width, in_height, AV_PIX_FMT_YUV420P,
1, 25, 1, 1);
AVFilter* bufferSrc = avfilter_get_by_name("buffer");
AVFilterContext* bufferSrc_ctx;
ret = avfilter_graph_create_filter(&bufferSrc_ctx, bufferSrc, "in", args, NULL, filter_graph);
if (ret < 0) {
printf("Fail to create filter bufferSrc\n");
return -1;
}
// sink filter
AVBufferSinkParams *bufferSink_params;
AVFilterContext* bufferSink_ctx;
AVFilter* bufferSink = avfilter_get_by_name("buffersink");
enum AVPixelFormat pix_fmts[] = { AV_PIX_FMT_YUV420P, AV_PIX_FMT_NONE };
bufferSink_params = av_buffersink_params_alloc();
bufferSink_params->pixel_fmts = pix_fmts;
ret = avfilter_graph_create_filter(&bufferSink_ctx, bufferSink, "out", NULL, bufferSink_params, filter_graph);
if (ret < 0) {
printf("Fail to create filter sink filter\n");
return -1;
}
// split filter
AVFilter *splitFilter = avfilter_get_by_name("split");
AVFilterContext *splitFilter_ctx;
ret = avfilter_graph_create_filter(&splitFilter_ctx, splitFilter, "split", "outputs=2", NULL, filter_graph);
if (ret < 0) {
printf("Fail to create split filter\n");
return -1;
}
// crop filter
AVFilter *cropFilter = avfilter_get_by_name("crop");
AVFilterContext *cropFilter_ctx;
ret = avfilter_graph_create_filter(&cropFilter_ctx, cropFilter, "crop", "out_w=iw:out_h=ih/2:x=0:y=0", NULL, filter_graph);
if (ret < 0) {
printf("Fail to create crop filter\n");
return -1;
}
// vflip filter
AVFilter *vflipFilter = avfilter_get_by_name("vflip");
AVFilterContext *vflipFilter_ctx;
ret = avfilter_graph_create_filter(&vflipFilter_ctx, vflipFilter, "vflip", NULL, NULL, filter_graph);
if (ret < 0) {
printf("Fail to create vflip filter\n");
return -1;
}
// overlay filter
AVFilter *overlayFilter = avfilter_get_by_name("overlay");
AVFilterContext *overlayFilter_ctx;
ret = avfilter_graph_create_filter(&overlayFilter_ctx, overlayFilter, "overlay", "y=0:H/2", NULL, filter_graph);
if (ret < 0) {
printf("Fail to create overlay filter\n");
return -1;
}
// src filter to split filter
ret = avfilter_link(bufferSrc_ctx, 0, splitFilter_ctx, 0);
if (ret != 0) {
printf("Fail to link src filter and split filter\n");
return -1;
}
// split filter's first pad to overlay filter's main pad
ret = avfilter_link(splitFilter_ctx, 0, overlayFilter_ctx, 0);
if (ret != 0) {
printf("Fail to link split filter and overlay filter main pad\n");
return -1;
}
// split filter's second pad to crop filter
ret = avfilter_link(splitFilter_ctx, 1, cropFilter_ctx, 0);
if (ret != 0) {
printf("Fail to link split filter's second pad and crop filter\n");
return -1;
}
// crop filter to vflip filter
ret = avfilter_link(cropFilter_ctx, 0, vflipFilter_ctx, 0);
if (ret != 0) {
printf("Fail to link crop filter and vflip filter\n");
return -1;
}
// vflip filter to overlay filter's second pad
ret = avfilter_link(vflipFilter_ctx, 0, overlayFilter_ctx, 1);
if (ret != 0) {
printf("Fail to link vflip filter and overlay filter's second pad\n");
return -1;
}
// overlay filter to sink filter
ret = avfilter_link(overlayFilter_ctx, 0, bufferSink_ctx, 0);
if (ret != 0) {
printf("Fail to link overlay filter and sink filter\n");
return -1;
}
// check filter graph
ret = avfilter_graph_config(filter_graph, NULL);
if (ret < 0) {
printf("Fail in filter graph\n");
return -1;
}
char *graph_str = avfilter_graph_dump(filter_graph, NULL);
FILE* graphFile = NULL;
fopen_s(&graphFile, "graphFile.txt", "w");
fprintf(graphFile, "%s", graph_str);
av_free(graph_str);
AVFrame *frame_in = av_frame_alloc();
unsigned char *frame_buffer_in = (unsigned char *)av_malloc(av_image_get_buffer_size(AV_PIX_FMT_YUV420P, in_width, in_height, 1));
av_image_fill_arrays(frame_in->data, frame_in->linesize, frame_buffer_in,
AV_PIX_FMT_YUV420P, in_width, in_height, 1);
AVFrame *frame_out = av_frame_alloc();
unsigned char *frame_buffer_out = (unsigned char *)av_malloc(av_image_get_buffer_size(AV_PIX_FMT_YUV420P, in_width, in_height, 1));
av_image_fill_arrays(frame_out->data, frame_out->linesize, frame_buffer_out,
AV_PIX_FMT_YUV420P, in_width, in_height, 1);
frame_in->width = in_width;
frame_in->height = in_height;
frame_in->format = AV_PIX_FMT_YUV420P;
while (1) {
if (fread(frame_buffer_in, 1, in_width*in_height * 3 / 2, inFile) != in_width*in_height * 3 / 2) {
break;
}
//input Y,U,V
frame_in->data[0] = frame_buffer_in;
frame_in->data[1] = frame_buffer_in + in_width*in_height;
frame_in->data[2] = frame_buffer_in + in_width*in_height * 5 / 4;
if (av_buffersrc_add_frame(bufferSrc_ctx, frame_in) < 0) {
printf("Error while add frame.\n");
break;
}
/* pull filtered pictures from the filtergraph */
ret = av_buffersink_get_frame(bufferSink_ctx, frame_out);
if (ret < 0)
break;
//output Y,U,V
if (frame_out->format == AV_PIX_FMT_YUV420P) {
for (int i = 0; i < frame_out->height; i++) {
fwrite(frame_out->data[0] + frame_out->linesize[0] * i, 1, frame_out->width, outFile);
}
for (int i = 0; i < frame_out->height / 2; i++) {
fwrite(frame_out->data[1] + frame_out->linesize[1] * i, 1, frame_out->width / 2, outFile);
}
for (int i = 0; i < frame_out->height / 2; i++) {
fwrite(frame_out->data[2] + frame_out->linesize[2] * i, 1, frame_out->width / 2, outFile);
}
}
printf("Process 1 frame!\n");
av_frame_unref(frame_out);
}
fclose(inFile);
fclose(outFile);
av_frame_free(&frame_in);
av_frame_free(&frame_out);
avfilter_graph_free(&filter_graph);
return 0;
}
