libcurl 使用的几个注意事项

 

0. 为使用的curl url 加入确定的协议头

原文：
If you specify URL without protocol:// prefix, curl will attempt to guess what protocol you might want. It will then default to HTTP but try other protocols based on often-used host name prefixes. For example, for host names starting with "ftp." curl will assume you want to speak FTP.

1. 把 curl_easy_perform() 回调数据直接写到文件里(FILE *)

原文：
libcurl offers its own default internal callback that will take care of the data if you don't set the callback with CURLOPT_WRITEFUNCTION. It will then simply output the received data to stdout. You can have the default callback write the data to a different file handle by passing a 'FILE *' to a file opened for writing with the CURLOPT_WRITEDATA option.
源代码中的实现：

 

这样，就能够少写一个回调函数了(喂，你是有多懒啊)，示比例如以下

FILE *fp;
fp = fopen("/root/test.bmp", "wb");
...
curl_easy_setopt(curl, CURLOPT_WRITEDATA, fp);
...
fclose(fp);

 

2. curl_easy_perform 返回值处理

使用 CURLOPT_ERRORBUFFER 保存错误, buf_size=CURL_ERROR_SIZE
或使用 curl_easy_strerror(res) (感觉这个简便)
演示样例：

 /* Perform the request, res will get the return code */ 
 res = curl_easy_perform(curl);
 /* Check for errors */ 
 if(res != CURLE_OK)
 {
   printf("%s curl_easy_perform() error! \n", __FUNCTION__);
   printf("error msg = %s\n",  curl_easy_strerror(res));
  curl_easy_cleanup(curl);
   return -1;
 }

 

3. 多线程环境配置 CURLOPT_NOSIGNAL

原文：
When using multiple threads you should set the CURLOPT_NOSIGNAL option to 1 for all handles. Everything will or might work fine except that timeouts are not honored during the DNS lookup - which you can work around by building libcurl with c-ares support. c-ares is a library that provides asynchronous name resolves. On some platforms, libcurl simply will not function properly multi-threaded unless this option is set.
对于 CURLOPT_TIMEOUT(默认0), CURLOPT_CONNECTTIMEOUT(默认300)选项:
In unix-like systems, this might cause signals to be used unless CURLOPT_NOSIGNAL is set.

4. 设置 CURLOPT_VERBOSE、CURLOPT_HEADER 的必要性

原文：
There's one golden rule when these things occur: set the CURLOPT_VERBOSE option to 1. It'll cause the library to spew out the entire protocol details it sends, some internal info and some received protocol data as well (especially when using FTP). If you're using HTTP, adding the headers in the received output to study is also a clever way to get a better understanding why the server behaves the way it does. Include headers in the normal body output with CURLOPT_HEADER set 1.
经试验：
设置 curl_easy_setopt(curl, CURLOPT_HEADER, 1L) 后，回调函数会返回 http头相关信息(原本是直接输出到stdout的)。考虑到还要过滤这些信息，所以还是不要设置这个了

5. curl post 注意事项

原文：
Using POST with HTTP 1.1 implies the use of a "Expect: 100-continue" header. You can disable this header with CURLOPT_HTTPHEADER as usual.
解释：
当使用libcurl的POST方式时，假设POST数据的大小大于1024个字节，libcurl不会直接发送POST请求，而是会分为两步运行请求：
<1> 发送一个请求，该请求头部包括一个Expect： 100-continue的字段，用来询问server是否愿意接受数据
<2> 当接收到从server返回的100-continue的应答后，它才会真正的发起POST请求。将数据发送给server。

对于“100-continue"这个字段，RFC文档（http://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html#sec8.2.3）是这么解释的：
它能够让client在发送请求数据之前去推断server是否愿意接收该数据。假设server愿意接收，client才会真正发送数据，
这么做的原因是假设client直接发送请求数据，可是server又将该请求拒绝的话。这样的行为将带来非常大的资源开销。

所以为了避免这样的情况，libcurl在发送大于1024字节的POST请求时採用了这样的方法，可是相对的，它会引起请求延迟的加大，
另外并非全部的server都会正确处理而且应答”100-continue“，比方lighttpd，就会返回417”Expectation Failed“，造成请求逻辑出错。
解决的方法:

// POST数据的大于1024个字节
struct curl_slist *headerlist = NULL;
static const char buf[] = "Expect:";
headerlist = curl_slist_append(headerlist, buf); /* initalize custom header list */
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headerlist); /* set header*/
curl_slist_free_all(headerlist); /* free slist */

 

6. 回调函数的正确返回

return (size * nmemb);

原因:
Your callback function should return the number of bytes it "took care of". If that is not the exact same amount of bytes that was passed to it, libcurl will abort the operation and return with an error code.

假设回调函数中接收的数据有误，个人感觉能够返回0或者返回你已经处理的数据数。
由于源代码的处理例如以下:

 /* If the previous block of data ended with CR and this block of data is
    just a NL, then the length might be zero */
 // len 为要发送给回调函数的数据长度
 if(len) {
   wrote = data->set.fwrite_func(ptr, 1, len, data->set.out);
 }
 else {
   wrote = len;
 }

if(wrote != len) {
  failf(data, "Failed writing body (%zu != %zu)", wrote, len);
  return CURLE_WRITE_ERROR;
}