【WEB 系列】RestTemplate 之代理访问
通过代理访问,对于 java 后端可能用得不多的,但有过爬虫开发经验的小伙伴可能一点也不会陌生,有时候不太方便直接去访问目标资源,借助代理是要给选择,对于 RestTemplate 而言,使用代理的姿势同样如设置超时一般,借助SimpleClientHttpRequestFactory
来实现,本文演示一下具体的使用 case
I. 环境准备
1. 项目环境
借助 SpringBoot 搭建一个 SpringWEB 项目,提供一些用于测试的 REST 服务
- SpringBoot 版本:
2.2.1.RELEASE
- 核心依赖:
spring-boot-stater-web
<dependencies>
<dependency>
<groupId>org.springframework.bootgroupId>
<artifactId>spring-boot-starter-webartifactId>
dependency>
dependencies>
<dependencies>
<dependency>
<groupId>org.springframework.bootgroupId>
<artifactId>spring-boot-starter-webartifactId>
dependency>
dependencies>
为了后续输出的日志更直观,这里设置了一下日志输出格式,在配置文件application.yml
中,添加
logging:
pattern:
console: (%msg%n%n){blue}
logging:
pattern:
console: (%msg%n%n){blue}
2. 测试端点
我们的测试端点,主要需要返回客户端主机信息,我们这里直接借助HttpServletRequest#getRemoteHost
+ HttpServlet#getRemotePort
来实现(当然实际的业务开发中不建议直接使用它)
@RestController
public class DemoRest {
private String getHeaders(HttpServletRequest request) {
Enumeration headerNames = request.getHeaderNames();
String name;
JSONObject headers = new JSONObject();while (headerNames.hasMoreElements()) {
name = headerNames.nextElement();
headers.put(name, request.getHeader(name));
}return headers.toJSONString();
}private String getParams(HttpServletRequest request) {return JSONObject.toJSONString(request.getParameterMap());
}private String getCookies(HttpServletRequest request) {
Cookie[] cookies = request.getCookies();if (cookies == null || cookies.length == 0) {return "";
}
JSONObject ck = new JSONObject();for (Cookie cookie : cookies) {
ck.put(cookie.getName(), cookie.getValue());
}return ck.toJSONString();
}private String buildResult(HttpServletRequest request) {return buildResult(request, null);
}private String buildResult(HttpServletRequest request, Object obj) {
String params = getParams(request);
String headers = getHeaders(request);
String cookies = getCookies(request);if (obj != null) {
params += " | " + obj;
}return "params: " + params + "\nheaders: " + headers + "\ncookies: " + cookies;
}@GetMapping(path = "proxy")private String proxy(HttpServletRequest request) {
String remote = request.getRemoteHost() + ":" + request.getRemotePort();return buildResult(request) + "\n>>>remote ipInfo: " + remote;
}
}
@RestController
public class DemoRest {
private String getHeaders(HttpServletRequest request) {
Enumeration headerNames = request.getHeaderNames();
String name;
JSONObject headers = new JSONObject();while (headerNames.hasMoreElements()) {
name = headerNames.nextElement();
headers.put(name, request.getHeader(name));
}return headers.toJSONString();
}private String getParams(HttpServletRequest request) {return JSONObject.toJSONString(request.getParameterMap());
}private String getCookies(HttpServletRequest request) {
Cookie[] cookies = request.getCookies();if (cookies == null || cookies.length == 0) {return "";
}
JSONObject ck = new JSONObject();for (Cookie cookie : cookies) {
ck.put(cookie.getName(), cookie.getValue());
}return ck.toJSONString();
}private String buildResult(HttpServletRequest request) {return buildResult(request, null);
}private String buildResult(HttpServletRequest request, Object obj) {
String params = getParams(request);
String headers = getHeaders(request);
String cookies = getCookies(request);if (obj != null) {
params += " | " + obj;
}return "params: " + params + "\nheaders: " + headers + "\ncookies: " + cookies;
}@GetMapping(path = "proxy")private String proxy(HttpServletRequest request) {
String remote = request.getRemoteHost() + ":" + request.getRemotePort();return buildResult(request) + "\n>>>remote ipInfo: " + remote;
}
}
3. 代理服务器搭建
我们这里借助 tinyproxy 来搭建代理服务器,详细步骤可以参考博文: http 代理服务器 tinyproxy 搭建手册 (https://blog.hhui.top/hexblog/2020/06/19/200619-http%E4%BB%A3%E7%90%86%E6%9C%8D%E5%8A%A1%E5%99%A8tinyproxy%E6%90%AD%E5%BB%BA%E6%89%8B%E5%86%8C/)
本文的演示中,是在192.168.0.241
状态 centos 机器上安装的,步骤如下
1. sudo yum install tinyproxy -y
# 设置配置
2. vim /etc/tinyproxy/tinyproxy.conf
# 下面这个ip是我测试用例的机器ip
Allow 192.168.0.174
3. 启动服务
systemctl start tinyproxy.service
1. sudo yum install tinyproxy -y
# 设置配置
2. vim /etc/tinyproxy/tinyproxy.conf
# 下面这个ip是我测试用例的机器ip
Allow 192.168.0.174
3. 启动服务
systemctl start tinyproxy.service
II. 代理访问
接下来进入正文演示,核心代码也比较简单
/**
* 代理访问
*/
public void proxy() {
RestTemplate restTemplate = new RestTemplate();
SimpleClientHttpRequestFactory requestFactory = new SimpleClientHttpRequestFactory();
// 请注意,我这里是在241机器上,借助tinyproxy搭建了一个http的代理,并设置端口为18888,所以可以正常演示代理访问
// 拉源码运行的小伙,需要注意使用自己的代理来替换
requestFactory.setProxy(new Proxy(Proxy.Type.HTTP, new InetSocketAddress("192.168.0.241", 18888)));
restTemplate.setRequestFactory(requestFactory);
// 因为使用代理访问,所以这个ip就不能是127.0.0.1,不然访问的就是代理服务器上了
HttpEntity ans =
restTemplate.getForEntity("http://192.168.0.174:8080/proxy?name=一灰灰&age=20", String.class);
log.info("proxy request ans: {}", ans.getBody());
}
/**
* 代理访问
*/
public void proxy() {
RestTemplate restTemplate = new RestTemplate();
SimpleClientHttpRequestFactory requestFactory = new SimpleClientHttpRequestFactory();
// 请注意,我这里是在241机器上,借助tinyproxy搭建了一个http的代理,并设置端口为18888,所以可以正常演示代理访问
// 拉源码运行的小伙,需要注意使用自己的代理来替换
requestFactory.setProxy(new Proxy(Proxy.Type.HTTP, new InetSocketAddress("192.168.0.241", 18888)));
restTemplate.setRequestFactory(requestFactory);
// 因为使用代理访问,所以这个ip就不能是127.0.0.1,不然访问的就是代理服务器上了
HttpEntity ans =
restTemplate.getForEntity("http://192.168.0.174:8080/proxy?name=一灰灰&age=20", String.class);
log.info("proxy request ans: {}", ans.getBody());
}
请注意,上面的使用姿势中
- Proxy 的方式除了 HTTP 之外还有 SOCKS,这个是与代理服务器的支持方式相关的
-
postForEntity
中 url 的 ip 是我本机的 ip,而不是127.0.0.1
测试输出如下:
(proxy request ans: <200,params: {"name":["一灰灰"],"age":["20"]}
headers: {"host":"192.168.0.174:8080","connection":"close","via":"1.1 tinyproxy (tinyproxy/1.8.3)","accept":"text/plain, application/json, application/*+json, */*","user-agent":"Java/1.8.0_171"}
cookies:
>>>remote ipInfo: 192.168.0.241:56122,[Via:"1.1 tinyproxy (tinyproxy/1.8.3)", Content-Type:"text/plain;charset=UTF-8", Date:"Mon, 29 Jun 2020 08:46:47 GMT", Content-Length:"286"]>
(proxy request ans: <200,params: {"name":["一灰灰"],"age":["20"]}
headers: {"host":"192.168.0.174:8080","connection":"close","via":"1.1 tinyproxy (tinyproxy/1.8.3)","accept":"text/plain, application/json, application/*+json, */*","user-agent":"Java/1.8.0_171"}
cookies:
>>>remote ipInfo: 192.168.0.241:56122,[Via:"1.1 tinyproxy (tinyproxy/1.8.3)", Content-Type:"text/plain;charset=UTF-8", Date:"Mon, 29 Jun 2020 08:46:47 GMT", Content-Length:"286"]>
II. 其他
0. 项目&系列博文
博文
- 【WEB 系列】RestTemplate 之超时设置
- 【WEB 系列】RestTemplate 之中文乱码问题 fix
- 【WEB 系列】RestTemplate 之自定义请求头
- 【WEB 系列】RestTemplate 基础用法小结
源码
- 工程:https://github.com/liuyueyi/spring-boot-demo
- 源码: https://github.com/liuyueyi/spring-boot-demo/tree/master/spring-boot/221-web-resttemplate
1. 一灰灰 Blog
尽信书则不如,以上内容,纯属一家之言,因个人能力有限,难免有疏漏和错误之处,如发现 bug 或者有更好的建议,欢迎批评指正,不吝感激
一灰灰blog