最近在和一个第三方的合作中不得已需要使用FTP文件接口。由于FTP Server由对方提供,而且双方背后各自的网络环境都很不单纯等等原因,造成测试环境无法模拟实际情况。测试环境中程序一切正常,但是在部署到生产环境之后发现FTP操作不规律性出现“卡死”现象:程序捕获不到任何异常一直卡着,导致轮巡无法正常工作(由于担心在轮巡时间间隔内处理不能完成,我没有采用类似quartz或者crontab的定时任务,而是采用while-true然后sleep的方式)。

    为了解决这个问题,我首先考虑的是对于FTPClient的使用上没有设置超时时间,于是设置了ConnectTimeout、DataTimeout、DefaultTimeout后在生产环境上继续观察,但是问题依旧没有解决。后来我有些怀疑FTPClient api本身是不是有什么问题,想实在不行自己实现一个超时机制吧,不过还是不甘心,还是想从FTPClient api本身去解决问题。又经过一翻研究之后发现:需要使用被动模式,以下摘抄别人的一段简单描述:

在项目中使用commons-net-3.0.1.jar实现FTP文件的下载,在windows xp上运行正常,但是放到linux上,却出现问题,程序运行到 FTPClient.listFiles()或者FTPClient.retrieveFile()方法时,就停止在那里,什么反应都没有,出现假死状态。google一把,发现很多人也出现了此类问题,最终在一个帖子里找到了解决办法。在调用这两个方法之前,调用FTPClient.enterLocalPassiveMode();这个方法的意思就是每次数据连接之前,ftp client告诉ftp server开通一个端口来传输数据。为什么要这样做呢,因为ftp server可能每次开启不同的端口来传输数据,但是在linux上,由于安全限制,可能某些端口没有开启,所以就出现阻塞。OK,问题解决。

于是我回滚了之前的修改,改为被动模式(关于FTP主动/被动模式的解释,这里我不多说了,关注的朋友可以自己查阅)。但是问题依旧。于是能想到的就是最有的绝招:实在不行自己实现一个超时机制吧。经过一翻研究最简单的方式就是使用:Future解决:

转载:Apache FTPClient操作“卡死”问题的分析和解决_.net

1 public static void main(String[] args) throws InterruptedException, ExecutionException {
2 final ExecutorService exec = Executors.newFixedThreadPool(1);
3
4 Callable<String> call = new Callable<String>() {
5 public String call() throws Exception {
6 Thread.sleep(1000 * 5);
7 return "线程执行完成.";
8 }
9 };
10
11 try {
12 Future<String> future = exec.submit(call);
13 String obj = future.get(4 * 1000, TimeUnit.MILLISECONDS); // 任务处理超时时间设置
14 System.out.println("任务成功返回:" + obj);
15 } catch (TimeoutException ex) {
16 System.out.println("处理超时啦....");
17 ex.printStackTrace();
18 } catch (Exception e) {
19 System.out.println("处理失败.");
20 e.printStackTrace();
21 }
22 // 关闭线程池
23 exec.shutdown();
24
25 System.out.println("完毕");
26 }

转载:Apache FTPClient操作“卡死”问题的分析和解决_.net

当然了还有很多其他方式:

http://tech.sina.com.cn/s/2008-07-04/1051720260.shtml

http://itindex.net/blog/2010/08/11/1281486125717.html

http://darkmasky.iteye.com/blog/1115047

http://coolxing.iteye.com/blog/1476289

虽然找到了终极的“必杀技”,但是此时我还是不甘心,还是想从FTPClient api本身去解决问题,但此时看来也别无它他法。只能试试:即设置被动模式又设置超时时间。经过实际测试,发现问题得以解决。下面把我的FTP工具类贴给大家分享,希望能帮到遇到同样问题的人。

转载:Apache FTPClient操作“卡死”问题的分析和解决_.net

1 import org.apache.commons.net.ftp.FTP;
2 import org.apache.commons.net.ftp.FTPClient;
3 import org.apache.commons.net.ftp.FTPFile;
4 import org.apache.commons.net.ftp.FTPReply;
5
6 import java.io.BufferedInputStream;
7 import java.io.BufferedOutputStream;
8 import java.io.File;
9 import java.io.FileInputStream;
10 import java.io.FileNotFoundException;
11 import java.io.FileOutputStream;
12 import java.io.IOException;
13 import java.io.InputStream;
14 import java.io.OutputStream;
15 import java.net.UnknownHostException;
16 import java.util.ArrayList;
17 import java.util.List;
18
19 public class FtpUtil {
20 public static final String ANONYMOUS_LOGIN = "anonymous";
21 private FTPClient ftp;
22 private boolean is_connected;
23
24 public FtpUtil() {
25 ftp = new FTPClient();
26 is_connected = false;
27 }
28
29 public FtpUtil(int defaultTimeoutSecond, int connectTimeoutSecond, int dataTimeoutSecond){
30 ftp = new FTPClient();
31 is_connected = false;
32
33 ftp.setDefaultTimeout(defaultTimeoutSecond * 1000);
34 ftp.setConnectTimeout(connectTimeoutSecond * 1000);
35 ftp.setDataTimeout(dataTimeoutSecond * 1000);
36 }
37
38 /**
39 * Connects to FTP server.
40 *
41 * @param host
42 * FTP server address or name
43 * @param port
44 * FTP server port
45 * @param user
46 * user name
47 * @param password
48 * user password
49 * @param isTextMode
50 * text / binary mode switch
51 * @throws IOException
52 * on I/O errors
53 */
54 public void connect(String host, int port, String user, String password, boolean isTextMode) throws IOException {
55 // Connect to server.
56 try {
57 ftp.connect(host, port);
58 } catch (UnknownHostException ex) {
59 throw new IOException("Can't find FTP server '" + host + "'");
60 }
61
62 // Check rsponse after connection attempt.
63 int reply = ftp.getReplyCode();
64 if (!FTPReply.isPositiveCompletion(reply)) {
65 disconnect();
66 throw new IOException("Can't connect to server '" + host + "'");
67 }
68
69 if (user == "") {
70 user = ANONYMOUS_LOGIN;
71 }
72
73 // Login.
74 if (!ftp.login(user, password)) {
75 is_connected = false;
76 disconnect();
77 throw new IOException("Can't login to server '" + host + "'");
78 } else {
79 is_connected = true;
80 }
81
82 // Set data transfer mode.
83 if (isTextMode) {
84 ftp.setFileType(FTP.ASCII_FILE_TYPE);
85 } else {
86 ftp.setFileType(FTP.BINARY_FILE_TYPE);
87 }
88 }
89
90 /**
91 * Uploads the file to the FTP server.
92 *
93 * @param ftpFileName
94 * server file name (with absolute path)
95 * @param localFile
96 * local file to upload
97 * @throws IOException
98 * on I/O errors
99 */
100 public void upload(String ftpFileName, File localFile) throws IOException {
101 // File check.
102 if (!localFile.exists()) {
103 throw new IOException("Can't upload '" + localFile.getAbsolutePath() + "'. This file doesn't exist.");
104 }
105
106 // Upload.
107 InputStream in = null;
108 try {
109
110 // Use passive mode to pass firewalls.
111 ftp.enterLocalPassiveMode();
112
113 in = new BufferedInputStream(new FileInputStream(localFile));
114 if (!ftp.storeFile(ftpFileName, in)) {
115 throw new IOException("Can't upload file '" + ftpFileName + "' to FTP server. Check FTP permissions and path.");
116 }
117
118 } finally {
119 try {
120 in.close();
121 } catch (IOException ex) {
122 }
123 }
124 }
125
126 /**
127 * Downloads the file from the FTP server.
128 *
129 * @param ftpFileName
130 * server file name (with absolute path)
131 * @param localFile
132 * local file to download into
133 * @throws IOException
134 * on I/O errors
135 */
136 public void download(String ftpFileName, File localFile) throws IOException {
137 // Download.
138 OutputStream out = null;
139 try {
140 // Use passive mode to pass firewalls.
141 ftp.enterLocalPassiveMode();
142
143 // Get file info.
144 FTPFile[] fileInfoArray = ftp.listFiles(ftpFileName);
145 if (fileInfoArray == null) {
146 throw new FileNotFoundException("File " + ftpFileName + " was not found on FTP server.");
147 }
148
149 // Check file size.
150 FTPFile fileInfo = fileInfoArray[0];
151 long size = fileInfo.getSize();
152 if (size > Integer.MAX_VALUE) {
153 throw new IOException("File " + ftpFileName + " is too large.");
154 }
155
156 // Download file.
157 out = new BufferedOutputStream(new FileOutputStream(localFile));
158 if (!ftp.retrieveFile(ftpFileName, out)) {
159 throw new IOException("Error loading file " + ftpFileName + " from FTP server. Check FTP permissions and path.");
160 }
161
162 out.flush();
163 } finally {
164 if (out != null) {
165 try {
166 out.close();
167 } catch (IOException ex) {
168 }
169 }
170 }
171 }
172
173 /**
174 * Removes the file from the FTP server.
175 *
176 * @param ftpFileName
177 * server file name (with absolute path)
178 * @throws IOException
179 * on I/O errors
180 */
181 public void remove(String ftpFileName) throws IOException {
182 if (!ftp.deleteFile(ftpFileName)) {
183 throw new IOException("Can't remove file '" + ftpFileName + "' from FTP server.");
184 }
185 }
186
187 /**
188 * Lists the files in the given FTP directory.
189 *
190 * @param filePath
191 * absolute path on the server
192 * @return files relative names list
193 * @throws IOException
194 * on I/O errors
195 */
196 public List<String> list(String filePath) throws IOException {
197 List<String> fileList = new ArrayList<String>();
198
199 // Use passive mode to pass firewalls.
200 ftp.enterLocalPassiveMode();
201
202 FTPFile[] ftpFiles = ftp.listFiles(filePath);
203 int size = (ftpFiles == null) ? 0 : ftpFiles.length;
204 for (int i = 0; i < size; i++) {
205 FTPFile ftpFile = ftpFiles[i];
206 if (ftpFile.isFile()) {
207 fileList.add(ftpFile.getName());
208 }
209 }
210
211 return fileList;
212 }
213
214 /**
215 * Sends an FTP Server site specific command
216 *
217 * @param args
218 * site command arguments
219 * @throws IOException
220 * on I/O errors
221 */
222 public void sendSiteCommand(String args) throws IOException {
223 if (ftp.isConnected()) {
224 try {
225 ftp.sendSiteCommand(args);
226 } catch (IOException ex) {
227 }
228 }
229 }
230
231 /**
232 * Disconnects from the FTP server
233 *
234 * @throws IOException
235 * on I/O errors
236 */
237 public void disconnect() throws IOException {
238
239 if (ftp.isConnected()) {
240 try {
241 ftp.logout();
242 ftp.disconnect();
243 is_connected = false;
244 } catch (IOException ex) {
245 }
246 }
247 }
248
249 /**
250 * Makes the full name of the file on the FTP server by joining its path and
251 * the local file name.
252 *
253 * @param ftpPath
254 * file path on the server
255 * @param localFile
256 * local file
257 * @return full name of the file on the FTP server
258 */
259 public String makeFTPFileName(String ftpPath, File localFile) {
260 if (ftpPath == "") {
261 return localFile.getName();
262 } else {
263 String path = ftpPath.trim();
264 if (path.charAt(path.length() - 1) != '/') {
265 path = path + "/";
266 }
267
268 return path + localFile.getName();
269 }
270 }
271
272 /**
273 * Test coonection to ftp server
274 *
275 * @return true, if connected
276 */
277 public boolean isConnected() {
278 return is_connected;
279 }
280
281 /**
282 * Get current directory on ftp server
283 *
284 * @return current directory
285 */
286 public String getWorkingDirectory() {
287 if (!is_connected) {
288 return "";
289 }
290
291 try {
292 return ftp.printWorkingDirectory();
293 } catch (IOException e) {
294 }
295
296 return "";
297 }
298
299 /**
300 * Set working directory on ftp server
301 *
302 * @param dir
303 * new working directory
304 * @return true, if working directory changed
305 */
306 public boolean setWorkingDirectory(String dir) {
307 if (!is_connected) {
308 return false;
309 }
310
311 try {
312 return ftp.changeWorkingDirectory(dir);
313 } catch (IOException e) {
314 }
315
316 return false;
317 }
318
319 /**
320 * Change working directory on ftp server to parent directory
321 *
322 * @return true, if working directory changed
323 */
324 public boolean setParentDirectory() {
325 if (!is_connected) {
326 return false;
327 }
328
329 try {
330 return ftp.changeToParentDirectory();
331 } catch (IOException e) {
332 }
333
334 return false;
335 }
336
337 /**
338 * Get parent directory name on ftp server
339 *
340 * @return parent directory
341 */
342 public String getParentDirectory() {
343 if (!is_connected) {
344 return "";
345 }
346
347 String w = getWorkingDirectory();
348 setParentDirectory();
349 String p = getWorkingDirectory();
350 setWorkingDirectory(w);
351
352 return p;
353 }
354
355 /**
356 * Get directory contents on ftp server
357 *
358 * @param filePath
359 * directory
360 * @return list of FTPFileInfo structures
361 * @throws IOException
362 */
363 public List<FfpFileInfo> listFiles(String filePath) throws IOException {
364 List<FfpFileInfo> fileList = new ArrayList<FfpFileInfo>();
365
366 // Use passive mode to pass firewalls.
367 ftp.enterLocalPassiveMode();
368 FTPFile[] ftpFiles = ftp.listFiles(filePath);
369 int size = (ftpFiles == null) ? 0 : ftpFiles.length;
370 for (int i = 0; i < size; i++) {
371 FTPFile ftpFile = ftpFiles[i];
372 FfpFileInfo fi = new FfpFileInfo();
373 fi.setName(ftpFile.getName());
374 fi.setSize(ftpFile.getSize());
375 fi.setTimestamp(ftpFile.getTimestamp());
376 fi.setType(ftpFile.isDirectory());
377 fileList.add(fi);
378 }
379
380 return fileList;
381 }
382
383 /**
384 * Get file from ftp server into given output stream
385 *
386 * @param ftpFileName
387 * file name on ftp server
388 * @param out
389 * OutputStream
390 * @throws IOException
391 */
392 public void getFile(String ftpFileName, OutputStream out) throws IOException {
393 try {
394 // Use passive mode to pass firewalls.
395 ftp.enterLocalPassiveMode();
396
397 // Get file info.
398 FTPFile[] fileInfoArray = ftp.listFiles(ftpFileName);
399 if (fileInfoArray == null) {
400 throw new FileNotFoundException("File '" + ftpFileName + "' was not found on FTP server.");
401 }
402
403 // Check file size.
404 FTPFile fileInfo = fileInfoArray[0];
405 long size = fileInfo.getSize();
406 if (size > Integer.MAX_VALUE) {
407 throw new IOException("File '" + ftpFileName + "' is too large.");
408 }
409
410 // Download file.
411 if (!ftp.retrieveFile(ftpFileName, out)) {
412 throw new IOException("Error loading file '" + ftpFileName + "' from FTP server. Check FTP permissions and path.");
413 }
414
415 out.flush();
416
417 } finally {
418 if (out != null) {
419 try {
420 out.close();
421 } catch (IOException ex) {
422 }
423 }
424 }
425 }
426
427 /**
428 * Put file on ftp server from given input stream
429 *
430 * @param ftpFileName
431 * file name on ftp server
432 * @param in
433 * InputStream
434 * @throws IOException
435 */
436 public void putFile(String ftpFileName, InputStream in) throws IOException {
437 try {
438 // Use passive mode to pass firewalls.
439 ftp.enterLocalPassiveMode();
440
441 if (!ftp.storeFile(ftpFileName, in)) {
442 throw new IOException("Can't upload file '" + ftpFileName + "' to FTP server. Check FTP permissions and path.");
443 }
444 } finally {
445 try {
446 in.close();
447 } catch (IOException ex) {
448 }
449 }
450 }
451 }

转载:Apache FTPClient操作“卡死”问题的分析和解决_.net