create integrated mesh scene layer package 大文件 files.lines() 大文件有无问题

转载

mob6454cc786d85 2024-05-25 18:04:01

Java8的标准库中增加了一个新的方法：

public final class Files
{
    public static Stream<String> lines(Path path) throws IOException
}

这个方法很简单，从path对应的文件中读取所有内容，并按行分割，返回一个 Stream<String>。

使用这个方法可以很容易的从一个文件中读取连续的某几行内容。如：

try{
    return Files.lines(Paths.get(file)).skip(start).limit(limit).collect(Collectors.toList());
}catch(IOExceptione){
    logger.error("get content from {} error,{}", file, e.getMessage());
}

我在系统中用上面这段代码从/proc/stat中定时读取系统状态。一切看着都很完美。
但是！
当系统运行了一天之后，系统突然不可用了。。。打开日志，满屏幕的这个错误：

/proc/stat: Too many open files

毫无疑问， Files.lines()这个方法有问题，没有关闭打开的文件。

仔细看了一下官方文档，里面有这么一句：

If timely disposal of file system resources is required, the try-with-resources construct should be used to ensure that the stream’s close method is invoked after the stream operations are completed.

啥意思呢？就是说如果需要周期性的读取文件，需要使用 try-with-resources语句来保证stream的close方法被调用，从而关闭打开的文件。
就像下面这样：

try(Stream<String> stream = Files.lines(Paths.get(file))){
    return stream.skip(start).limit(limit).collect(Collectors.toList());
} catch (IOException e){
    logger.error("get content from{} error,{}",file, e.getMessage());
}

改用上面这个方式之后，一切安好。。。

为啥非要这样写呢？
首先需要了解Java的 try-with-resources语句，参考文档。
try-with-resources语句可以自动调用资源的close方法。因此，上面那个正确的调用方式等价于下面这种：

Stream<String> stream = Files.lines(Paths.get(file));
try {
    return stream.skip(start).limit(limit).collect(Collectors.toList());
} catch (IOException e){
    logger.error("get content from{} error,{}",file, e.getMessage());
} finally {
    stream.close();
}

重点在finally块里，调用了Stream的close方法。Stream的close方法的作用在其注释里面有说明：

Closes this stream, causing all close handlers for this stream pipeline to be called.

Stream的close方法会调用所有 close handlers。
Stream还有一个onClose方法：

/**
* Returns an equivalent stream with an additional close handler.  Close
* handlers are run when the {@link #close()} method
* is called on the stream, and are executed in the order they were
* added.  All close handlers are run, even if earlier close handlers throw
* exceptions.  If any close handler throws an exception, the first
* exception thrown will be relayed to the caller of {@code close()}, with
* any remaining exceptions added to that exception as suppressed exceptions
* (unless one of the remaining exceptions is the same exception as the
* first exception, since an exception cannot suppress itself.)  May
* return itself.
*
* <p>This is an <a href="package-summary.html#StreamOps">intermediate
* operation</a>.
*
* @param closeHandler A task to execute when the stream is closed
* @return a stream with a handler that is run if the stream is closed
*/
S onClose(Runnable closeHandler);

注释比较长，总结一下就是给Stream添加一个 close handler。Stream的close方法调用的 close handlers正是通过这个方法添加的。

那么问题来了， Files.lines()返回的Stream有添加 close handler么？
打开 Files.lines()的代码：

public static Stream<String> lines(Path path, Charset cs) throws IOException {
    BufferedReader br = Files.newBufferedReader(path, cs);
    try {
        return br.lines().onClose(asUncheckedRunnable(br));
    } catch (Error|RuntimeException e) {
        try {
            br.close();
        } catch (IOException ex) {
            try {
                e.addSuppressed(ex);
            } catch (Throwable ignore) {}
        }
        throw e;
    }
}

重点是这行 return br.lines().onClose(asUncheckedRunnable(br));。
BufferedReader.lines()方法返回的也是一个 Stream<String>。然后，调用了这个Stream的 onClose方法设置了一个close handler。
打开 asUncheckedRunnable的代码：

private static Runnable asUncheckedRunnable(Closeable c) {
    return () -> {
        try {
            c.close();
        } catch (IOException e) {
            throw new UncheckedIOException(e);
        }
    };
}

破案了~