一、如何写入文件到硬盘I/O

首先了解下换行符在不同平台不一样 String line = System.lineSeparator();

if ("\r\n".equals(line)) {
System.out.println("windows");
} else if ("\n".equals(line)) {
System.out.println("Mac");
}else  if ("\r".equals(line)) {
System.out.println("linux/unix");
}

1、FileOutputStream

public static void method2(String file, String conent) {
BufferedWriter out = null;
try {
out = new BufferedWriter(new OutputStreamWriter(
new FileOutputStream(file, true)));
out.write(conent+"\r\n");
} catch (Exception e) {
e.printStackTrace();
} finally {
try {
out.close();
} catch (IOException e) {
e.printStackTrace();
}
}

2、RandomAccessFile + NIO内存映射文件

public static void method3(String fileName, String content) {
try {
// 打开一个随机访问文件流,按读写方式
RandomAccessFile randomFile = new RandomAccessFile(fileName, "rw");
// 文件长度,字节数
long fileLength = randomFile.length();
// 将写文件指针移到文件尾。
randomFile.seek(fileLength);
randomFile.writeBytes(content+"\r\n");//不同平台的换行符不一样
randomFile.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}

RandomAccessFile的绝大多数功能,但不是全部,已经被JDK 1.4的NIO的”内存映射文件(memory-mapped files)”给取代了,你该考虑一下是不是用”内存映射文件”来代替RandomAccessFile了。

内存映射文件(memory-mapped file)能让你创建和修改那些大到无法读入内存的文件。

public class LargeMappedFiles {
static int length = 0x8000000; // 128 Mb

public static void main(String[] args) throws Exception {
    *// 为了以可读可写的方式打开文件,这里使用RandomAccessFile来创建文件。*
    FileChannel fc = new RandomAccessFile("test.dat", "rw").getChannel();
    *//注意,文件通道的可读可写要建立在文件流本身可读写的基础之上*
    MappedByteBuffer out = fc.map(FileChannel.MapMode.READ_WRITE, 0, length);
    *//写128M的内容*
    for (int i = 0; i < length; i++) {
        out.put((byte) 'x');
    }
    System.out.println("Finished writing");
    *//读取文件中间6个字节内容*
    for (int i = length / 2; i < length / 2 + 6; i++) {
        System.out.print((char) out.get(i));
    }
    fc.close();
}

}

3、FileWriter BufferedWriter PrintWriter

1、FileWriter

https://sunnylocus.iteye.com/blog/694666?page=2

/** 
     * 将信息记录到日志文件 
     * @param logFile 日志文件 
     * @param mesInfo 信息 
     * @throws IOException  
     */  
public void logMsg(File logFile,String mesInfo) throws IOException{  
if(logFile == null) {  
  throw new IllegalStateException("logFile can not be null!");  
  }  
 Writer txtWriter = new FileWriter(logFile,true);  
 txtWriter.write(dateFormat.format(new Date()) +"\t"+mesInfo+"\n");  
 txtWriter.flush();  
}

2、BufferedWriter

https://github.com/itgoyo/LogToFile/blob/master/logtofilelibrary/src/main/java/com/itgoyo/logtofilelibrary/LogToFileUtils.java

try {
BufferedWriter bw = new BufferedWriter(new FileWriter(logFile, true));
bw.write(logStr);
bw.write("\r\n");
bw.flush();
} catch (Exception e) {
Log.e(tag, "Write failure !!! " + e.toString());
}

FileWriter fw=new FileWriter(“d:/x.log”,true);//true代表不覆盖文件的内容,而是紧跟着添加内容
BufferedWriter bw=new BufferedWriter(fw);
bw.writer(message); //message是String类型的参数
bw.close() ; //关闭流
fw.close(); //关闭流
区别:如果同时使用bw fw,那么性能会大大提高,而单独使用FileWriter操作字符,每写一次数据,磁盘就有一个写操作,性能很差。
如果加了缓冲,那么会等到缓冲满了以后才会有写操作,效率和性能都有很大提高。

3、PrintWriter

PrintWriter out = new PrintWriter(new BufferedWriter(new FileWriter(“foo.out”)));
PrintWriter pw = new PrintWriter(System.out);
PrintWriter out
= new PrintWriter(new BufferedOutputStream(new FileOutputStream(“foo.out”)));
将缓冲 PrintWriter 对文件的输出。如果没有缓冲,则每次调用 print() 方法会导致将字符转换为字节,然后立即写入到文件,而这是极其低效的。

https://blog.csdn.net/qq1175421841/article/details/52411607Socket编程中,尽量用PrintWriter取代BufferedWriter

  1. PrintWriter的print、println方法可以接受任意类型的参数(字符,字节流),而BufferedWriter的write方法只能接受字符、字符数组和字符串;
  2. PrintWriter的println方法自动添加换行,BufferedWriter需要显示调用newLine方法;
  3. PrintWriter的方法不会抛异常,若关心异常,需要调用checkError方法看是否有异常发生;
  4. PrintWriter构造方法可指定参数,实现自动刷新缓存(autoflush);
  5. PrintWriter的构造方法更广。

4、flush

不使用flush()

String s = “Hello World”;
 try {
 PrintWriter pw = new PrintWriter(System.out);
 pw.write(s);
 // pw.flush();
 } catch (Exception ex) {
 ex.printStackTrace();
 }

输出:
buffer没有满,输出为空。
使用flush()

String s = “Hello World”;
 try {
 PrintWriter pw = new PrintWriter(System.out);
 pw.write(s);
 pw.flush();
 } catch (Exception ex) {
 ex.printStackTrace();
 }


得到期望的输出结果。

5、close()和flush()作用有交集!

public static void main(String[] args) {
 BufferedWriter fw =null;
 try {
 fw = new BufferedWriter(new FileWriter(“e:\test.txt”));
 fw.write(“wo shi lucky girl.”);
//fw.flush();
 fw.close();
 } catch (Exception e) {
 e.printStackTrace();
 }
 }


//fw.flush();这句有和无并不影响输出结果,不太明白词句是否必要?
因为close的时候,会把你没flush掉的一起flush掉。
缓冲区中的数据保存直到缓冲区满后才写出,也可以使用flush方法将缓冲区中的数据强制写出或使用close()方法关闭流,关闭流之前,缓冲输出流将缓冲区数据一次性写出。在这个例子中,flash()和close()都使数据强制写出,所以两种结果是一样的,如果都不写的话,会发现不能成功写出

6、Java默认缓冲区大小是多少?

默认缓冲去大小8192字节。

4、Java 各种文件writer 性能大比拼

http://ju.outofmemory.cn/entry/181638 Map File’s performance is the highest

  • RandomAccessFile with ByteBuffer:153600 test2.tmp spent time=508ms
  • RandomAccessFile with ByteBuffer:1048576 test3.tmp spent time=422ms
  • RandomAccessFile + with MappedByteBuffer and FileMap:104857600 test4.tmp spent time=110ms
  • DataOutputStream with BufferedOutputStream :153600 test5.tmp spent time=1103 ms
  • DataOutputStream without BufferedOutputStream :153600 test6.tmp spent time=680543 ms
  • PrintWriter/FileWriter with BufferedWriter:153600 test7.tmp spent time=2209 ms
  • PrintWriter/FileWriter without BufferedWriter:153600 test8.tmp spent time=2284 ms
  • PrintWriter/OutputStreamWriter with BufferedWriter:153600 test9.tmp spent time=1326 ms
  • PrintWriter/OutputStreamWriter without BufferedWriter:153600 test10.tmp spent time=2336 ms

最快代码

test3("test4.tmp", 1000000, 1024 * 1024 * 100);RandomAccessFile raf1 = new RandomAccessFile(file, “rw”);
 FileChannel fc = raf1.getChannel();
 MappedByteBuffer raf = fc.map(MapMode.READ_WRITE, 0, mapsize);
 raf.clear();
 // ByteBuffer raf = ByteBuffer.allocateDirect(mapsize);
 byte[] b1 = new byte[] { ‘a’, ‘b’, ‘c’, ‘d’, ‘e’, ‘f’, ‘g’, ‘h’ };
 byte[] utfstr = “this is a test”.getBytes(“UTF-8”);
 long s = System.nanoTime();
 long count = 0;
 for (int i = 0; i < loop; i++) {
 if (raf.remaining() < 140) {
 System.out.println(“remap”);
 count += raf.position();
 raf = fc.map(MapMode.READ_WRITE, count, mapsize);
 // raf = fc.map(MapMode.READ_WRITE, raf.position(),
 // raf.position()+mapsize);
 }
 raf.put(b1);
 raf.putInt(i);
 raf.putInt(i + 1);
 raf.put(utfstr);
 raf.put((byte) ‘\n’);
 }
 fc.close();
 raf1.close();

全部代码

public class TestAppendSpeed {
  public static void main(String[] args) throws Exception {
    // Raw RandomAccessFile
    // it is too slow to comment it
    // test1("test1.tmp",100000);
    // nio ByteBuffer RandomAccessFile
    test2("test2.tmp", 1000000, 1024 * 150);
    // nio ByteBuffer RandomAccessFile, with big buffer
    test2("test3.tmp", 1000000, 1024 * 1024);
    // using map file with MappedByteBuffer
    test3("test4.tmp", 1000000, 1024 * 1024 * 100);
    // old io, BufferedOutputStream with BufferedOutputStream
    test4("test5.tmp", 1000000, 1024 * 150);
    // old io, BufferedOutputStream without BufferedOutputStream
    // it is too slow to comment it
    // test5("test6.tmp", 1000000, 1024 * 150);
    // old io, PrintWriter/FileWriter with BufferedWriter
    test6("test7.tmp", 1000000, 1024 * 150);
    // old io, PrintWriter/FileWriter without BufferedWriter
    test7("test8.tmp", 1000000, 1024 * 150);
    // old io, PrintWriter/OutputStreamWriter without BufferedWriter
    test8("test9.tmp", 1000000, 1024 * 150);
    // old io, PrintWriter/FileWriter without BufferedWriter
    test9("test10.tmp", 1000000, 1024 * 150);
  }
  public static void test9(String file, int loop, int mapsize)
      throws Exception {
    new File(file).delete();
    PrintWriter raf = new PrintWriter(new OutputStreamWriter(
        new FileOutputStream(file, true)));
    byte[] b1 = new byte[] { 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h' };
    byte[] utfstr = "this is a test".getBytes("UTF-8");
    long s = System.nanoTime();
    for (int i = 0; i < loop; i++) {
      raf.print(b1);
      raf.print(i);
      raf.print(i + 1);
      raf.print(utfstr);
      raf.print((byte) '\n');
    }
    raf.close();
    long d = System.nanoTime() - s;
    System.out
        .println("PrintWriter/OutputStreamWriter without BufferedWriter:"
            + mapsize
            + " "
            + file
            + " spent time="
            + (d / 1000000)
            + " ms");
  }
  public static void test8(String file, int loop, int mapsize)
      throws Exception {
    new File(file).delete();
    PrintWriter raf = new PrintWriter(new BufferedWriter(
        new OutputStreamWriter(new FileOutputStream(file, true)),
        mapsize));
    byte[] b1 = new byte[] { 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h' };
    byte[] utfstr = "this is a test".getBytes("UTF-8");
    long s = System.nanoTime();
    for (int i = 0; i < loop; i++) {
      raf.print(b1);
      raf.print(i);
      raf.print(i + 1);
      raf.print(utfstr);
      raf.print((byte) '\n');
    }
    raf.close();
    long d = System.nanoTime() - s;
    System.out
        .println("PrintWriter/OutputStreamWriter with BufferedWriter:"
            + mapsize + " " + file + " spent time=" + (d / 1000000)
            + " ms");
  }
  public static void test7(String file, int loop, int mapsize)
      throws Exception {
    new File(file).delete();
    PrintWriter raf = new PrintWriter(new FileWriter(file));
    // output = new PrintWriter(new OutputStreamWriter(new
    // FileOutputStream(file, true)));
    byte[] b1 = new byte[] { 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h' };
    byte[] utfstr = "this is a test".getBytes("UTF-8");
    long s = System.nanoTime();
    for (int i = 0; i < loop; i++) {
      raf.print(b1);
      raf.print(i);
      raf.print(i + 1);
      raf.print(utfstr);
      raf.print((byte) '\n');
    }
    raf.close();
    long d = System.nanoTime() - s;
    System.out
        .println("PrintWriter/FileWriter without BufferedWriter:"
            + mapsize + " " + file + " spent time=" + (d / 1000000)
            + " ms");
  }
  public static void test6(String file, int loop, int mapsize)
      throws Exception {
    new File(file).delete();
    PrintWriter raf = new PrintWriter(new BufferedWriter(new FileWriter(
        file), mapsize));
    // output = new PrintWriter(new OutputStreamWriter(new
    // FileOutputStream(file, true)));
    byte[] b1 = new byte[] { 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h' };
    byte[] utfstr = "this is a test".getBytes("UTF-8");
    long s = System.nanoTime();
    for (int i = 0; i < loop; i++) {
      raf.print(b1);
      raf.print(i);
      raf.print(i + 1);
      raf.print(utfstr);
      raf.print((byte) '\n');
    }
    raf.close();
    long d = System.nanoTime() - s;
    System.out
        .println("PrintWriter/FileWriter with BufferedWriter:"
            + mapsize + " " + file + " spent time=" + (d / 1000000)
            + " ms");
  }
  public static void test5(String file, int loop, int mapsize)
      throws Exception {
    new File(file).delete();
    DataOutputStream raf = new DataOutputStream(new FileOutputStream(
        new File(file), true));
    byte[] b1 = new byte[] { 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h' };
    byte[] utfstr = "this is a test".getBytes("UTF-8");
    long s = System.nanoTime();
    for (int i = 0; i < loop; i++) {
      raf.write(b1);
      raf.writeInt(i);
      raf.writeInt(i + 1);
      raf.write(utfstr);
      raf.write((byte) '\n');
    }
    raf.close();
    long d = System.nanoTime() - s;
    System.out
        .println(" DataOutputStream without BufferedOutputStream :"
            + mapsize + " " + file + " spent time=" + (d / 1000000)
            + " ms");
  }
  public static void test4(String file, int loop, int mapsize)
      throws Exception {
    new File(file).delete();
    DataOutputStream raf = new DataOutputStream(new BufferedOutputStream(
        new FileOutputStream(new File(file), true), mapsize));
    byte[] b1 = new byte[] { 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h' };
    byte[] utfstr = "this is a test".getBytes("UTF-8");
    long s = System.nanoTime();
    for (int i = 0; i < loop; i++) {
      raf.write(b1);
      raf.writeInt(i);
      raf.writeInt(i + 1);
      raf.write(utfstr);
      raf.write((byte) '\n');
    }
    raf.close();
    long d = System.nanoTime() - s;
    System.out
        .println(" DataOutputStream with BufferedOutputStream :"
            + mapsize + " " + file + " spent time=" + (d / 1000000)
            + " ms");
  }
  public static void test3(String file, int loop, int mapsize)
      throws Exception {
    new File(file).delete();
    RandomAccessFile raf1 = new RandomAccessFile(file, "rw");
    FileChannel fc = raf1.getChannel();
    MappedByteBuffer raf = fc.map(MapMode.READ_WRITE, 0, mapsize);
    raf.clear();
    // ByteBuffer raf = ByteBuffer.allocateDirect(mapsize);
    byte[] b1 = new byte[] { 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h' };
    byte[] utfstr = "this is a test".getBytes("UTF-8");
    long s = System.nanoTime();
    long count = 0;
    for (int i = 0; i < loop; i++) {
      if (raf.remaining() < 140) {
        System.out.println("remap");
        count += raf.position();
        raf = fc.map(MapMode.READ_WRITE, count, mapsize);
        // raf = fc.map(MapMode.READ_WRITE, raf.position(),
        // raf.position()+mapsize);
      }
      raf.put(b1);
      raf.putInt(i);
      raf.putInt(i + 1);
      raf.put(utfstr);
      raf.put((byte) '\n');
    }
    fc.close();
    raf1.close();
    long d = System.nanoTime() - s;
    System.out
        .println("RandomAccessFile + with MappedByteBuffer and FileMap:"
            + mapsize
            + " "
            + file
            + " spent time="
            + (d / 1000000)
            + "ms");
  }
  public static void test2(String file, int loop, int mapsize)
      throws Exception {
    new File(file).delete();
    RandomAccessFile raf1 = new RandomAccessFile(file, "rw");
    FileChannel fc = raf1.getChannel();
    ByteBuffer raf = ByteBuffer.allocate(mapsize);
    raf.clear();
    // ByteBuffer raf = ByteBuffer.allocateDirect(mapsize);
    byte[] b1 = new byte[] { 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h' };
    byte[] utfstr = "this is a test".getBytes("UTF-8");
    long s = System.nanoTime();
    for (int i = 0; i < loop; i++) {
      raf.put(b1);
      raf.putInt(i);
      raf.putInt(i + 1);
      raf.put(utfstr);
      raf.put((byte) '\n');
      if (raf.remaining() < 140) {
        raf.flip();
        fc.write(raf);
        raf.compact();
      }
    }
    raf.flip();
    fc.write(raf);
    fc.close();
    raf1.close();
    long d = System.nanoTime() - s;
    System.out.println("RandomAccessFile with ByteBuffer:" + mapsize + " "
        + file + " spent time=" + (d / 1000000) + "ms");
  }
  public static void test1(String file, int loop) throws Exception {
    new File(file).delete();
    RandomAccessFile raf = new RandomAccessFile(file, "rw");
    byte[] b1 = new byte[] { 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h' };
    byte[] utfstr = "this is a test".getBytes("UTF-8");
    long s = System.nanoTime();
    for (int i = 0; i < loop; i++) {
      raf.write(b1);
      raf.writeInt(i);
      raf.writeInt(i + 1);
      raf.write(utfstr);
      raf.write('\n');
    }
    raf.close();
    long d = System.nanoTime() - s;
    System.out.println("RandomAccessFile without any buffer " + file
        + "  spent time=" + (d / 1000000) + "ms");
  }
}

二、管理多个日志文件,用实例

1、用实例

public void writeLogToFile(String logMessage) {
 if (mCurrentLogFile == null || mCurrentLogFile.length() >= LOG_FILE_MAX_SIZE) {
 mCurrentLogFile = getNewLogFile();
 }
 FileUtils.writeToFile(logMessage, mCurrentLogFile.getPath());
 }public static void writeToFile(String content, String filePath) {
 FileWriter fileWriter = null;
 try {
 //fileWriter = new FileWriter(filePath, true);
 PrintWriter fileWriter = new PrintWriter(new BufferedWriter(new FileWriter(filePath,true)));
 fileWriter.write(content);
 fileWriter.flush();
 } catch (Throwable t) {
 t.printStackTrace();
 } finally {
 if (fileWriter != null) {
 try {
 fileWriter.close();
 } catch (Exception e) {
 e.printStackTrace();
 }
 }
 }


}

三、异步

四、对象池

五、连接池