Android文件多线程下载(二)中为了使调用更加简单,做了一个简单的封装。可以直接拷贝代码。



文章目录

  • 1. 相关逻辑
  • 1.1 HTTP首部信息
  • 1.2 RandomAccessFile
  • 1.3 编码
  • 1.4 线程池
  • 1.5 自定义线程类
  • 2. 完整代码
  • 3. 后记


为了实现多线程下载,我们需要使用下面几个部分的知识来实现:

1. 相关逻辑

    主要思路为,第一次HTTP请求,可以得到待下载的文件的大小。然后我们可以根据我们设置的最大线程数目,计算每个线程需要下载的部分文件大小。然后为每个线程指定它需要下载的文件的数据范围。我们就可以让每个线程去访问网络,请求各自的数据。最后,将各个线程下载的数据拼接到文件中,组合为整个文件。也就完成了一次多线程文件下载操作。

为了完成上述的操作,我们需要了解下面的一些基础知识。

1.1 HTTP首部信息

HTTP请求头部字段Range,可以用来标识当前请求所请求的这个文件的数据范围,这个范围是byte类型的范围,比如:

connection = (HttpURLConnection) url_c.openConnection();
...
connection.setRequestProperty("Range", "bytes=" + startPos +"-" + endPos);

这样就可以在下载一个文件开启多个线程的时候,各个线程下载的数据不会重复下载,即各自完成自己负责的部分即可。

而且,当我们使用了HTTPRange这个字段,那么在请求的时候返回值为206,即表示Partial Content。也就是说浏览器只会返回对应文件的请求数据段。当然,我们每个线程只需要自己部分的数据即可。

1.2 RandomAccessFile

上面可以请求到这个文件的各个部分的数据,请求到后,我们需要写入到文件的对应位置。这里使用RandomAccessFile这个类来实现,使用seek来进行定位写入的起始位置,比如:

randomAccessFile = new RandomAccessFile(this.file, "rwd");
randomAccessFile.seek(this.startPos);  // 定位

// 写入
byte[] buffer = new byte[2048];
int len = -1;
while ((len = inputStream.read(buffer)) != -1) {
    randomAccessFile.write(buffer, 0, len);
}

注意每个流的关闭等。

1.3 编码

因为网络请求可能从URL中看不到直观的文件名,所以我们为了辨别这个文件是否下载过,可以使用MD5来将这个URL进行计算信息摘要,可以简单用来避免重复下载。

/**
 * 将url转化为一个较短的字符串表示
 * @param url
 * @return
 */
public static String hashKeyFromUrl(String url){
    try {
        MessageDigest md5 = MessageDigest.getInstance("MD5");
        byte[] bytes = md5.digest(url.getBytes());
        StringBuilder sb = new StringBuilder();
        for (byte aByte : bytes) {
            String a = Integer.toHexString(aByte >> 4 & 0b00001111);
            String b = Integer.toHexString(aByte & 0b00001111);
            sb.append(a).append(b);
        }
        return sb.toString();  // 返回16进制的MD5值的字符串表示
    }catch (NoSuchAlgorithmException e){  //找不到md5算法,或者加密过程出现异常
        return String.valueOf(url.hashCode());  // 返回哈希码的字符串表示
    }
}

1.4 线程池

因为我们需要多线程下载,故而这里使用线程池来解决这个问题,因为线程池对线程可以复用,比较节约资源。

private static final ThreadFactory mThreadFactory = new ThreadFactory(){
    private final AtomicInteger mCount = new AtomicInteger(1);

    @Override
    public Thread newThread(Runnable r) {
        return new Thread(r, "Thread#" + mCount.getAndIncrement());
    }
};


private Executor executor = new ThreadPoolExecutor(corePoolSize,
        maximumPoolSize,
        10L,
        TimeUnit.SECONDS,
        new LinkedBlockingDeque<>(),
        mThreadFactory);

1.5 自定义线程类

因为使用了线程池,所以我们至少需要一个Runnable接口的实现,而出于代码复用考虑,这里不妨定义一个继承Thread的类。在这个类的构造方法中,传入我们所需的urlRange起始、Range结束、文件最大长度和RandomAccessFile的当前的File。比如:

private static class DownloadThread extends Thread{
	public DownloadThread(String url, long startPos, long endPos, long maxFileSize, File file) {
		...
	}
	...
	@Override
    public void run() {
		...
	}
}

2. 完整代码

public class Downloader {
    private static final String TAG = "Downloader";
    private String url;
    private int connectionTimeout;
    private String method;
    private int range;
    private Context context;
    private String cachePath = "imgs";
    private static final int corePoolSize = Runtime.getRuntime().availableProcessors() + 1;
    private static final int maximumPoolSize = Runtime.getRuntime().availableProcessors() * 2 + 1;
    private HttpURLConnection connection = null;
    private String suffix;

    public Downloader(Context context, String suffix){
        connectionTimeout = 500; // 500毫秒
        method = "GET";
        range = 0;
        this.context = context;
        this.suffix = suffix;
    }

    public Downloader url(String url){
        this.url = url;
        return this;
    }

    public Downloader(Builder builder){
        this.url = builder.url;
        this.connectionTimeout = builder.connectionTimeout;
        this.method = builder.method;
        this.range = builder.range;
        this.context = builder.context;
    }

    public static class Builder{
        private String url;
        private int connectionTimeout;
        private String method;
        private int range;
        private Context context;

        public Builder(Context context){
            this.context = context;
        }

        public Builder url(String url){
            this.url = url;
            return this;
        }

        public Builder timeout(int ms){
            this.connectionTimeout = ms;
            return this;
        }

        public Builder method(String method) {
            if (!(method.toUpperCase().equals("GET") || method.toUpperCase().equals("POST"))) {
                throw new AssertionError("Assertion failed");
            }
            this.method = method;
            return this;
        }

        public Builder start(int range){
            this.range = range;
            return this;
        }

        public Downloader build(){
            return new Downloader(this);
        }
    }

    private interface DownloadListener{
        void onSuccess(File file);
        void onError(String msg);
    }

    private static class DownloadThread extends Thread{
        private long startPos;
        private long endPos;
        private RandomAccessFile randomAccessFile;
        private File file;
        private String url;
        private int connectionTimeout = 5 * 1000;  // 5秒钟
        private String method = "GET";
        private long maxFileSize;
        private DownloadListener listener;

        public DownloadThread(String url, long startPos, long endPos, long maxFileSize, File file) {
            this.startPos = startPos;
            this.endPos = endPos;
            this.url = url;
            this.file = file;
            this.maxFileSize = maxFileSize;
        }

        public void setDownloadListener(DownloadListener listener){
            this.listener = listener;
        }

        @Override
        public void run() {
            Log.e(TAG, "=========> " + Thread.currentThread().getName());
            HttpURLConnection connection = null;
            URL url_c = null;
            try{
                randomAccessFile = new RandomAccessFile(this.file, "rwd");
                randomAccessFile.seek(this.startPos);
                url_c = new URL(url);
                connection = (HttpURLConnection) url_c.openConnection();
                connection.setConnectTimeout(this.connectionTimeout);
                connection.setRequestMethod(this.method);
                connection.setRequestProperty("Charset", "UTF-8");
                connection.setRequestProperty("accept", "*/*");
                connection.setRequestProperty("Range", "bytes=" + startPos +"-" + endPos);

                Log.e(TAG, "Range: bytes=" + startPos +"-" + endPos);

                InputStream inputStream = connection.getInputStream();

                Log.e(TAG, "connection.getContentLength() == " + connection.getContentLength());

                int contentLength  = connection.getContentLength();
                if(contentLength  < 0) {
                    Log.e(TAG, "Download fail!");
                    return;
                }

                try {
                    if (connection.getResponseCode() == 206) {
                        byte[] buffer = new byte[2048];
                        int len = -1;
                        while ((len = inputStream.read(buffer)) != -1) {
                            randomAccessFile.write(buffer, 0, len);
                        }
                    }
                } catch (IOException e) {
                    e.printStackTrace();
                }finally {
                    try{
                        if(inputStream != null) inputStream.close();
                    }catch (IOException e){
                        e.printStackTrace();
                    }
                }
            }catch (IOException e){
                Log.e(TAG, "Download bitmap failed.", e);
                if(listener!=null) listener.onError(e.getLocalizedMessage());
                e.printStackTrace();
            }finally {
                if(connection != null) connection.disconnect();
                // todo 通知下载完毕
                if(this.endPos == this.maxFileSize){
                    Log.e(TAG, "Download bitmap success.");
                    if(listener!=null) listener.onSuccess(this.file);
                }
            }
        }
    }


    private static final ThreadFactory mThreadFactory = new ThreadFactory(){
        private final AtomicInteger mCount = new AtomicInteger(1);

        @Override
        public Thread newThread(Runnable r) {
            return new Thread(r, "Thread#" + mCount.getAndIncrement());
        }
    };


    private Executor executor = new ThreadPoolExecutor(corePoolSize,
            maximumPoolSize,
            10L,
            TimeUnit.SECONDS,
            new LinkedBlockingDeque<>(),
            mThreadFactory);


    public void download(){
        File path = buildPath(cachePath);
        if(Looper.myLooper() == Looper.getMainLooper()){
            throw new RuntimeException("Can't visit network from UI thread.");
        }

        try{
            URL url1 = new URL(url);
            connection = (HttpURLConnection) url1.openConnection();
            connection.setConnectTimeout(this.connectionTimeout);
            connection.setRequestMethod(this.method);
            connection.setRequestProperty("Charset", "UTF-8");
            connection.setRequestProperty("accept", "*/*");
            connection.connect();

            int contentLength  = connection.getContentLength();
            if(contentLength  < 0) {
                Log.e(TAG, "Download fail!");
                return;
            }

            // TODO 分为多个线程,进行执行
            int step = contentLength / maximumPoolSize;

            Log.e(TAG, "maximumPoolSize: " + maximumPoolSize +" , step:" + step);
            Log.e(TAG, "contentLength: " + contentLength);

            File file = new File(path, Utils.hashKeyFromUrl(url) + "." + this.suffix);
            if(contentLength == file.length()){
                Log.e(TAG, "Nothing changed!"); // already downlaod.
                return;
            }
            // 否则就下载
            for (int i = 0; i < maximumPoolSize; i++) {
                if(i != maximumPoolSize - 1) {
                    DownloadThread downloadThread = new DownloadThread(url, i * step, (i + 1) * step - 1, contentLength, file);
                    executor.execute(downloadThread);
                }else{
                    DownloadThread downloadThread = new DownloadThread(url, i * step, contentLength, contentLength, file);
                    downloadThread.setDownloadListener(new DownloadListener() {
                        @Override
                        public void onSuccess(File file) {
                            Log.e(TAG, "onSuccess: ");
                        }

                        @Override
                        public void onError(String msg) {
                            Log.e(TAG, "onError: ");
                        }
                    });
                    executor.execute(downloadThread);
                }
            }

        }catch (IOException e){
            Log.e(TAG, "Download bitmap failed.", e);
            e.printStackTrace();
        }finally {
            if(connection != null) connection.disconnect();
        }
    }

    private File buildPath(String filePath) {
        // 是否有SD卡
        boolean flag = Environment.getExternalStorageState().equals(Environment.MEDIA_MOUNTED);
        // 如果有SD卡就存在外存,否则就位于这个应用的data/package name/cache目录下
        final String cachePath;
        if(flag) cachePath = context.getExternalCacheDir().getPath();
        else cachePath = context.getCacheDir().getPath();

        File directory = new File(cachePath + File.separator + filePath);
        // 目录不存在就创建
        if(!directory.exists()) directory.mkdirs();
        return directory;
    }

}

使用:

String url2 = "https://download-ssl.firefox.com.cn/releases-sha2/stub/official/zh-CN/Firefox-latest.exe";

new Thread(new Runnable() {
    @Override
    public void run() {
        Downloader downloader = new Downloader(getApplicationContext(), "exe");
        downloader.url(url2).download();
    }
}).start();

然后可以看见文件下载成功:

android 多线程下载 速度慢 android实现多线程下载_文件多线程下载


可以看下日志:

android 多线程下载 速度慢 android实现多线程下载_数据_02


android 多线程下载 速度慢 android实现多线程下载_线程池_03

3. 后记

关于文件下载,将继续理解和封装,同时会尝试断点下载功能。