最近工作中有使用到OSS的分片上传API，整体流程就是前端将大文件进行分割，每个分片大小是5MB，分片个数是：(文件总大小 / 单个分片大小)，前端多线程处理上传分片到后端，后端接收到分片后调用OSS验证是否存在接口校验之前有没有传输过（后端集群式部署的时候这个步骤非常关键），如果分片在OSS上不存在则调用分片上传API进行上传，所有分片上传完成后调用OSS分片合并API，将所有分片在OSS上合并为我们最初的大文件，特此记录便于日后查阅。
注：本文只介绍后端分片校验、上传、整合部分，前端大文件分割代码请移步另一篇文章，传送门如下：
SpringBoot 实现大文件断点续传（前端基于WebUploader实现，1G文件上传只需5s）
1、maven依赖
<dependency>
<groupId>com.aliyun.oss</groupId>
<artifactId>aliyun-sdk-oss</artifactId>
<version>3.11.0</version>
</dependency>
<dependency>
<groupId>commons-fileupload</groupId>
<artifactId>commons-fileupload</artifactId>
<version>1.3.1</version>
</dependency>
<dependency>
<groupId>commons-lang</groupId>
<artifactId>commons-lang</artifactId>
<version>2.6</version>
<scope>compile</scope>
</dependency>2、Response代码
/**
* @author zhangzhixiang on 2020/8/6
*/
@Data
@Accessors(chain = true)
@JsonInclude(JsonInclude.Include.NON_NULL)
@ApiModel(value = "接口响应数据", description = "用于返回接口响应的内容")
public class Response<T> {
@ApiModelProperty(value = "请求响应状态码，0-成功，-1-未知，其他自定义", example = "0")
private Integer code = 0;
@ApiModelProperty(value = "请求响应消息", example = "")
private String message = "";
@ApiModelProperty(value = "请求响应详细消息", example = "")
private Object detailMessage;
@ApiModelProperty(value = "请求响应数据")
private T data;
public Response() {
this(null);
}
public Response(T data) {
this.data = data;
}
public static Response success() {
return new Response();
}
public static <T> Response success(T data) {
return new Response(data);
}
public static Response error() {
return error("");
}
public static Response error(String message) {
Response response = new Response();
response.setCode(-1);
response.setMessage(message);
return response;
}
public static Response error(int code, String message) {
Response response = new Response();
response.setCode(code);
response.setMessage(message);
return response;
}
public static Response error(IotException ex) {
Response response = new Response();
response.setCode(ex.getCode());
response.setMessage(ex.getDesc());
return response;
}
public static Response error(Status status) {
Response response = new Response();
response.setCode(status.getCode());
response.setMessage(status.getDesc());
return response;
}
public static Response error(Status status, Object... args) {
Response response = new Response();
response.setCode(status.getCode());
response.setMessage(format(status.getDesc(), args));
return response;
}
private static String format(String pattern, Object... arguments) {
return MessageFormat.format(pattern, arguments);
}
}3、Oss工具类接口代码
/**
* @author zhangzhixiang on 2020/8/1
*/
public interface OssService {
String initMultiPartUpload(String fileName, String filePath);
PartETag uploadPart(String objectName, String uploadId, InputStream instream, Integer curPartSize, Integer partNumber);
CompleteMultipartUploadResult completeMultipartUpload(String objectName, String uploadId, List<PartETag> partETags);
String getUrl(String ossKey);
InputStream getFileStream(String ossKey);
void delete(String ossKey);
String doesObjectExist(String ossKey);
}4、Oss工具类实现代码
/**
* @author zhangzhixiang on 2020/8/1
*/
@Service
public class OssServiceImpl implements OssService {
private final static GLog LOG = LogFactory.getLogger(OssServiceImpl.class);
@Autowired
private OssConstants bootstrapConstants;
private final int MB20 = 20971520;
@Override
public String initMultiPartUpload(String fileName, String filePath) {
OSS ossClient = new OSSClientBuilder().build(bootstrapConstants.getEndPoint(), bootstrapConstants.getAccessKeyId(),
bootstrapConstants.getAccessKeySecret());
try {
String ext = this.getFileExtName(fileName);
if (StringUtils.isEmpty(ext)) {
ext = "";
}
// 创建InitiateMultipartUploadRequest对象。
InitiateMultipartUploadRequest request = new InitiateMultipartUploadRequest(bootstrapConstants.getFsBucket(),
bootstrapConstants.getFsRootDir() + filePath);
// 初始化分片。
InitiateMultipartUploadResult upresult = ossClient.initiateMultipartUpload(request);
// 返回uploadId，它是分片上传事件的唯一标识，您可以根据这个uploadId发起相关的操作，如取消分片上传、查询分片上传等。
return upresult.getUploadId();
} catch (Exception e) {
LOG.error("Failed to init multiPart upload,fileName:{0},filePath:{1} \n", fileName, filePath, e);
throw new OssException(e);
} finally {
ossClient.shutdown();
}
}
@Override
public PartETag uploadPart(
String objectName,
String uploadId,
InputStream instream,
Integer curPartSize,
Integer partNumber
) {
OSS ossClient = new OSSClientBuilder().build(bootstrapConstants.getEndPoint(), bootstrapConstants.getAccessKeyId(),
bootstrapConstants.getAccessKeySecret());
try {
UploadPartRequest uploadPartRequest = new UploadPartRequest();
uploadPartRequest.setBucketName(bootstrapConstants.getFsBucket());
uploadPartRequest.setKey(bootstrapConstants.getFsRootDir() + objectName);
uploadPartRequest.setUploadId(uploadId);
uploadPartRequest.setInputStream(instream);
// 设置分片大小。除了最后一个分片没有大小限制，其他的分片最小为100 KB。
uploadPartRequest.setPartSize(curPartSize);
// 设置分片号。每一个上传的分片都有一个分片号，取值范围是1~10000，如果超出这个范围，OSS将返回InvalidArgument的错误码。
uploadPartRequest.setPartNumber(partNumber);
// 每个分片不需要按顺序上传，甚至可以在不同客户端上传，OSS会按照分片号排序组成完整的文件。
UploadPartResult uploadPartResult = ossClient.uploadPart(uploadPartRequest);
// 每次上传分片之后，OSS的返回结果包含PartETag。PartETag将被保存在redis中。
return uploadPartResult.getPartETag();
} catch (Exception e) {
LOG.error("Failed to upload part,bucketName:{0},objectName:{1},uploadId:{2} \n",
bootstrapConstants.getFsBucket(), objectName, uploadId, e);
throw new OssException(e);
} finally {
ossClient.shutdown();
}
}
@Override
public CompleteMultipartUploadResult completeMultipartUpload(
String objectName,
String uploadId,
List<PartETag> partETags
) {
OSS ossClient = new OSSClientBuilder().build(bootstrapConstants.getEndPoint(), bootstrapConstants.getAccessKeyId(),
bootstrapConstants.getAccessKeySecret());
try {
// 创建CompleteMultipartUploadRequest对象。
// 在执行完成分片上传操作时，需要提供所有有效的partETags。OSS收到提交的partETags后，会逐一验证每个分片的有效性。当所有的数据分片验证通过后，OSS将把这些分片组合成一个完整的文件。
CompleteMultipartUploadRequest completeMultipartUploadRequest =
new CompleteMultipartUploadRequest(bootstrapConstants.getFsBucket(),
bootstrapConstants.getFsRootDir() + objectName, uploadId, partETags);
// 如果需要在完成文件上传的同时设置文件访问权限，请参考以下示例代码。
// completeMultipartUploadRequest.setObjectACL(CannedAccessControlList.PublicRead);
// 完成上传。
CompleteMultipartUploadResult completeMultipartUploadResult = ossClient.completeMultipartUpload(completeMultipartUploadRequest);
return completeMultipartUploadResult;
} catch (Exception e) {
LOG.error("Failed to merge files,bucketName:{0},bucketName:{1},bucketName:{2} \n", bootstrapConstants.getFsBucket(), objectName, uploadId, e);
throw new OssException(e);
} finally {
ossClient.shutdown();
}
}
@Override
public String getUrl(String ossKey) {
Map<String, String> result = getEndPointAndBucketByOssKey(ossKey);
String bucketName = result.get("bucketName");
OSS ossClient = new OSSClientBuilder().build(bootstrapConstants.getEndPoint(), bootstrapConstants.getAccessKeyId(),
bootstrapConstants.getAccessKeySecret());
try {
Date expiration = new Date(System.currentTimeMillis() + 3600L * 1000 * 24 * 365 * 10);
URL url = ossClient.generatePresignedUrl(bucketName, ossKey, expiration);
LOG.audit("Success to get url,ossKey:{0},bucketName:{1}",
ossKey, bucketName);
if (url != null) {
return url.toString();
}
} catch (Exception e) {
LOG.error("Failed to get url,ossKey:{0},bucketName:{1}\n",
ossKey, bucketName, e);
throw new OssException(e);
} finally {
ossClient.shutdown();
}
return null;
}
@Override
public InputStream getFileStream(String ossKey) {
Map<String, String> result = getEndPointAndBucketByOssKey(ossKey);
String bucketName = result.get("bucketName");
OSS ossClient = new OSSClientBuilder().build(bootstrapConstants.getEndPoint(), bootstrapConstants.getAccessKeyId(),
bootstrapConstants.getAccessKeySecret());
OSSObject ossObject = ossClient.getObject(bucketName, ossKey);
return ossObject.getObjectContent();
}
@Override
public void delete(String ossKey) {
Map<String, String> result = getEndPointAndBucketByOssKey(ossKey);
String bucketName = result.get("bucketName");
OSS ossClient = new OSSClientBuilder().build(bootstrapConstants.getEndPoint(), bootstrapConstants.getAccessKeyId(),
bootstrapConstants.getAccessKeySecret());
try {
ossClient.deleteObject(bucketName, ossKey);
LOG.audit("Success to delete file,ossKey:{0},bucketName:{1}",
ossKey, bucketName);
} catch (Exception e) {
LOG.error("Failed to delete file,ossKey:{0},bucketName:{1} \n",
ossKey, bucketName, e);
throw new OssException(e);
} finally {
ossClient.shutdown();
}
}
@Override
public String doesObjectExist(String ossKey) {
Map<String, String> result = getEndPointAndBucketByOssKey(ossKey);
String bucketName = result.get("bucketName");
OSS ossClient = new OSSClientBuilder().build(bootstrapConstants.getEndPoint(), bootstrapConstants.getAccessKeyId(),
bootstrapConstants.getAccessKeySecret());
try {
Boolean ret = ossClient.doesObjectExist(bucketName, ossKey);
LOG.audit("Success to judge file exist,ossKey:{0},bucketName:{1}",
ossKey, bootstrapConstants.getFsBucket());
if (ret) {
return ossKey;
}
} catch (Exception e) {
LOG.error("Failed to judge file exist,ossKey:{0},bucketName:{1} \n",
ossKey, bucketName, e);
throw new OssException(e);
} finally {
ossClient.shutdown();
}
return null;
}
///
// private functions
///
private String getFileExtName(String filename) {
int index = filename.lastIndexOf(".");
if (index == -1) {
return null;
}
return filename.substring(index);
}
private Map<String, String> getEndPointAndBucketByOssKey(String ossKey) {
Map<String, String> ret = Maps.newHashMap();
if (ossKey.trim().startsWith(bootstrapConstants.getFsRootDir())) {
ret.put("bucketName", bootstrapConstants.getFsBucket());
ret.put("rootDir", bootstrapConstants.getFsRootDir());
} else if (ossKey.trim().startsWith(bootstrapConstants.getImgRootDir())) {
ret.put("bucketName", bootstrapConstants.getImgBucket());
ret.put("rootDir", bootstrapConstants.getImgRootDir());
} else{
ret.put("bucketName", bootstrapConstants.getDataBucket());
ret.put("rootDir", bootstrapConstants.getDataRootDir());
}
return ret;
}
private Map<String, String> getEndPointAndBucketByFileSize(int fileSize) {
Map<String, String> ret = Maps.newHashMap();
if (fileSize > MB20) {
ret.put("bucketName", bootstrapConstants.getFsBucket());
ret.put("rootDir", bootstrapConstants.getFsRootDir());
} else {
ret.put("bucketName", bootstrapConstants.getImgBucket());
ret.put("rootDir", bootstrapConstants.getImgRootDir());
}
return ret;
}
}5、Controller层代码
/**
* @author hanyuanliang on 2020/9/3
*/
@RestController
@RequestMapping("/v1.0/sys/admin/files")
@Api(tags = "后台公共模块")
public class SysUploadFileAdmin {
private static Logger LOG = LoggerFactory.getLogger(SysUploadFileAdmin.class);
@Autowired
private OssService ossService;
@Autowired
private SysFileService fileService;
@ApiOperation("断点叙传注册")
@PostMapping(value = "/breakpointRenewal/register")
public Response breakpointRegister(@RequestParam("fileMd5") String fileMd5,
@RequestParam("fileName") String fileName,
@RequestParam("fileSize") Long fileSize,
@RequestParam("mimetype") String mimetype,
@RequestParam("fileExt") String fileExt) {
try {
return fileService.breakpointRegister(fileMd5, fileName, fileSize, mimetype, fileExt);
} catch (Exception e) {
LOG.error("********FileController->breakpointRegister throw Exception.fileMd5:{},fileName:{}********", fileMd5, fileName, e);
}
return Response.error("断点叙传注册");
}
@ApiOperation("断点叙传")
@PostMapping(value = "/breakpointRenewal",
produces = {MediaType.APPLICATION_JSON_UTF8_VALUE},
consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
public Response breakpointRenewal(@RequestPart("file") MultipartFile file,
@RequestParam("fileMd5") String fileMd5,
@RequestParam("uploadId") String uploadId,
@RequestParam("chunk") Integer chunk,
@RequestParam("fileExt") String fileExt
) {
try {
return fileService.breakpointRenewal(file, fileMd5, uploadId, chunk, fileExt);
} catch (Exception e) {
LOG.error("********FileController->breakpointRenewal throw Exception.fileMd5:{},chunk:{}********", fileMd5, chunk, e);
}
return Response.error("断点叙传失败");
}
@ApiOperation("检查分块是否存在")
@PostMapping(value = "/breakpointRenewal/existsChunk")
public Response existsChunk(@RequestParam("fileMd5") String fileMd5,
@RequestParam("chunk") Integer chunk,
@RequestParam("chunkSize") Integer chunkSize) {
try {
return fileService.existsChunk(fileMd5, chunk, chunkSize);
} catch (Exception e) {
LOG.error("********FileController->breakpointRenewal throw Exception.fileMd5:{},chunk:{}********", fileMd5, chunk, e);
}
return Response.error("检查分块是否存在失败");
}
@ApiOperation("合并文件块")
@PostMapping(value = "/breakpointRenewal/mergeChunks")
public Response mergeChunks(@RequestParam("fileMd5") String fileMd5,
@RequestParam("uploadId") String uploadId,
@RequestParam("fileName") String fileName,
@RequestParam("fileSize") Long fileSize,
@RequestParam("mimetype") String mimetype,
@RequestParam("fileExt") String fileExt) {
try {
return fileService.mergeChunks(fileMd5, uploadId, fileName, fileSize, mimetype, fileExt);
} catch (Exception e) {
LOG.error("********FileController->breakpointRenewal throw Exception.fileMd5:{},fileName:{}********", fileMd5, fileName, e);
}
return Response.error("合并文件块失败");
}
@DeleteMapping
@ApiOperation("删除文件")
public Response<Boolean> delete(@ApiParam("文件代号") @RequestParam("code") String code) {
ossService.delete(code);
return Response.success(true);
}
}6、Service层接口代码
/**
* @author zhangzhixiang on 2020/8/5
*/
public interface SysFileService {
Response breakpointRegister(String fileMd5, String fileName, Long fileSize, String mimetype, String fileExt);
Response breakpointRenewal(MultipartFile file, String fileMd5, String uploadId, Integer chunk, String fileExt);
Response existsChunk(String fileMd5, Integer chunk, Integer chunkSize);
Response mergeChunks(String fileMd5, String uploadId, String fileName, Long fileSize, String mimetype, String fileExt);
}7、Service层实现代码
下面代码中有用到redis保存分片信息，大家配置成自己的redis就可以了，我后面代码中会给出redis工具的代码
/**
* @author zhangzhixiang on 2020/8/5
*/
@Service
public class SysFileServiceImpl implements SysFileService {
@Resource
private RedisService redisService;
@Autowired
private OssService ossService;
@Value("${file.biz.file.upload}")
private String uploadDir;
@Value("${file.biz.file.breakpoint}")
private String breakpointDir;
@Value("${file.oss.fsRootDir}")
private String fsRootDir;
private String getFileFolderPath(String fileMd5) {
return getUploadFilePath("/") + fileMd5.substring(0, 1) + "/" + fileMd5.substring(1, 2) + "/";
}
private String getFilePath(String fileMd5, String fileExt) {
return getFileFolderPath(fileMd5) + fileMd5 + "." + fileExt;
}
private String getFileRelativePath(String fileMd5, String fileExt) {
return uploadDir + "/" + fileMd5.substring(0, 1) + "/" + fileMd5.substring(1, 2) + "/" + fileMd5 + "." + fileExt;
}
private String getChunkFileFolderPath(String fileMd5) {
return breakpointDir + "/" + fileMd5 + "/";
}
@Override
public Response breakpointRegister(String fileMd5, String fileName, Long fileSize, String mimetype, String fileExt) {
//检查文件是否存在于OSS，不存在则返回错误码，否则返回文件相关信息
String filePath = this.getFilePath(fileMd5, fileExt);
String ossCode = ossService.doesObjectExist(fsRootDir + filePath);
//若不存在则返回注册成功
if (!ObjectUtils.isEmpty(ossCode)) {
String url = ossService.getUrl(ossCode);
SysUploadItemVo sysUploadItemVo = new SysUploadItemVo(ossCode, fileMd5, url, "done", fileName);
return Response.success(sysUploadItemVo).setMessage("文件已存在");
} else {
String uploadId = ossService.initMultiPartUpload(fileName, filePath);
return Response.success(uploadId).setMessage("注册成功");
}
}
@Override
public Response breakpointRenewal(
MultipartFile file,
String fileMd5,
String uploadId,
Integer chunk,
String fileExt
) {
try {
// 检查分块目录是否存在
String objectName = this.getFilePath(fileMd5, fileExt);
PartETag uploadPart = ossService.uploadPart(objectName, uploadId, file.getInputStream(),
(int) file.getSize(), (chunk + 1));
redisService.hset(RedisPrefixConst.BREAKPOINT_H_KEY_PREFIX + fileMd5, (chunk + 1) + "",
JSONObject.toJSONString(uploadPart), 86400);
} catch (IOException e) {
e.printStackTrace();
redisService.hdel(RedisPrefixConst.BREAKPOINT_H_KEY_PREFIX + fileMd5, (chunk + 1) + "");
}
return null;
}
@Override
public Response existsChunk(
String fileMd5,
Integer chunk,
Integer chunkSize
) {
//检查分块信息是否存在于redis中，若存在则返回true，否则false
String partMd5 = redisService.hget(RedisPrefixConst.BREAKPOINT_H_KEY_PREFIX + fileMd5, (chunk + 1) + "");
if (StringUtils.isEmpty(partMd5)) {
return Response.success(false);
}
return Response.success(true);
}
@Override
public Response mergeChunks(
String fileMd5,
String uploadId,
String fileName,
Long fileSize,
String mimetype,
String fileExt
) {
//从redis中获取改文件的所有分块信息列表（ETag）
Map<String, String> dataMap = redisService.hscan(RedisPrefixConst.BREAKPOINT_H_KEY_PREFIX + fileMd5, null);
//调用oss合并文件方法，获取完整文件标识信息
List<PartETag> partETagList = Lists.newArrayList();
for (Map.Entry<String, String> entry : dataMap.entrySet()) {
partETagList.add(JSONObject.parseObject(entry.getValue(), PartETag.class));
}
//删除redis分块记录
redisService.delete(RedisPrefixConst.BREAKPOINT_H_KEY_PREFIX + fileMd5);
//组装文件
String objectName = this.getFilePath(fileMd5, fileExt);
CompleteMultipartUploadResult result = ossService.completeMultipartUpload(objectName, uploadId, partETagList);
//组装返回结果到前端
String url = ossService.getUrl(result.getKey());
SysUploadItemVo sysUploadItemVo = new SysUploadItemVo(result.getKey(), fileMd5, url, "done", fileName);
return Response.success(sysUploadItemVo).setMessage("文件上传成功");
}
///
// private functions
///
private File mergeFile(List<File> chunkFileList, File mergeFile) {
try {
// 有删 无创建
if (mergeFile.exists()) {
mergeFile.delete();
} else {
File newParentFile = mergeFile.getParentFile();
if (!newParentFile.exists()) {
newParentFile.mkdirs();
}
mergeFile.createNewFile();
}
// 排序
Collections.sort(chunkFileList, (o1, o2) -> {
if (Integer.parseInt(o1.getName()) > Integer.parseInt(o2.getName())) {
return 1;
}
return -1;
});
byte[] b = new byte[1024];
RandomAccessFile writeFile = new RandomAccessFile(mergeFile, "rw");
for (File chunkFile : chunkFileList) {
RandomAccessFile readFile = new RandomAccessFile(chunkFile, "r");
int len = -1;
while ((len = readFile.read(b)) != -1) {
writeFile.write(b, 0, len);
}
readFile.close();
}
writeFile.close();
return mergeFile;
} catch (IOException e) {
e.printStackTrace();
return null;
}
}
private String getUploadFilePath(String filePath) {
return uploadDir + filePath;
}
}8、RedisService接口代码
/**
* @author chengmo on 2020/7/16
*/
public interface RedisService {
Long increment(String key);
boolean hasKey(String key);
String get(String key);
boolean delete(String key);
boolean set(String key, String value);
boolean set(String key, String value, Long timeout);
boolean setExpire(String key, Long expire);
long getExpire(String key);
String hget(String key, String hKey);
Map<String, String> hscan(String key, String pattern);
boolean hset(String key, String hKey, String hValue);
boolean hset(String key, String hKey, String hValue, long timeout);
boolean hdel(String key, String hKey);
boolean sadd(String key, String... value);
Set<String> smembers(String key);
boolean szset(String key, String value, double score);
Long szdel(String key, String... value);
Set<String> szrange(String key, long start, long end);
Set<String> szdelAndGet(String key, long start, long end);
Double szscore(String key, String value);
Long lLeftPush(String key, String value);
Long lLeftPushAll(String key, Collection<String> values);
String lRightPop(String key, long expire, TimeUnit unit);
RLock getLock(String key);
}9、RedisServiceImpl代码
/**
* @author chengmo on 2020/7/16
*/
@Service
public class RedisServiceImpl implements RedisService {
private static final GLog LOG = LogFactory.getLogger(RedisServiceImpl.class);
@Autowired
private RedissonClient client;
@Autowired
private StringRedisTemplate stringRedisTemplate;
private final String namespace;
public RedisServiceImpl(@Value("${spring.application.name}") String namespace) {
this.namespace = Constants.GLZ + Constants.COLON +
StringUtils.replace(namespace, "-", "_") +
Constants.COLON;
}
public Long increment(String key) {
String redisKey = this.rebuildKey(key);
Long req = null;
try {
ValueOperations<String, String> op = stringRedisTemplate.opsForValue();
req = op.increment(redisKey);
} catch (Exception e) {
LOG.error("increment [Key:{0}] from redis failed!", e, redisKey);
}
return req;
}
@Override
public boolean hasKey(String key) {
String redisKey = this.rebuildKey(key);
try {
return stringRedisTemplate.hasKey(redisKey);
} catch (Exception e) {
LOG.error("hasKey [Key:{0}] from redis failed!", e, redisKey);
}
return false;
}
@Override
public String get(String key) {
String redisKey = this.rebuildKey(key);
String result;
try {
result = stringRedisTemplate.opsForValue().get(redisKey);
if (null == result) {
return null;
}
return result;
} catch (Exception e) {
LOG.error("Get [Key:{0}] from redis failed!", e, redisKey);
}
return null;
}
@Override
public boolean delete(String key) {
String redisKey = this.rebuildKey(key);
try {
stringRedisTemplate.delete(redisKey);
return true;
} catch (Exception e) {
LOG.error("Delete [Key:{0}]from redis failed!", e, redisKey);
}
return false;
}
@Override
public boolean set(String key, String value) {
String redisKey = this.rebuildKey(key);
try {
stringRedisTemplate.opsForValue().set(redisKey, value);
return true;
} catch (Exception e) {
LOG.error("Set [Key:{0}][Value:{1}] into redis failed!", e, redisKey, value);
}
return false;
}
@Override
public boolean set(String key, String value, Long timeout) {
String redisKey = this.rebuildKey(key);
try {
stringRedisTemplate.opsForValue().set(redisKey, value, timeout, TimeUnit.SECONDS);
return true;
} catch (Exception e) {
LOG.error("Set [Key:{0}][Value:{1}][TimeOut:{2}] into redis failed!", e, redisKey, value, timeout);
}
return false;
}
@Override
public boolean setExpire(String key, Long expire) {
String redisKey = this.rebuildKey(key);
try {
stringRedisTemplate.expire(redisKey, expire, TimeUnit.SECONDS);
return true;
} catch (Exception e) {
LOG.error("setExpire [Key:{0}][handleAutoRenew:{1}] into redis failed!", e, redisKey, expire);
}
return false;
}
@Override
public long getExpire(String key) {
String redisKey = this.rebuildKey(key);
try {
return stringRedisTemplate.getExpire(redisKey, TimeUnit.SECONDS);
} catch (Exception e) {
LOG.error("getExpire [Key:{0}][handleAutoRenew:{1}] into redis failed!", e, redisKey);
}
return 0l;
}
@Override
public String hget(String key, String hKey) {
String redisKey = this.rebuildKey(key);
try {
return (String) stringRedisTemplate.opsForHash().get(redisKey, hKey);
} catch (Exception e) {
LOG.error("hget key:[{0}] hkey:[{1}] fail ", e, redisKey, hKey);
}
return null;
}
@Override
public Map<String, String> hscan(String key, String pattern) {
String redisKey = this.rebuildKey(key);
Cursor<Map.Entry<Object, Object>> cursor = null;
try {
ScanOptions scanOptions = null;
if (StringUtils.isEmpty(pattern))
scanOptions = ScanOptions.NONE;
else
scanOptions = ScanOptions.scanOptions().match(pattern).build();
cursor = stringRedisTemplate.opsForHash().scan(redisKey, scanOptions);
Map<String, String> map = new HashMap<>();
while (cursor.hasNext()) {
Map.Entry<Object, Object> entry = cursor.next();
Object value = entry.getValue();
map.put(entry.getKey().toString(), null != value ? value.toString() : null);
}
return map;
} catch (Exception e) {
LOG.error("hscan key:[{0}] fail ", e, redisKey);
} finally {
if (null != cursor) {
try {
cursor.close();
} catch (IOException e) {
LOG.error("hscan close cursor fail ", e);
}
}
}
return null;
}
@Override
public boolean hset(String key, String hKey, String hValue) {
String redisKey = this.rebuildKey(key);
try {
stringRedisTemplate.opsForHash().put(redisKey, hKey, hValue);
return true;
} catch (Exception e) {
LOG.error("hset key:[{0}] hkey:[{1}] fail ", e, redisKey, hKey);
}
return true;
}
@Override
public boolean hset(String key, String hKey, String hValue, long timeout) {
String redisKey = this.rebuildKey(key);
try {
stringRedisTemplate.opsForHash().put(redisKey, hKey, hValue);
stringRedisTemplate.expire(redisKey, timeout, TimeUnit.SECONDS);
return true;
} catch (Exception e) {
LOG.error("hset key:[{0}] hkey:[{1}] fail ", e, redisKey, hKey);
}
return true;
}
@Override
public boolean hdel(String key, String hKey) {
String redisKey = this.rebuildKey(key);
try {
stringRedisTemplate.opsForHash().delete(redisKey, hKey);
return true;
} catch (Exception e) {
LOG.error("hdel key:[{0}] hkey:[{1}] fail ", e, redisKey, hKey);
}
return false;
}
@Override
public boolean sadd(String key, String... value) {
String redisKey = this.rebuildKey(key);
try {
Long add = stringRedisTemplate.opsForSet().add(redisKey, value);
return Objects.nonNull(add) && add > 0;
} catch (Exception e) {
LOG.error("sadd value:[{0}] to key:[{1}] fail", redisKey, value, e);
}
return false;
}
@Override
public Set<String> smembers(String key) {
String redisKey = this.rebuildKey(key);
try {
return stringRedisTemplate.opsForSet().members(redisKey);
} catch (Exception e) {
LOG.error("scard values from key:[{0}] fail", redisKey, e);
}
return new HashSet();
}
@Override
public boolean szset(String key, String value, double score) {
String redisKey = this.rebuildKey(key);
try {
return stringRedisTemplate.opsForZSet().add(redisKey, value, score);
}catch (Exception e){
LOG.error("szset value:[{0}] from key:[{1}] fail", value, redisKey, e);
}
return false;
}
@Override
public Long szdel(String key, String... value) {
String redisKey = this.rebuildKey(key);
try {
return stringRedisTemplate.opsForZSet().remove(redisKey, value);
}catch (Exception e){
LOG.error("szdel values:[{0}] from key:[{1}] fail", value, redisKey, e);
}
return 0L;
}
@Override
public Set<String> szrange(String key, long start, long end) {
String redisKey = this.rebuildKey(key);
try {
return stringRedisTemplate.opsForZSet().range(redisKey, start, end);
}catch (Exception e){
LOG.error("szrange start:[{0}] end:[{1}] from key:[{2}] fail", start, end, redisKey, e);
}
return null;
}
@Override
public Set<String> szdelAndGet(String key, long start, long end) {
String redisKey = this.rebuildKey(key);
RLock lock = client.getLock(redisKey+"_Lock");
try {
lock.lock();
ZSetOperations<String, String> operation = stringRedisTemplate.opsForZSet();
Set<String> set = operation.range(redisKey, start, end);
if(null == set || set.size()==0)return set;
operation.remove(redisKey, set.toArray());
return set;
}catch (Exception e){
LOG.error("szdelAndGet start:[{0}] end:[{1}] from key:[{2}] fail", start, end, redisKey, e);
}finally {
lock.unlock();
}
return null;
}
@Override
public Double szscore(String key, String value) {
String redisKey = this.rebuildKey(key);
try {
return stringRedisTemplate.opsForZSet().score(redisKey, value);
}catch (Exception e){
LOG.error("szscore value:[{0}] from key:[{1}] fail", value, redisKey, e);
}
return null;
}
@Override
public Long lLeftPush(String key, String value) {
String redisKey = this.rebuildKey(key);
try {
return stringRedisTemplate.opsForList().leftPush(redisKey, value);
}catch (Exception e){
LOG.error("lLeftPush value:[{0}] from key:[{1}] fail", value, redisKey, e);
}
return null;
}
@Override
public Long lLeftPushAll(String key, Collection<String> values) {
String redisKey = this.rebuildKey(key);
try {
return stringRedisTemplate.opsForList().leftPushAll(key, values);
}catch (Exception e){
LOG.error("lLeftPushAll values:[{0}] from key:[{1}] fail", values, redisKey, e);
}
return null;
}
@Override
public String lRightPop(String key, long expire, TimeUnit unit) {
String redisKey = this.rebuildKey(key);
try {
return stringRedisTemplate.opsForList().rightPop(key, expire, unit);
}catch (Exception e){
LOG.error("lRightPop key:[{1}] fail", redisKey, e);
}
return null;
}
@Override
public RLock getLock(String key) {
String redisKey = this.rebuildKey(key);
try {
RLock lock = client.getLock(redisKey);
return lock;
} catch (Exception e) {
LOG.error("getLock key:[{0}] fail ", e, redisKey);
}
return null;
}
///
// private functions
///
private String rebuildKey(String redisKey) {
return namespace + redisKey;
}
}10、application-local.properties代码
#==========================================
# tomcat
#==========================================
server.port=10030
#==========================================
# cloud
#==========================================
spring.application.name=iot-sys
spring.cloud.nacos.discovery.server-addr=localhost:8848
spring.main.allow-bean-definition-overriding=true
spring.servlet.multipart.enabled=true
spring.servlet.multipart.max-file-size=1024MB
spring.servlet.multipart.max-request-size=1024MB
#==========================================
# jackson
#==========================================
spring.jackson.property-naming-strategy=SNAKE_CASE
#==========================================
# redis
#==========================================
spring.redis.host=localhost
spring.redis.port=6379
spring.redis.password=xxxx
#spring.redis.database=5
spring.redis.jedis.pool.max-idle=8
spring.redis.jedis.pool.min-idle=0
spring.redis.jedis.pool.max-active=8
spring.redis.jedis.pool.max-wait=-1
#==========================================
# mysql
#==========================================
spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver
spring.datasource.url=jdbc:mysql://localhost:3306/iot_sys_dev?autoReconnection=true&useSSL=true&useUnicode=true&characterEncoding=UTF-8&allowMultiQueries=true&rewriteBatchedStatements=true&serverTimezone=GMT%2B8
spring.datasource.username=xxxx
spring.datasource.password=xxxx
spring.datasource.type=com.alibaba.druid.pool.DruidDataSource
spring.datasource.tomcat.max-idle=10
spring.datasource.tomcat.max-wait=20000
spring.datasource.tomcat.min-idle=5
spring.datasource.tomcat.initial-size=10
spring.datasource.tomcat.max-active=20000
#==========================================
# mybatis-plus
#==========================================
mybatis-plus.mapper-locations=classpath*:mapper/**/*.xml
mybatis-plus.configuration.log-impl=org.apache.ibatis.logging.stdout.StdOutImpl
#==========================================
# authorization
#==========================================
authorization.switch=false
#==========================================
# oss
#==========================================
file.oss.endPoint=https://oss-cn-shenzhen.aliyuncs.com
file.oss.accessKeyId=LTAI4GKJvA3ZRhqqvMko9Ly5
file.oss.accessKeySecret=lGu5RY6lYGFl44N3njwRY8U9AydBfv
file.oss.dataBucket=galanz-iot
file.oss.dataRootDir=data
file.oss.imgBucket=galanz-next1-img
file.oss.imgRootDir=img
file.oss.fsBucket=galanz-next1-fs
file.oss.fsRootDir=fs
file.oss.rootDir.device.upload=device/upload
file.oss.rootDir.device.download=device/download
file.oss.device.template.batch-import.url=xxxx
Expires=1598696322&OSSAccessKeyId=TMP.3KiMniHEYKN5WkRx3naZs2xghfxWCTYCowFpn3ivmzU4qTvbQmaRHLdCW7zgfaiadcZ3gVz4QRfKJ2wJ5XvTbRAnx5NEdN&Signature=7NY5GyFwmwTSkoIkDaPjfGhzenQ%3D
file.oss.tmp=xxx
file.root=C:/Users/zzx/galanz/file
file.biz.file.upload=/upload
file.biz.file.breakpoint=/breakpoint到此 SpringBoot通过OSS实现大文件分片上传介绍完成。
本文章为转载内容，我们尊重原作者对文章享有的著作权。如有内容错误或侵权问题，欢迎原作者联系我们进行内容更正或删除文章。
