S3 亚马逊附件的上传、下载、批量下载、删除、创建桶~_s3桶 复制url下载

S3 亚马逊附件的上传、下载、批量下载、删除、创建桶~

一、配置文件

• 说明：需要亚马逊对象存储服务对应三个重要的配置 地址 、accessKeyId、secretAccessKey

例如 参考不可用
url : https://xxx-xxx-xxx.com               S3对象服务的部署地址
accessKeyId：O5YNKAECYZYTQZL1P6IE           AWS访问密钥，用于识别与AWS交互的用户
secretAccessKey：ky1N12hUIJENO5V37XMzbh9DygtPYx0RU9g14xDVAIS    秘密访问密钥，
用于验证与AWS交互的用户。
1
2
3
4
5

• 配置

package com.luobo.config;

import lombok.Data;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.stereotype.Component;
import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
import software.amazon.awssdk.auth.credentials.AwsCredentials;
import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3Client;
import java.net.URI;
@Data
@Component
public class OssConfig {
    @Value("${s3Url}")
    private String s3Url;
    
    @Value("${accessKeyId}")
    private String accessKeyId;
    
    @Value("${secretAccessKey}")
    private String secretAccessKey;
    /**
     * 创建客户端
     * */
    @Bean
    public S3Client createS3Client(){
        //创建凭证（相当于需要认证的账号密码）
        AwsCredentials credentials = AwsBasicCredentials.create(s3Accesskey, s3SercretKey);
        StaticCredentialsProvider credentialsProvider = StaticCredentialsProvider.create(credentials);
        return  S3Client.builder()
                .endpointOverride(URI.create(s3Url))
                .credentialsProvider(credentialsProvider)
                .forcePathStyle(true)
                .region(Region.US_EAST_1)
                .build();
    }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

二、附件具体操作

• 2.1、下载封装桶和文件信息类

package com.luobo.bean.bo;

import io.swagger.annotations.ApiModelProperty;
import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Data;
import lombok.NoArgsConstructor;
@Builder
@Data
@AllArgsConstructor
@NoArgsConstructor
public class S3FileBo {
    @ApiModelProperty(value = "桶名称", required = true)
    private String bucketName;
    @ApiModelProperty(value = "附件UUID", required = true)
    private String fileId;
    @ApiModelProperty(value = "附件名称", required = false)
    private String fileName;
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

• 2.2、附件上传方法

public boolean uploadFile(MultipartFile file, S3FileBo s3FileBo) {
        boolean ifSuccess = false;
        log.info("上传附件桶名称：{}，附件id：{}", s3FileBo.getBucketName(), s3FileBo.getFileId());
        //创建S3客户端
        S3Client s3Client = ossConfig.createS3Client();
        try {
            //构建存储附件的请求
            PutObjectRequest request = PutObjectRequest.builder()
                    .bucket(s3FileBo.getBucketName())
                    .key(s3FileBo.getFileId())
                    .contentType(file.getContentType())
                    .build();
            //根据客户端上传附件
            PutObjectResponse uploadResult = s3Client.putObject(request, RequestBody.fromInputStream(file.getInputStream(), file.getSize()));
            ifSuccess = uploadResult.sdkHttpResponse().isSuccessful();
            if (ifSuccess) {
                log.info("附件上传成功:{}", uploadResult.eTag());
                ifSuccess = true;
            } else {
                log.info("附件上传失败:{}", uploadResult.eTag());
            }
        } catch (IOException e) {
            log.info("上传附件异常~");
            e.printStackTrace();
            ifSuccess = false;
        } finally {
            s3Client.close();
        }
        return ifSuccess;
    }
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

• 2.3、附件下载方法

public boolean downloadFile(S3FileBo s3FileBo)  {
        log.info("下载附件桶名称：{}，附件id：{}", s3FileBo.getBucketName(), s3FileBo.getFileId());
        S3Client s3Client = ossConfig.createS3Client();
        OutputStream os =null;
        InputStream inputStream =null;
        ResponseInputStream<GetObjectResponse> downLoadResult=null;
        try {
            GetObjectRequest request = GetObjectRequest.builder()
                    .bucket(s3FileBo.getBucketName())
                    .key(s3FileBo.getFileId())
                    .build();
            downLoadResult= s3Client.getObject(request);
            inputStream = new BufferedInputStream(downLoadResult);
            String fileName=ossFileBo.getFileName();
            String contentDisposition = "attachment;filename="
                    + new String(fileName.getBytes(StandardCharsets.UTF_8), StandardCharsets.ISO_8859_1);
            response.setContentType(downLoadResult.response().contentType());
            response.setCharacterEncoding("UTF-8");
            response.setHeader("Content-Disposition", contentDisposition);
            os = response.getOutputStream();
            //用的hutool的工具类进行流拷贝
            IoUtil.copy(inputStream,os);
            //刷新响应缓冲区
            response.flushBuffer();
            log.info("下载文件成功~");
            return true;
        } catch (Exception e) {
            log.info("下载附件异常~");
            e.printStackTrace();
        } finally {
            //关闭流
            try {
                os.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
            try {
                inputStream.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
            try {
                downLoadResult.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
            s3Client.close();
        }
        return false;
    }
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50

• 2.4、附件批量下载方法

批量下载的原理其实就是，将对象储存中的每个附件以流的形式下载下来，然后把这文件放入带压缩流中
去打包下载
1
2

/**
 * 附件批量下载方法
 * */
 public void batchDownloadFile(List<S3FileBo> s3FileBo) {
        ZipOutputStream zipOutputStream =null;
        try {
             zipOutputStream = new ZipOutputStream(response.getOutputStream());
            String contentDisposition = "attachment;filename="
                    + new String(PmoConfConstants.PLAN_BATCH_DOWNLOAD_NAME.getBytes(StandardCharsets.UTF_8), StandardCharsets.ISO_8859_1);
            response.setHeader("Content-Disposition", contentDisposition);
            response.setContentType("application/octet-stream");
            response.setCharacterEncoding("UTF-8");
            for (S3FileBo s3 : s3FileBo) {
                //可能存在附件重名的请款，需要自己根据算法处理，我这里就不处理重名的了
                zipOutputStream.putNextEntry(new ZipEntry(s3.getFileName()));
                batchDownloadFileToStream(s3,zipOutputStream);
            }
            log.info("批量下载下载文件成功，文件数量：{}",s3FileBo.size());
        } catch (IOException e) {
            e.printStackTrace();
        }finally {
            try {
                if (zipOutputStream!=null){
                    zipOutputStream.close();
                }
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
}
/**
 * 附件批量下载方法
 * */
private boolean batchDownloadFileToStream(S3FileBo s3FileBo,ZipOutputStream zipOutputStream)  {
        log.info("下载新对象存储附件桶名称：{}，附件id：{}", s3FileBo.getBucketName(), s3FileBo.getFileId());
        S3Client s3Client = ossConfig.createS3Client();
        InputStream inputStream =null;
        ResponseInputStream<GetObjectResponse> downLoadResult=null;
        try {
            GetObjectRequest request = GetObjectRequest.builder()
                    .bucket(s3FileBo.getBucketName())
                    .key(s3FileBo.getFileId())
                    .build();
            downLoadResult= s3Client.getObject(request);
            inputStream = new BufferedInputStream(downLoadResult);
            //将下载的文件流写入文件中。
            IoUtil.copy(inputStream,zipOutputStream);
            log.info("下载文件成功~");
            return true;
        } catch (Exception e) {
            log.info("下载附件异常~");
            e.printStackTrace();
        } finally {
            //关闭流
            try {
                inputStream.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
            try {
                downLoadResult.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
            s3Client.close();
        }
        return false;
    }
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68

• 2.5、删除指定附件

public boolean deleteFile(S3FileBo s3FileBo) {
        boolean ifSuccess = false;
        log.info("删除附件桶名称：{}，附件id：{}", s3FileBo.getBucketName(), s3FileBo.getFileId());
        S3Client s3Client = ossConfig.createS3Client();
        try {
            DeleteObjectRequest build = DeleteObjectRequest.builder()
                    .key(s3FileBo.getFileId())
                    .bucket(s3FileBo.getBucketName())
                    .build();
            DeleteObjectResponse deleteObjectResponse = s3Client.deleteObject(build);
            if (deleteObjectResponse.sdkHttpResponse().isSuccessful()) {
                log.info("删除文件成功~");
                ifSuccess = true;
            } else {
                log.info("删除文件失败~");
            }
        } catch (AwsServiceException e) {
            e.printStackTrace();
        } catch (SdkClientException e) {
            e.printStackTrace();
        } finally {
            s3Client.close();
        }
        return ifSuccess;
    }
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

• 2.6、创建桶

public boolean createS3Bucket(String bucketName) {
        boolean ifSuccess = false;
        S3Client s3Client =ossConfig.createS3Client();;
        try {
            CreateBucketRequest createBucketRequest = CreateBucketRequest.builder()
                    .bucket(bucketName)
                    .build();
            CreateBucketResponse bucket = s3Client.createBucket(createBucketRequest);
            ifSuccess=bucket.sdkHttpResponse().isSuccessful();
            if (ifSuccess) {
                log.info("创建桶成功~");
                ifSuccess = true;
            } else {
                log.info("创建桶失败~");
            }
        } catch (AwsServiceException e) {
            e.printStackTrace();
        } catch (SdkClientException e) {
            e.printStackTrace();
        } finally {
            s3Client.close();
        }
        return ifSuccess;
    }
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

• 2.7、获取对象存储中所有的桶名称

public List<String> getAllBucketName() {
        S3Client s3Client = ossConfig.createS3Client();
        List<String> allBucketName = null;
        try {
            ListBucketsResponse listBucketsResponse = s3Client.listBuckets();
            List<Bucket> buckets = listBucketsResponse.buckets();
            allBucketName = buckets.stream().map(Bucket::name).collect(Collectors.toList());
        } catch (AwsServiceException e) {
            e.printStackTrace();
        } catch (SdkClientException e) {
            e.printStackTrace();
        } finally {
            s3Client.close();
        }
        return allBucketName;
    }
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

• 2.8、获取指定桶中的附件(判断桶中文件的数量)

public List<S3Object> getBucketFile(String bucketName) {
        S3Client s3Client = ossConfig.createS3Client();
        List<S3Object> file = new ArrayList<>();
        try {
            ListObjectsV2Request build = ListObjectsV2Request.builder()
                    .bucket(bucketName)
                    .build();
            //因为单次查询只能查询1000条数据，所以需要循环查询
            ListObjectsV2Response listResponse;
            do {
                listResponse = s3Client.listObjectsV2(build);
                List<S3Object> objects = listResponse.contents();
                file.addAll(objects);
                //获取是否有下一页数据
                String token = listResponse.nextContinuationToken();
                build = build.toBuilder()
                        .continuationToken(token)
                        .build();
            } while (listResponse.isTruncated());
            log.info("获取文件成功~");
            return  file;
        } catch (AwsServiceException e) {
            e.printStackTrace();
        } catch (SdkClientException e) {
            e.printStackTrace();
        } finally {
            s3Client.close();
        }
        return null;
    }
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

以上就是对S3象存储对应的操作~