devbox
我家小花儿
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

猿创征文｜技术成长之路-【Java编程系列】文件存储实践：Amazon S3实现文件上传下载，总结坑点，积累成长经验_amazons3上传文件

作者：我家小花儿 | 2024-03-22 06:02:41

赞

踩

amazons3上传文件

热门系列：

【Java编程系列】Minio实现文件上传下载

1、前言

        最近有一客户，需要独立部署文件存储到自己的私有化服务器，于是选择了让我们使用在AWS服务器的s3存储桶方式来实现。

        在开发时间过程中，学习到了很多，也碰到了很多坑，感觉在io方面，成长了许多，特此记录一下积累的成长经验（详见【总结】部分），也希望可以帮助到有需要的朋友~~~

2、Amazon S3实战代码

2.1 S3的Sdk对接实现

2.1.1 S3 Maven依赖

        这里需要说明一下，目前封装好的java可用的sdk有两种实现依赖，第一种：

  1. <dependency>
  2.     <groupId>com.amazonaws</groupId>
  3.     <artifactId>aws-java-sdk-s3</artifactId>
  4.     <version>1.12.276</version>
  5. </dependency>

        以上是第一种封装依赖，但是这里面会有部分依赖缺失，在文末的总结中，可以看到完整的依赖方式。

        第二种封装依赖：

  1. <!--amazon s3-->
  2. <dependency>
  3.     <groupId>software.amazon.awssdk</groupId>
  4.     <artifactId>s3</artifactId>
  5.     <version>2.17.100</version>
  6. </dependency>
  7. <dependency>
  8.     <groupId>software.amazon.awssdk</groupId>
  9.     <artifactId>kms</artifactId>
  10.     <version>2.17.100</version>
  11. </dependency>
  12. <dependency>
  13.     <groupId>software.amazon.awssdk</groupId>
  14.     <artifactId>s3control</artifactId>
  15.     <version>2.17.100</version>
  16. </dependency>

        本来是使用第一种依赖来做的开发编码。后来因为一个403权限拒绝原因，一开始不确定是否是api问题，所以后来又使用了s3官方文档推荐的第二种依赖完成了开发实现，因此以下示例代码，全都是引用第二种maven依赖来完成的！！！


2.1.2 s3 client配置与初始化

        在开始实践编码前，先列一下本次开发中，我参考的一些api文档，希望能给有需要的朋友一些参考和帮助。

        S3官方开发指南文档：使用 AWS SDK for Java 的 Amazon S3示例 - AWS SDK for Java

        开发示例代码github地址：aws-doc-sdk-examples/javav2/example_code/s3/src/main/java/com/example/s3 at main · awsdocs/aws-doc-sdk-examples · GitHub

        还有第一种依赖的实现参考代码：如何使用AWS SDK for Java操作MinIO Server | Minio中文文档        

        s3存储桶的配置信息，我这边是使用数据表存储配置的，因为我们系统的多元化接入。可能会有多套配置，所以请按需配置。

        初始化代码如下，PrivateUploadConfig类是我的s3配置信息实体类：

  1. import lombok.extern.slf4j.Slf4j;
  2. import org.springframework.stereotype.Component;
  3. import org.springframework.web.multipart.MultipartFile;
  4. import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
  5. import software.amazon.awssdk.auth.credentials.AwsCredentials;
  6. import software.amazon.awssdk.auth.credentials.AwsCredentialsProvider;
  7. import software.amazon.awssdk.auth.credentials.AwsCredentialsProviderChain;
  8. import software.amazon.awssdk.core.sync.RequestBody;
  9. import software.amazon.awssdk.core.waiters.WaiterResponse;
  10. import software.amazon.awssdk.regions.Region;
  11. import software.amazon.awssdk.services.s3.S3Client;
  12. import software.amazon.awssdk.services.s3.model.CreateBucketRequest;
  13. import software.amazon.awssdk.services.s3.model.GetObjectRequest;
  14. import software.amazon.awssdk.services.s3.model.HeadBucketRequest;
  15. import software.amazon.awssdk.services.s3.model.HeadBucketResponse;
  16. import software.amazon.awssdk.services.s3.model.PutObjectRequest;
  17. import software.amazon.awssdk.services.s3.presigner.S3Presigner;
  18. import software.amazon.awssdk.services.s3.presigner.model.GetObjectPresignRequest;
  19. import software.amazon.awssdk.services.s3.presigner.model.PresignedGetObjectRequest;
  20. import software.amazon.awssdk.services.s3.waiters.S3Waiter;
  21.  
  22. import javax.annotation.Resource;
  23. import java.io.ByteArrayInputStream;
  24. import java.io.IOException;
  25. import java.io.InputStream;
  26. import java.time.Duration;
  27. import java.util.Objects;
  28.  
  29. /**
  30.  * @Date: 2022/8/19 19:40
  31.  * @Description
  32.  */
  33. @Component
  34. @Slf4j
  35. public class S3ClientUtil {
  36. 	private S3Client s3Client;
  37.  
  38. //	private S3AsyncClient s3AsyncClient;
  39.  
  40. 	@Resource(name = CommonBeanNames.STRING_REDIS_HELPER)
  41. 	private RedisHelper redisHelper;
  42. 	
  43. 	/**
  44. 	* @Description
  45. 	* @Date 19:53 2022/8/19
  46. 	* @Param [uploadConfig=s3配置信息实体类]
  47. 	* @return software.amazon.awssdk.auth.credentials.AwsCredentialsProviderChain
  48. 	**/
  49. 	private AwsCredentialsProviderChain getProvider(PrivateUploadConfig uploadConfig){
  50. 		return AwsCredentialsProviderChain
  51.         .builder()
  52.         .addCredentialsProvider(new AwsCredentialsProvider() {
  53.             @Override
  54.             public AwsCredentials resolveCredentials() {
  55.                 return AwsBasicCredentials.create(uploadConfig.getAccessKey(), uploadConfig.getSecretKey());
  56.             }
  57.         }).build();
  58. 	}
  59. 	
  60. 	/**
  61. 	 * @return void
  62. 	 * @Description
  63. 	 * @Date 19:47 2022/8/19
  64. 	 * @Param [uploadConfig=s3配置信息实体类]
  65. 	 **/
  66. 	private void init(PrivateUploadConfig uploadConfig) {
  67. 		if (null != s3Client) {
  68. 			return;
  69. 		}
  70.  
  71. 		s3Client = S3Client.builder()
  72. 				.credentialsProvider(getProvider(uploadConfig))
  73. 				.region(Region.of(uploadConfig.getRegion()))
  74. 				.build();
  75. 	}
  76. 	
  77. 	/**
  78. 	* @Description
  79. 	* @Date 19:53 2022/8/19
  80. 	* @Param [bucket=存储桶名称]
  81. 	* @return void
  82. 	**/
  83. 	public void makeBucket(String bucket){
  84. 		S3Waiter s3Waiter = s3Client.waiter();
  85.         CreateBucketRequest bucketRequest = CreateBucketRequest.builder()
  86.             .bucket(bucket)
  87.             .build();
  88.  
  89.         s3Client.createBucket(bucketRequest);
  90.         HeadBucketRequest bucketRequestWait = HeadBucketRequest.builder()
  91.             .bucket(bucket)
  92.             .build();
  93.  
  94.         // Wait until the bucket is created and print out the response.
  95.         WaiterResponse<HeadBucketResponse> waiterResponse = s3Waiter.waitUntilBucketExists(bucketRequestWait);
  96.         waiterResponse.matched().response().ifPresent(r->log.info(r.toString()));
  97. 	}
  98.  
  99. }

2.1.3 上传文件

  1. import lombok.extern.slf4j.Slf4j;
  2. import org.springframework.stereotype.Component;
  3. import org.springframework.web.multipart.MultipartFile;
  4. import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
  5. import software.amazon.awssdk.auth.credentials.AwsCredentials;
  6. import software.amazon.awssdk.auth.credentials.AwsCredentialsProvider;
  7. import software.amazon.awssdk.auth.credentials.AwsCredentialsProviderChain;
  8. import software.amazon.awssdk.core.sync.RequestBody;
  9. import software.amazon.awssdk.core.waiters.WaiterResponse;
  10. import software.amazon.awssdk.regions.Region;
  11. import software.amazon.awssdk.services.s3.S3Client;
  12. import software.amazon.awssdk.services.s3.model.CreateBucketRequest;
  13. import software.amazon.awssdk.services.s3.model.GetObjectRequest;
  14. import software.amazon.awssdk.services.s3.model.HeadBucketRequest;
  15. import software.amazon.awssdk.services.s3.model.HeadBucketResponse;
  16. import software.amazon.awssdk.services.s3.model.PutObjectRequest;
  17. import software.amazon.awssdk.services.s3.presigner.S3Presigner;
  18. import software.amazon.awssdk.services.s3.presigner.model.GetObjectPresignRequest;
  19. import software.amazon.awssdk.services.s3.presigner.model.PresignedGetObjectRequest;
  20. import software.amazon.awssdk.services.s3.waiters.S3Waiter;
  21.  
  22. import javax.annotation.Resource;
  23. import java.io.ByteArrayInputStream;
  24. import java.io.IOException;
  25. import java.io.InputStream;
  26. import java.time.Duration;
  27. import java.util.Objects;
  28.  
  29. /**
  30.  * @Date: 2022/8/19 19:40
  31.  * @Description
  32.  */
  33. @Component
  34. @Slf4j
  35. public class S3ClientUtil {
  36. 	private S3Client s3Client;
  37.  
  38. //	private S3AsyncClient s3AsyncClient;
  39.  
  40. 	@Resource(name = CommonBeanNames.STRING_REDIS_HELPER)
  41. 	private RedisHelper redisHelper;
  42.  
  43.  
  44. 	public void uploadFile(PrivateUploadConfig uploadConfig, String fileName, MultipartFile multipartFile, String contentType)
  45. 			throws IOException {
  46. 		if (null == s3Client) {
  47. 			init(uploadConfig);
  48. 		}
  49.  
  50. 		PutObjectRequest.Builder putOb = PutObjectRequest.builder();
  51. 		putOb.bucket(uploadConfig.getBucket());
  52. 		putOb.key(fileName);
  53. 		if (StringUtils.isNotEmpty(contentType)) {
  54. 			putOb.contentType(contentType);
  55. 		}
  56.  
  57. 		long begin = System.currentTimeMillis();
  58. 		s3Client.putObject(putOb.build(), RequestBody.fromInputStream(multipartFile.getInputStream(), multipartFile.getInputStream().available()));
  59. 		log.info("S3 putObject time :{}", System.currentTimeMillis() - begin);
  60. 	}
  61.  
  62. }

2.1.4 下载文件

  1. import lombok.extern.slf4j.Slf4j;
  2. import org.springframework.stereotype.Component;
  3. import org.springframework.web.multipart.MultipartFile;
  4. import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
  5. import software.amazon.awssdk.auth.credentials.AwsCredentials;
  6. import software.amazon.awssdk.auth.credentials.AwsCredentialsProvider;
  7. import software.amazon.awssdk.auth.credentials.AwsCredentialsProviderChain;
  8. import software.amazon.awssdk.core.sync.RequestBody;
  9. import software.amazon.awssdk.regions.Region;
  10. import software.amazon.awssdk.services.s3.S3Client;
  11. import software.amazon.awssdk.services.s3.model.GetObjectRequest;
  12. import software.amazon.awssdk.services.s3.model.PutObjectRequest;
  13. import software.amazon.awssdk.services.s3.presigner.S3Presigner;
  14. import software.amazon.awssdk.services.s3.presigner.model.GetObjectPresignRequest;
  15. import software.amazon.awssdk.services.s3.presigner.model.PresignedGetObjectRequest;
  16.  
  17. import javax.annotation.Resource;
  18. import java.io.ByteArrayInputStream;
  19. import java.io.IOException;
  20. import java.io.InputStream;
  21. import java.time.Duration;
  22. import java.util.Objects;
  23.  
  24. /**
  25.  * @Date: 2022/8/19 19:40
  26.  * @Description
  27.  */
  28. @Component
  29. @Slf4j
  30. public class S3ClientUtil {
  31. 	private S3Client s3Client;
  32.  
  33. //	private S3AsyncClient s3AsyncClient;
  34.  
  35. 	private InputStream amazonS3Download(PrivateUploadConfig uploadConfig, String fileName) {
  36.  
  37. 		if (null == s3Client) {
  38. 			init(uploadConfig);
  39. 		}
  40.  
  41. 		//此处需要注意，如果fileName第一位有/，则需要剔除掉，否则会报错找不到
  42. 		if (fileName.indexOf("/") == 0) {
  43. 			fileName = fileName.replaceFirst("/", "");
  44. 		}
  45.  
  46. 		GetObjectRequest objectRequest = GetObjectRequest
  47. 				.builder()
  48. 				.key(fileName)
  49. 				.bucket(uploadConfig.getBucket())
  50. 				.build();
  51.  
  52. 		long begin = System.currentTimeMillis();
  53. 		InputStream is = s3Client.getObject(objectRequest);
  54. 		log.info("S3 getObject time :{}", System.currentTimeMillis() - begin);
  55.  
  56. 		return is;
  57. //		byte [] bytes = s3AsyncClient.getObject(objectRequest,AsyncResponseTransformer.toBytes()).thenApply(BytesWrapper::asByteArray).join();
  58. //		return new ByteArrayInputStream(bytes);
  59.  
  60. 	}
  61.  
  62. }

2.1.5 生成文件预签名url

  1. import lombok.extern.slf4j.Slf4j;
  2. import org.springframework.stereotype.Component;
  3. import org.springframework.web.multipart.MultipartFile;
  4. import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
  5. import software.amazon.awssdk.auth.credentials.AwsCredentials;
  6. import software.amazon.awssdk.auth.credentials.AwsCredentialsProvider;
  7. import software.amazon.awssdk.auth.credentials.AwsCredentialsProviderChain;
  8. import software.amazon.awssdk.core.sync.RequestBody;
  9. import software.amazon.awssdk.regions.Region;
  10. import software.amazon.awssdk.services.s3.S3Client;
  11. import software.amazon.awssdk.services.s3.model.GetObjectRequest;
  12. import software.amazon.awssdk.services.s3.model.PutObjectRequest;
  13. import software.amazon.awssdk.services.s3.presigner.S3Presigner;
  14. import software.amazon.awssdk.services.s3.presigner.model.GetObjectPresignRequest;
  15. import software.amazon.awssdk.services.s3.presigner.model.PresignedGetObjectRequest;
  16.  
  17. import javax.annotation.Resource;
  18. import java.io.ByteArrayInputStream;
  19. import java.io.IOException;
  20. import java.io.InputStream;
  21. import java.time.Duration;
  22. import java.util.Objects;
  23.  
  24. /**
  25.  * @Date: 2022/8/19 19:40
  26.  * @Description
  27.  */
  28. @Component
  29. @Slf4j
  30. public class S3ClientUtil {
  31. 	private S3Client s3Client;
  32.  
  33. //	private S3AsyncClient s3AsyncClient;
  34.  
  35. 	@Resource(name = CommonBeanNames.STRING_REDIS_HELPER)
  36. 	private RedisHelper redisHelper;
  37.  
  38.  
  39. 	public String getObjectURL(PrivateUploadConfig uploadConfig, String fileName) {
  40. 		if (fileName.indexOf("/") == 0) {
  41. 			fileName = fileName.replaceFirst("/", "");
  42. 		}
  43.  
  44. 		S3Presigner presigner = S3Presigner.builder()
  45. 				.credentialsProvider(getProvider(uploadConfig))
  46. 				.region(Region.of(uploadConfig.getRegion()))
  47. 				.build();
  48.  
  49. 		GetObjectRequest getObjectRequest = GetObjectRequest.builder()
  50. 				.bucket(uploadConfig.getBucket())
  51. 				.key(fileName)
  52. 				.build();
  53.  
  54. 		//默认生成的预览url最大时效时间为：7天，aws s3能允许的最大时限也是7天
  55. 		GetObjectPresignRequest getObjectPresignRequest = GetObjectPresignRequest.builder()
  56. 				.signatureDuration(Duration.ofDays(7))
  57. 				.getObjectRequest(getObjectRequest)
  58. 				.build();
  59.  
  60. 		PresignedGetObjectRequest presignedGetObjectRequest = presigner.presignGetObject(getObjectPresignRequest);
  61. 		String theUrl = presignedGetObjectRequest.url().toString();
  62. 		log.info("Origin path :{} , Presigned URL: {}", fileName, theUrl);
  63.  
  64. 		return theUrl;
  65. 	}
  66.  
  67. }

2.1.6 文件“永久性”访问实现

        其实S3存储桶，通过策略配置，是可以直接通过 域名+桶名称+文件路径名称 ，来直接永久访问到的。但是，这么做的话，安全性就大大降低了。所以，我们需要采取其他的方式来间接实现永久访问的功能。于是，有了一下曲线救国的方针：

        通过定时任务，根据你生成预签名url的最大时效来，往后推一点时间，来生成需要更新的时间周期。

        例如，我选择最大时效为7天，那么我会定时任务，每天查询图片url时效已达过6天的记录，然后重新生成新的url更新到记录里，如此，就可以间接性的永久有效啦~~~

2.1.7 其他操作（解决下载慢）

        还有很多其他api，如删除文件、获取存储桶列表、获取文件列表；另外还有一些，针对不同region或是高版本api，还有transfer等高效高性能方法可用。

        另外，使用第一种依赖，还能实现一些分段上传、下载。官方建议，当单个文件大小超过100M时，则需要使用分段上传下载处理。

        但是，我这里遇到了一个其他问题：上传和下载很慢的问题 ，这个问题我分析主要是来自于网络传输效率低所导致，不在代码层面。测试用例：一个10M左右的文件，上传需要10+秒，下载更是达到60秒左右。

        得出以上结论的原因有：

1、同样的s3实现，我们其他使用aws服务器的朋友，他们使用上传下载时，就没有这个慢的情况出现，至少不会像我这个情况。

2、我使用本地文件，对代码进行测试，除了s3的api调用处，其他代码运行效率都很高，只在毫秒级别，使用的是同一个大小的原文件测试的。

3、使用断点调试时，发现s3Client.putObject(）方法，阻塞所占时间为主要耗时；另外，对于s3Client.getObject(）下载方法，虽然程序调用在毫秒级，但最终将文件流输出时，底层实际是通过s3服务端的url实时传输的，所以也是传出效率低所导致！

        最终，我先想了一个方式，解决下载慢的问题。因为一个文件下载要1min多，实在是太不能接受了。上传10s左右，还得过且过吧。

        对于下载慢的问题，我使用的方式时，在上传完成后，使用redis缓存文件流来解决！

  1. String fis = new String(multipartFile.getBytes(),"ISO8859-1");
  2. redisHelper.hset(cacheKey,fileId,fis);

        以上是缓存实现的关键代码。对于为什么字符串编码选择使用 ISO8859-1 ，因为也有与base64转码做过性能对比，发现前者的性能更优。大家可以参考使用！

3、总结

         整体实践完成后，有一些开发过程中遇到的问题，在这里总结一下。帮助大家避坑~~~

3.1 异常报错

错误一：关键依赖类缺失（EndpointToRegion类找不到）

        运行时异常主要信息为：

Caused by: java.lang.ClassNotFoundException: com.amazonaws.regions.EndpointToRegion

        全部报错信息如下：

  1. org.springframework.web.util.NestedServletException: Handler dispatch failed; nested exception is java.lang.NoClassDefFoundError: com/amazonaws/regions/EndpointToRegion
  2. 	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1055)
  3. 	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943)
  4. 	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
  5. 	at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
  6. 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:660)
  7. 	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
  8. 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
  9. 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
  10. 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
  11. 	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
  12. 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
  13. 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
  14. 	at ***.GzipFilter.doFilter(GzipFilter.java:47)
  15. 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
  16. 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
  17. 	at ***.AbstractCorsFilter.doFilter(AbstractCorsFilter.java:63)
  18. 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
  19. 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
  20. 	at ***.ChannelFilter.doFilter(ChannelFilter.java:33)
  21. 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
  22. 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
  23. 	at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
  24. 	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
  25. 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
  26. 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
  27. 	at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
  28. 	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
  29. 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
  30. 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
  31. 	at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:109)
  32. 	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
  33. 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
  34. 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
  35. 	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
  36. 	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
  37. 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
  38. 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
  39. 	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
  40. 	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
  41. 	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
  42. 	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
  43. 	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
  44. 	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
  45. 	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
  46. 	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:367)
  47. 	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
  48. 	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
  49. 	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1639)
  50. 	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
  51. 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  52. 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  53. 	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
  54. 	at java.lang.Thread.run(Thread.java:748)
  55. Caused by: java.lang.NoClassDefFoundError: com/amazonaws/regions/EndpointToRegion
  56. 	at com.amazonaws.services.s3.internal.auth.S3SignerProvider.getSigner(S3SignerProvider.java:62)
  57. 	at com.amazonaws.http.ExecutionContext.getSigner(ExecutionContext.java:135)
  58. 	at com.amazonaws.http.AmazonHttpClient$RequestExecutor$ExecOneRequestParams.newSigner(AmazonHttpClient.java:1843)
  59. 	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1216)
  60. 	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1074)
  61. 	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:745)
  62. 	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:719)
  63. 	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:701)
  64. 	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:669)
  65. 	at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:651)
  66. 	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:515)
  67. 	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5456)
  68. 	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5403)
  69. 	at com.amazonaws.services.s3.AmazonS3Client.getAcl(AmazonS3Client.java:4062)
  70. 	at com.amazonaws.services.s3.AmazonS3Client.getBucketAcl(AmazonS3Client.java:1278)
  71. 	at com.amazonaws.services.s3.AmazonS3Client.getBucketAcl(AmazonS3Client.java:1268)
  72. 	at com.amazonaws.services.s3.AmazonS3Client.doesBucketExistV2(AmazonS3Client.java:1406)
  73. 	at ***.***.PrivateMinioClientHelper.makeBucket(PrivateMinioClientHelper.java:56)
  74. 	at ***.***.PrivateMinioClientHelper.amazonS3Init(PrivateMinioClientHelper.java:140)
  75. 	at ***.***.PrivateMinioClientHelper.init(PrivateMinioClientHelper.java:110)
  76. 	at ***.***.PrivateMinioClientHelper.amazonS3Upload(PrivateMinioClientHelper.java:163)
  77. 	at ***.***.PrivateMinioClientHelper.uploadFile(PrivateMinioClientHelper.java:72)
  78. 	at ***.***.service.impl.PrivateMinioServiceImpl.uploadFile(PrivateMinioServiceImpl.java:79)
  79. 	at ***.***.service.impl.AbstractFileServiceImpl.upload(AbstractFileServiceImpl.java:269)
  80. 	at ***.***.controller.FileController.upload(FileController.java:108)
  81. 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  82. 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  83. 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  84. 	at java.lang.reflect.Method.invoke(Method.java:498)
  85. 	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190)
  86. 	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138)
  87. 	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:106)
  88. 	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:879)
  89. 	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:793)
  90. 	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
  91. 	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040)
  92. 	... 52 common frames omitted
  93. Caused by: java.lang.ClassNotFoundException: com.amazonaws.regions.EndpointToRegion
  94. 	at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
  95. 	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
  96. 	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
  97. 	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
  98. 	... 88 common frames omitted

解决方式：因为缺少部分依赖，只需要引入以下maven依赖即可

  1. <dependency>
  2. 	<groupId>com.amazonaws</groupId>
  3. 	<artifactId>aws-java-sdk-s3</artifactId>
  4. 	<version>1.12.276</version>
  5. </dependency>
  6. <dependency>
  7. 	<groupId>com.amazonaws</groupId>
  8. 	<artifactId>aws-java-sdk-kms</artifactId>
  9. 	<version>1.12.276</version>
  10. </dependency>
  11. <!-- https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-core -->
  12. <dependency>
  13. 	<groupId>com.amazonaws</groupId>
  14. 	<artifactId>aws-java-sdk-core</artifactId>
  15. 	<version>1.12.276</version>
  16. </dependency>
  17. <!-- https://mvnrepository.com/artifact/com.amazonaws/jmespath-java -->
  18. <dependency>
  19. 	<groupId>com.amazonaws</groupId>
  20. 	<artifactId>jmespath-java</artifactId>
  21. 	<version>1.12.276</version>
  22. </dependency>

错误二：权限配置问题，拒绝访问

        另一个运行异常报错信息如下：

com.amazonaws.services.s3.model.AmazonS3Exception: Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: 7GR03AT1BWSMPNYM; S3 Extended Request ID: 8fu4lS0r0N6hNhKQbqunAnkmx3HxxRwjqA9oHobv61P7U8vYra23L4MuNlfC6sKjdFr73U1EmX4=; Proxy: null)

        整体错误日志如下：

  1. com.amazonaws.services.s3.model.AmazonS3Exception: Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: 7GR03AT1BWSMPNYM; S3 Extended Request ID: 8fu4lS0r0N6hNhKQbqunAnkmx3HxxRwjqA9oHobv61P7U8vYra23L4MuNlfC6sKjdFr73U1EmX4=; Proxy: null)
  2. 	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1879)
  3. 	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleServiceErrorResponse(AmazonHttpClient.java:1418)
  4. 	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1387)
  5. 	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1157)
  6. 	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:814)
  7. 	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:781)
  8. 	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:755)
  9. 	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:715)
  10. 	at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:697)
  11. 	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:561)
  12. 	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:541)
  13. 	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5456)
  14. 	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5403)
  15. 	at com.amazonaws.services.s3.AmazonS3Client.access$300(AmazonS3Client.java:421)
  16. 	at com.amazonaws.services.s3.AmazonS3Client$PutObjectStrategy.invokeServiceCall(AmazonS3Client.java:6531)
  17. 	at com.amazonaws.services.s3.AmazonS3Client.uploadObject(AmazonS3Client.java:1861)
  18. 	at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1821)
  19. 	at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1753)
  20. 	at ***.helper.PrivateMinioClientHelper.amazonS3Upload(PrivateMinioClientHelper.java:174)
  21. 	at ***.helper.PrivateMinioClientHelper.uploadFile(PrivateMinioClientHelper.java:73)
  22. 	at ***.service.impl.PrivateMinioServiceImpl.uploadFile(PrivateMinioServiceImpl.java:79)
  23. 	at ***.service.impl.AbstractFileServiceImpl.upload(AbstractFileServiceImpl.java:269)
  24. 	at ***.controller.FileController.upload(FileController.java:108)
  25. 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  26. 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  27. 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  28. 	at java.lang.reflect.Method.invoke(Method.java:498)
  29. 	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190)
  30. 	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138)
  31. 	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:106)
  32. 	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:879)
  33. 	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:793)
  34. 	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
  35. 	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040)
  36. 	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943)
  37. 	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
  38. 	at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
  39. 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:660)
  40. 	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
  41. 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
  42. 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
  43. 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
  44. 	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
  45. 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
  46. 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
  47. 	at ***.GzipFilter.doFilter(GzipFilter.java:47)
  48. 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
  49. 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
  50. 	at ***.AbstractCorsFilter.doFilter(AbstractCorsFilter.java:63)
  51. 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
  52. 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
  53. 	at ***.ChannelFilter.doFilter(ChannelFilter.java:33)
  54. 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
  55. 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
  56. 	at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
  57. 	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
  58. 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
  59. 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
  60. 	at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
  61. 	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
  62. 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
  63. 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
  64. 	at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:109)
  65. 	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
  66. 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
  67. 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
  68. 	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
  69. 	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
  70. 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
  71. 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
  72. 	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
  73. 	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
  74. 	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
  75. 	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
  76. 	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
  77. 	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
  78. 	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
  79. 	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:367)
  80. 	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
  81. 	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
  82. 	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1639)
  83. 	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
  84. 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  85. 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  86. 	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
  87. 	at java.lang.Thread.run(Thread.java:748)
  88. 2022-08-10 10:37:52.535 [5m7a7kaVC5xwggeT] ERROR (GlobalExceptionHandler.java:135) - custom exception: code: 30006, message: null, data:null, Exception: ***.service.impl.PrivateMinioServiceImpl.uploadFile(PrivateMinioServiceImpl.java:89)
  89. 		***.service.impl.AbstractFileServiceImpl.upload(AbstractFileServiceImpl.java:269)
  90. 		***.controller.FileController.upload(FileController.java:108)
  91. 		sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  92. 		sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  93. 		sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  94. 		java.lang.reflect.Method.invoke(Method.java:498)
  95. 		org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190)
  96. 		org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138)
  97. 		org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:106)

这个问题，提示已经很明显了，就是s3服务端的拒绝通过啦。当时，还一阵以为少了什么配置所致，其实不然，就是权限配置的问题。

解决方式：1、将IAM权限设置好，对指定账户放开哪些权限，如读写等等

                  2、将指定存储桶的权限放开，保证该桶对指定用户可以上传、下载、删除等等

         分享内容就到这里啦，欢迎大家留言探讨。希望自我成长的同时，也能帮助到有需要的童靴！有问题，也会第一时间回复~~

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/我家小花儿/article/detail/285757
推荐阅读
相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。

  

闽ICP备14008679号