devbox
很楠不爱3
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

Amazon S3 API操作示例-java版_java s3 样例

作者：很楠不爱3 | 2024-03-22 06:02:05

赞

踩

java s3 样例

     amazon (S3) 是一个公开的服务，Web 应用程序开发人员可以使用它存储数字资产，包括图片、视频、音乐和文档等。除去aws本身的服务可以将一些内容直接存储到s3桶之外 ，在很多情况下，还需要使用s3桶和外部服务进行交互，比如下面一些场景：

  • 场景一：配置使用aws服务产生的账单费用以文件的形式定期发送到s3桶，本地服务获取这些文件，生成新的账单或进行费用监控。
  • 场景二：配置使用aws服务产生的日志文件定期发送到s3桶，本地服务获取这些文件，做数据分析，预防风险。
  • 场景三：将s3桶当做无限量永久文件存储器，定期将文件上传到s3桶。

     无论哪种场景，使用控制台手动同步这些文件显然不是明智的选择，那么就需要用到Amazon提供的各种API来实现这些操作。下面是使用java对s3桶操作的一些示例，需要的小伙伴可以使用cv大法啦。
     首先引入java版的sdk。如果是本地开发，一般可以只引入aws-java-sdk一个jar包，它包含了所有aws服务的包。而通常我们只使用aws服务的一种或几种，那么我们可以单独引入使用的相关服务的包。其中kms和core是所有服务包的依赖包，为减少因版本可能带来的问题，建议也进行显式引入。这里我选择的版本是：1.11.538。

pom.xml

  1. <!-- aws -->
  2. <dependency>
  3.     <groupId>com.amazonaws</groupId>
  4.     <artifactId>aws-java-sdk-core</artifactId>
  5.     <version>${awsjavasdk.version}</version>
  6.     <scope>compile</scope>
  7. </dependency>
  8. <dependency>
  9.     <groupId>com.amazonaws</groupId>
  10.     <artifactId>aws-java-sdk-kms</artifactId>
  11.     <version>${awsjavasdk.version}</version>
  12.     <scope>compile</scope>
  13. </dependency>
  14. <dependency>
  15.     <groupId>com.amazonaws</groupId>
  16.     <artifactId>aws-java-sdk-s3</artifactId>
  17.     <version>${awsjavasdk.version}</version>
  18.     <scope>compile</scope>
  19. </dependency>

 

连接awsAPI基本信息类，替换其中的ak、sk和区域。

AwsBaseClient

  1. package com.study.awsbase;
  2.  
  3. import com.amazonaws.regions.Regions;
  4.  
  5. /**
  6.  * 连接aws的api的基本信息
  7.  *
  8.  * @author guoj
  9.  * @date 2019年10月8日 10:32:17
  10.  */
  11. public class AwsBaseClient {
  12.  
  13.     /**
  14.      * Access key ID
  15.      */
  16.     protected String awsAccessKey = "ak";
  17.     /**
  18.      * Secret access key
  19.      */
  20.     protected String awsSecretKey = "sk";
  21.     /**
  22.      * 区域
  23.      */
  24.     protected Regions regions = Regions.CN_NORTH_1;
  25.  
  26.     public AwsBaseClient() {
  27.     }
  28.  
  29.     public AwsBaseClient(String awsAccessKey, String awsSecretKey, Regions regions) {
  30.         this.awsAccessKey = awsAccessKey;
  31.         this.awsSecretKey = awsSecretKey;
  32.         this.regions = regions;
  33.     }
  34.  
  35.     public void setBaseClient(String awsAccessKey, String awsSecretKey, Regions regions) {
  36.         this.awsAccessKey = awsAccessKey;
  37.         this.awsSecretKey = awsSecretKey;
  38.         this.regions = regions;
  39.     }
  40.  
  41.     public AwsBaseClient withAwsAccessKey(String awsAccessKey) {
  42.         this.awsAccessKey = awsAccessKey;
  43.         return this;
  44.     }
  45.  
  46.     public AwsBaseClient withAwsSecretKey(String awsSecretKey) {
  47.         this.awsSecretKey = awsSecretKey;
  48.         return this;
  49.     }
  50.  
  51.     public AwsBaseClient withRegions(Regions regions) {
  52.         this.regions = regions;
  53.         return this;
  54.     }
  55. }

    获取连接s3桶客户端的类。这里提供了供子类获取默认s3客户端方法，供子类获取自定义s3客户端的方法，代外部类获取s3客户端的默认以自定义方法。

AwsS3Client

  1. package com.study.s3;
  2.  
  3. import com.amazonaws.auth.AWSStaticCredentialsProvider;
  4. import com.amazonaws.auth.BasicAWSCredentials;
  5. import com.amazonaws.regions.Regions;
  6. import com.amazonaws.services.s3.AmazonS3;
  7. import com.amazonaws.services.s3.AmazonS3ClientBuilder;
  8. import com.study.awsbase.AwsBaseClient;
  9.  
  10. /**
  11.  * 连接s3
  12.  *
  13.  * @author guoj
  14.  * @date 2019年10月8日 10:32:17
  15.  */
  16. public class AwsS3Client extends AwsBaseClient {
  17.  
  18.     protected AmazonS3 s3;
  19.  
  20.     public AwsS3Client() {
  21.         super();
  22.         BasicAWSCredentials awsCreds = new BasicAWSCredentials(awsAccessKey, awsSecretKey);
  23.         this.s3 = AmazonS3ClientBuilder.standard().withRegion(regions).withCredentials(new AWSStaticCredentialsProvider(awsCreds)).build();
  24.     }
  25.  
  26.     public AwsS3Client(String awsAccessKey, String awsSecretKey, Regions regions) {
  27.         super(awsAccessKey, awsSecretKey, regions);
  28.         BasicAWSCredentials awsCreds = new BasicAWSCredentials(awsAccessKey, awsSecretKey);
  29.         this.s3 = AmazonS3ClientBuilder.standard().withRegion(regions).withCredentials(new AWSStaticCredentialsProvider(awsCreds)).build();
  30.     }
  31.  
  32.     public void setS3Client(String awsAccessKey, String awsSecretKey, Regions regions) {
  33.         setBaseClient(awsAccessKey, awsSecretKey, regions);
  34.         BasicAWSCredentials awsCreds = new BasicAWSCredentials(awsAccessKey, awsSecretKey);
  35.         this.s3 = AmazonS3ClientBuilder.standard().withRegion(regions).withCredentials(new AWSStaticCredentialsProvider(awsCreds)).build();
  36.     }
  37.  
  38.     public AmazonS3 getS3Client() {
  39.         return s3;
  40.     }
  41.  
  42.     public AmazonS3 getS3Client(String awsAccessKey, String awsSecretKey, Regions regions) {
  43.         BasicAWSCredentials awsCreds = new BasicAWSCredentials(awsAccessKey, awsSecretKey);
  44.         s3 = AmazonS3ClientBuilder.standard().withRegion(regions).withCredentials(new AWSStaticCredentialsProvider(awsCreds)).build();
  45.         return s3;
  46.     }
  47.  
  48. }
s3桶常用操作类。

AwsS3Bucket

  1. package com.study.s3;
  2.  
  3. import com.amazonaws.regions.Regions;
  4. import com.amazonaws.services.s3.model.*;
  5. import com.amazonaws.services.s3.model.analytics.AnalyticsConfiguration;
  6.  
  7. import java.util.ArrayList;
  8. import java.util.List;
  9.  
  10. /**
  11.  * s3桶常用操作
  12.  *
  13.  * @author guoj
  14.  * @date 2019年10月8日 11:06:46
  15.  */
  16. public class AwsS3Bucket extends AwsS3Client {
  17.  
  18.     /**
  19.      * s3桶名称
  20.      * 全球唯一，如果是中国区域，则中国区域唯一
  21.      */
  22.     private String bucketName = "sinnet-test-bucket-001";
  23.     /**
  24.      * s3区域
  25.      */
  26.     private String region = Regions.CN_NORTH_1.getName();
  27.  
  28.     /**
  29.      * 列出所有的s3桶
  30.      */
  31.     public void listBuckets() {
  32.         List<Bucket> buckets = s3.listBuckets();
  33.         buckets.forEach(item -> {
  34.             System.out.println(item.toString());
  35.         });
  36.     }
  37.  
  38.     /**
  39.      * 创建一个s3桶-默认
  40.      */
  41.     public void createBucketBase() {
  42.         Bucket bucket = s3.createBucket(bucketName);
  43.         System.out.println(bucket.toString());
  44.     }
  45.  
  46.     /**
  47.      * 创建一个s3桶-带参数
  48.      */
  49.     public void createBucketWithParams() {
  50.         //指定名称和区域
  51.         CreateBucketRequest createBucketRequest = new CreateBucketRequest(bucketName, region);
  52.         //是否启用对象锁-启用后，阻止删除对象
  53.         createBucketRequest.setObjectLockEnabledForBucket(true);
  54.         Bucket bucket = s3.createBucket(createBucketRequest);
  55.         System.out.println(bucket.toString());
  56.     }
  57.  
  58.     /**
  59.      * 删除一个s3桶
  60.      */
  61.     public void deleteBucket() {
  62.         DeleteBucketRequest deleteBucketRequest = new DeleteBucketRequest(bucketName);
  63.         s3.deleteBucket(deleteBucketRequest);
  64.         System.out.println("delete bucket success");
  65.     }
  66.  
  67.     /**
  68.      * s3桶配置
  69.      */
  70.     public void configBucket() {
  71.         /**
  72.          * 加速配置
  73.          */
  74.         BucketAccelerateConfiguration bucketAccelerateConfiguration = new BucketAccelerateConfiguration(BucketAccelerateStatus.Enabled);
  75.         s3.setBucketAccelerateConfiguration(bucketName, bucketAccelerateConfiguration);
  76.         /**
  77.          * 权限配置
  78.          */
  79.         //公共访问权限
  80.         SetBucketAclRequest setBucketAclRequest = new SetBucketAclRequest(bucketName, CannedAccessControlList.Private);
  81.         s3.setBucketAcl(setBucketAclRequest);
  82.         //访问控制列表
  83.         AccessControlList accessControlList = new AccessControlList();
  84.         accessControlList.setRequesterCharged(true);
  85.         accessControlList.setOwner(null);
  86.         SetBucketAclRequest setBucketAclRequest2 = new SetBucketAclRequest(bucketName, accessControlList);
  87.         s3.setBucketAcl(setBucketAclRequest2);
  88.         /**
  89.          * 分析配置
  90.          */
  91.         AnalyticsConfiguration analyticsConfiguration = new AnalyticsConfiguration();
  92.         analyticsConfiguration.setId(null);
  93.         SetBucketAnalyticsConfigurationRequest setBucketAnalyticsConfigurationRequest = new SetBucketAnalyticsConfigurationRequest(bucketName, analyticsConfiguration);
  94.         s3.setBucketAnalyticsConfiguration(setBucketAnalyticsConfigurationRequest);
  95.         /**
  96.          * 生命周期配置
  97.          */
  98.         BucketLifecycleConfiguration bucketLifecycleConfiguration = new BucketLifecycleConfiguration();
  99.         List<BucketLifecycleConfiguration.Rule> rules = new ArrayList();
  100.         //需要预先制定规则
  101.         BucketLifecycleConfiguration.Rule rule = new BucketLifecycleConfiguration.Rule().withId(null);
  102.         rules.add(rule);
  103.         bucketLifecycleConfiguration.setRules(rules);
  104.         SetBucketLifecycleConfigurationRequest setBucketLifecycleConfigurationRequest = new SetBucketLifecycleConfigurationRequest(bucketName, bucketLifecycleConfiguration);
  105.         s3.setBucketLifecycleConfiguration(setBucketLifecycleConfigurationRequest);
  106.         /**
  107.          * 加密配置
  108.          * 当对象存储在s3时默认是加密的
  109.          */
  110.         SetBucketEncryptionRequest setBucketEncryptionRequest = new SetBucketEncryptionRequest();
  111.         setBucketEncryptionRequest.setBucketName(bucketName);
  112.         ServerSideEncryptionConfiguration serverSideEncryptionConfiguration = new ServerSideEncryptionConfiguration();
  113.         //同样，需要预先制定规则
  114.         serverSideEncryptionConfiguration.setRules(null);
  115.         setBucketEncryptionRequest.setServerSideEncryptionConfiguration(serverSideEncryptionConfiguration);
  116.         s3.setBucketEncryption(setBucketEncryptionRequest);
  117.         /**
  118.          * 版本控制配置
  119.          */
  120.         BucketVersioningConfiguration bucketVersioningConfiguration = new BucketVersioningConfiguration();
  121.         bucketVersioningConfiguration.setMfaDeleteEnabled(true);
  122.         bucketVersioningConfiguration.setStatus(BucketVersioningConfiguration.ENABLED);
  123.         SetBucketVersioningConfigurationRequest setBucketVersioningConfigurationRequest = new SetBucketVersioningConfigurationRequest(bucketName, bucketVersioningConfiguration);
  124.         s3.setBucketVersioningConfiguration(setBucketVersioningConfigurationRequest);
  125.  
  126.         /**
  127.          * 为s3指定一个策略-s3的策略是唯一的
  128.          */
  129.         s3.setBucketPolicy(null);
  130.         /**
  131.          * 日志记录配置
  132.          */
  133.         s3.setBucketLoggingConfiguration(null);
  134.         /**
  135.          * 通知配置
  136.          */
  137.         s3.setBucketNotificationConfiguration(null);
  138.         /**
  139.          * 复制配置
  140.          */
  141.         s3.setBucketReplicationConfiguration(null);
  142.         /**
  143.          * 标签配置
  144.          */
  145.         s3.setBucketTaggingConfiguration(null);
  146.         /**
  147.          * 静态网站托管配置
  148.          */
  149.         s3.setBucketWebsiteConfiguration(null);
  150.         /**
  151.          * 指标配置
  152.          */
  153.         s3.setBucketMetricsConfiguration(null);
  154.  
  155.     }
  156.  
  157. }

s3桶文件操作类：

AwsS3File

  1. package com.study.s3;
  2.  
  3. import com.amazonaws.services.s3.model.*;
  4.  
  5. import java.io.*;
  6. import java.util.List;
  7.  
  8. /**
  9.  * s3桶文件的常用操作
  10.  *
  11.  * @author guoj
  12.  * @date 2019年10月8日 14:30:36
  13.  */
  14. public class AwsS3File extends AwsS3Client {
  15.  
  16.     /**
  17.      * s3桶名称
  18.      */
  19.     private String bucketName = "sinnet-test-bucket-001";
  20.     /**
  21.      * 上传到s3桶后的文件名
  22.      */
  23.     private String key = "test-folder/sinnet-test-file.txt";
  24.     /**
  25.      * 本地文件
  26.      */
  27.     private String localFilePath = "e:/test/s3/";
  28.     private String localFileName = "1.txt";
  29.     private String localFilePathName = localFilePath + localFileName;
  30.  
  31.     /**
  32.      * 列出一个s3桶内的文件
  33.      */
  34.     public void listFiles() {
  35.         //可以加入文件名前缀当作搜索条件
  36.         String prefix = null;
  37.         ObjectListing objectListing = s3.listObjects(bucketName, prefix);
  38.         List<S3ObjectSummary> s3ObjectSummaryList = objectListing.getObjectSummaries();
  39.         s3ObjectSummaryList.forEach(item -> {
  40.             //文件路径及名称-如果是文件夹，以"/"标识路径
  41.             System.out.print(item.getKey() + ",");
  42.             //文件UUID
  43.             System.out.println(item.getETag());
  44.         });
  45.     }
  46.  
  47.     /**
  48.      * 上传一个文件到s3桶
  49.      * 注意1：如果key相同，则替换
  50.      * 注意2：如果key是一个层次路径，那么s3会自动创建相应文件夹
  51.      */
  52.     public void uploadFile() {
  53.         File file = new File(localFilePathName);
  54.         PutObjectRequest putObjectRequest = new PutObjectRequest(bucketName, key, file);
  55.         //设置权限属性等-非必需
  56. //        putObjectRequest.setCannedAcl(null);
  57.         //上传
  58.         PutObjectResult putObjectResult = s3.putObject(putObjectRequest);
  59.         System.out.println(putObjectResult.getETag());
  60.     }
  61.  
  62.     /**
  63.      * 从s3桶删除一个文件
  64.      * 注意：如果删除的是最后一个文件，那么，上层文件夹也会被同时删除
  65.      */
  66.     public void deleteFile() {
  67.         s3.deleteObject(bucketName, key);
  68.     }
  69.  
  70.     /**
  71.      * 获取一个文件
  72.      *
  73.      * @throws IOException
  74.      */
  75.     public S3Object downloadFile() throws IOException {
  76.         //获取文件
  77.         S3Object s3Object = s3.getObject(bucketName, key);
  78.         //s3文件输入流，继承了FilterInputStream
  79.         S3ObjectInputStream s3ObjectInputStream = s3Object.getObjectContent();
  80.         //文件属性等数据
  81.         ObjectMetadata objectMetadata = s3Object.getObjectMetadata();
  82.         //文件的key-可以用来获取文件名
  83.         System.out.println(s3Object.getKey());
  84.         //桶名
  85.         System.out.println(s3Object.getBucketName());
  86.         System.out.println(s3Object.getRedirectLocation());
  87.         System.out.println(s3Object.getTaggingCount());
  88.         return s3Object;
  89.     }
  90.  
  91.     /**
  92.      * 获取一个文件,并打印出来
  93.      *
  94.      * @throws IOException
  95.      */
  96.     public void downloadFileWithPrint() throws IOException {
  97.         S3Object s3Object = s3.getObject(bucketName, key);
  98.         S3ObjectInputStream s3ObjectInputStream = s3Object.getObjectContent();
  99.         //如果是txt文件，可以把内容打印出来
  100.         StringBuffer stringBuffer = new StringBuffer();
  101.         byte[] buffer = new byte[1024];
  102.         while ((s3ObjectInputStream.read(buffer)) != -1) {
  103.             stringBuffer.append(new String(buffer, "UTF-8"));
  104.         }
  105.         s3ObjectInputStream.close();
  106.         System.out.println(stringBuffer.toString().trim());
  107.     }
  108.  
  109.     /**
  110.      * 获取一个文件，并保存到本地
  111.      *
  112.      * @throws IOException
  113.      */
  114.     public void downloadFileWithSave() throws IOException {
  115.         S3Object s3Object = s3.getObject(bucketName, key);
  116.         S3ObjectInputStream s3ObjectInputStream = s3Object.getObjectContent();
  117.         OutputStream os = null;
  118.         try {
  119.             //文件名
  120.             String fileName = s3Object.getKey().substring(s3Object.getKey().lastIndexOf("/"));
  121.             os = new FileOutputStream(new File(localFilePath + fileName));
  122.             byte[] buffer = new byte[1024];
  123.             while ((s3ObjectInputStream.read(buffer)) != -1) {
  124.                 os.write(buffer);
  125.             }
  126.             os.flush();
  127.         } catch (IOException e) {
  128.             e.printStackTrace();
  129.         } finally {
  130.             if (os != null) {
  131.                 os.close();
  132.             }
  133.             if (s3ObjectInputStream != null) {
  134.                 s3ObjectInputStream.close();
  135.             }
  136.         }
  137.     }
  138.  
  139.     /**
  140.      * 复制一个文件
  141.      * 比较简单，就是复制，可以选择原文件的版本
  142.      */
  143.     public void copyFile() {
  144.         String sourceBucketName = bucketName;
  145.         String sourceKey = key;
  146.         String sourceVersionId = null;
  147.         String destinationBucketName = bucketName;
  148.         String destinationKey = key;
  149.         CopyObjectRequest copyObjectRequest = new CopyObjectRequest(sourceBucketName, sourceKey, sourceVersionId, destinationBucketName, destinationKey);
  150.         CopyObjectResult copyObjectResult = s3.copyObject(copyObjectRequest);
  151.         System.out.println(copyObjectResult.toString());
  152.     }
  153.  
  154. }

    另外附上两个test类：

AwsS3BucketTest
  1. package com.study.s3;
  2.  
  3. import org.junit.Test;
  4.  
  5.  
  6. public class AwsS3BucketTest {
  7.  
  8.     private AwsS3Bucket awsS3Bucket = new AwsS3Bucket();
  9.  
  10.     @Test
  11.     public void listBuckets() {
  12.         awsS3Bucket.listBuckets();
  13.     }
  14.  
  15.     @Test
  16.     public void createBucketBase() {
  17.         awsS3Bucket.createBucketBase();
  18.     }
  19.  
  20.     @Test
  21.     public void createBucketWithParams() {
  22.     }
  23.  
  24.     @Test
  25.     public void deleteBucket() {
  26.         awsS3Bucket.deleteBucket();
  27.     }
  28.  
  29.     @Test
  30.     public void configBucket() {
  31.         awsS3Bucket.configBucket();
  32.     }
  33. }
AwsS3FileTest
  1. package com.study.s3;
  2.  
  3. import org.junit.Test;
  4.  
  5. import java.io.IOException;
  6.  
  7.  
  8. public class AwsS3FileTest {
  9.  
  10.     private AwsS3File awsS3File = new AwsS3File();
  11.  
  12.     @Test
  13.     public void listFiles() {
  14.         awsS3File.listFiles();
  15.     }
  16.  
  17.     @Test
  18.     public void uploadFile() {
  19.         awsS3File.uploadFile();
  20.     }
  21.  
  22.     @Test
  23.     public void deleteFile() {
  24.     }
  25.  
  26.     @Test
  27.     public void downloadFile() {
  28.     }
  29.  
  30.     @Test
  31.     public void downloadFileWithPrint() {
  32.         try {
  33.             awsS3File.downloadFileWithPrint();
  34.         } catch (IOException e) {
  35.             e.printStackTrace();
  36.         }
  37.     }
  38.  
  39.     @Test
  40.     public void downloadFileWithSave() {
  41.         try {
  42.             awsS3File.downloadFileWithSave();
  43.         } catch (IOException e) {
  44.             e.printStackTrace();
  45.         }
  46.     }
  47.  
  48.     @Test
  49.     public void copyFile() {
  50.     }
  51. }

扫描下方二维码，加入我们，获取更多关于云计算的文章

 

声明：本文内容由网友自发贡献，转载请注明出处：【wpsshop】
推荐阅读
相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。

  

闽ICP备14008679号