vue+springboot实现大文件分片上传、断点续传_springboot+vue大文件上传,分片上传,断点续传

一、前言

前一时间没事自己做了个个人网盘小项目，中间遇到大文件分片上传的问题，第一次解决还是比较坎坷，这里记录下我的实现原理及过程。

效果图：

二、思路

VUE前端

1. 选择要上传的文件
2. 计算选择文件的md5信息
3. 调用文件创建接口将文件名、大小、md5等信息传给后台，用来校验文件是否已经上传，如果之前已经上传完成过，这里就停止了
4. 如果没上传将选择的文件按提前设定好大小分片
5. 得到分反的文件，循环依次调用 文件上传接口上传，片段文件上传成功后，接口会返回已经上传的大小
6. 根据已经传文件的大小来判断是否上传成功，并跳出循环结束上传

springboot服务端

1. 文件创建接口：

• 根据md5判断文件记录是否创建，如果已经创建状态文件信息返回给前端，其中包含是否已经上传成功，如果没有创建，在表中创建一第文件记录，记下件名、大小、md5等信息，然后再返回。

2. 文件上传接口：

• 查询文件是否创建记录，如果已创建并记录状态是上传完成，结束。
• 查询文件上传的缓存文件是否存在，如果存在判断缓存文件的大小，并直接返回，前端接口这个大小再改为这个长度开始上传（实现断点续传的功能）
• 每次上传的分文件都追加到缓存的文件中，当缓存文件的长度和表中记录的总上度一样且md5值相等，证明文件上传完成了，这里将缓存文件重命名就算完成了

三、代码

前端我用的是vue+elementui组件做的

<template>
    <div style="width: 680px; padding:0 10px 0 10px;">
        <el-upload
            v-if="uploadIf"
            class="upload-demo"
            drag
            multiple
            action="123"
            :limit="50"
            :show-file-list="false"
            :http-request="myUpload"
            :on-exceed="handleExceed"
        >
            <i class="el-icon-upload"></i>
            <div class="el-upload__text">
                将文件拖到此处，或
                <em>点击上传</em>
            </div>
            <div class="el-upload__tip" slot="tip" style="font-size:12px;font-weight:400;color:red">
                提示：<br/>
                1.支持断点续传，最多一次上传50个文件<br/>
                2.中断的任务可在缓存列表里查看, 成功的任务在资源列表里查看<br/>
                3.同一资源不同名称只能上传一次
            </div>
        </el-upload>

        <el-table class="table" :data="uploadData" style="width: 100%;" :max-height="tableHeight" v-if="!uploadIf">
            <el-table-column prop="name" label="名称" show-overflow-tooltip></el-table-column>
            <el-table-column prop="size" label="大小" width="100">
                <template slot-scope="scope">{{ scope.row.size | getSize }} </template>
            </el-table-column>
            <el-table-column label="进度" width="180">
                <template slot-scope="scope">
                    <el-progress :percentage="scope.row.progress" >{{scope.row.progress}}</el-progress>
                </template>
            </el-table-column>
            <el-table-column label="速度" width="100">
                <template slot-scope="scope">{{scope.row.progress==100?'完成':scope.row.speed}}</template>
            </el-table-column>
        </el-table>
    </div>
</template>

<script>
import {formatFileSize} from '../util/common-util'
import SparkMD5 from "spark-md5";

export default {
    name: "ResourceUpload",
    props: {
        pid: {
            type: Number,
            required: true
        }
    },
    data() {
        return {
            uploadData: [],
            eachSize: 1 * 1024 * 1024,
            maxSize: 2048 * 1024 * 1024,
            uploadIf:true,
            tableHeight:500,
            uploadStop:false,
        };
    },
    created(){
        this.computeHeight();
    },
    methods: {
        uploadStore(){
            this.$store.commit('resource/setUploadList', this.uploadData);
        },
        handleExceed(){
            this.$notify.error({title: '操作失败',message: '文件个数超出最大限制50'});
        },
        async myUpload(params) {
            this.uploadIf = false;
            //console.log("开始上传...");
            const file = params.file;
            
            const { eachSize,uploadData,maxSize } = this;
            if(file.size > maxSize){
                this.$message.error('文件《'+file.name+'》已超2GB，禁止上传');
                return;
            }
            var selectFile = {
                id: 0,
                pid: this.pid,
                state:0,
                name:file.name,
                size:file.size,
                progress: 0,
                speed: '计算md5',
                speedStart: 0,
                speedEnd: 0
            }
            uploadData.push(selectFile);
            this.uploadStore();
            let fileMd5 = await this.calculateMd5(file);
            selectFile['md5'] = fileMd5;
            //计算速度
            setInterval(()=>{
                var speed = formatFileSize(selectFile.speedEnd - selectFile.speedStart);
                selectFile.speed = speed + '/秒';
                selectFile.speedStart = selectFile.speedEnd;
                if(selectFile.progress == 100 || this.uploadStop){
                    return;
                }
            }, 1000);
            //看之前有没有上传过
            await this.$api.post('resource/create', selectFile).then( data => {
                selectFile.id = data.id;  
                selectFile.state = data.state;
            })
            if(selectFile.state == 1){
                selectFile.progress = 100;
                return;
            }
            //开始分片上传
            for (let startSize = 0; ; ) {
                if(this.uploadStop){
                    break;
                }
                const chunkFile = file.slice(startSize, startSize + eachSize);
                const formData = new FormData();
                formData.append("file", chunkFile);
                formData.append("id", selectFile.id);
                formData.append("startSize", startSize);
                startSize = await this.$api({
                    url: "resource/upload",
                    method: "post",
                    data: formData,
                    onUploadProgress: e => {
                        let num = (((startSize + e.loaded) / file.size) * 100) | 0;
                        if(num > 100)num = 100;
                        selectFile.progress = num;
                        //计算速度用
                        selectFile.speedEnd = startSize + e.loaded;
                    }
                }).then(res => {
                    return res;
                });
                if (startSize >= file.size) {
                    selectFile.progress = 100;
                    this.uploadStore();
                    break;
                }
            }
        },
        calculateMd5(file) {
            return new Promise((resolve, reject) => {
                var fileReader = new FileReader(),
                    blobSlice =
                        File.prototype.mozSlice ||
                        File.prototype.webkitSlice ||
                        File.prototype.slice,
                    chunkSize = 5242880, //5MB
                    chunks = Math.ceil(file.size / chunkSize),
                    currentChunk = 0,
                    spark = new SparkMD5();
                fileReader.onload = function(e) {
                    spark.appendBinary(e.target.result); // append binary string
                    currentChunk++;
                    if (currentChunk < chunks) {
                        loadNext();
                    } else {
                        resolve(spark.end());
                    }
                };
                function loadNext() {
                    var start = currentChunk * chunkSize,
                        end =
                            start + chunkSize >= file.size
                                ? file.size
                                : start + chunkSize;

                    fileReader.readAsBinaryString(
                        blobSlice.call(file, start, end)
                    );
                }
                loadNext();
            });
        },
        computeHeight(){
            this.tableHeight = document.body.clientHeight*0.80;
        }
    },
    mounted() {
        window.onresize = () => {
            return (() => {
                this.computeHeight();
            })();
        }
    },
    beforeDestroy(){
        this.uploadStop = true;  
    },
    filters:{
        getSize:function (size) {
             return formatFileSize(size);
        },
    }
};
</script>

<style scoped>
.el-table {
    font-size: 12px;
}
</style>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210

接口

表

@Getter
@Setter
@Entity(name = "resource")
public class Resource {

    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;
    private String name;
    private String md5;
    private Long size;
    private String path;
    private String url;
    private String type;//音频|视频|软件|文件|图片|压缩包|文件夹
    private Integer state;//0未完成1已完成
    private Date createTime;
    private Long pid;
    private Integer deleted;
    private Double seq;

}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

创建文件接口

@Getter
@Setter
public class ResourceCreateDTO {

    @ApiModelProperty("有md5就认为是文件")
    private String md5;
    private Long size;
    @NotBlank
    private String name;
    @NotNull
    private Long pid = 0L;

    public String getFileSuffix(){
        if(name.indexOf(".")>-1){
            return name.substring(name.lastIndexOf(".")+1);
        }
        return "";
    }

}

@Transactional
public Resource create(ResourceCreateDTO dto){
    if(StringUtils.isEmpty(dto.getMd5())){
        Resource resource = new Resource();
        resource.setName(dto.getName());
        resource.setState(1);
        resource.setCreateTime(new Date());
        resource.setPid(dto.getPid());
        resource.setType("0");
        resource.setSize(0L);
        resource.setDeleted(0);
        resource.setSeq(9999d);
        resourceDAO.save(resource);
        return resource;
    }else {
        Resource resource = resourceDAO.findByMd5(dto.getMd5());
        if(resource == null){
            //判断文件有没有超最大值
            validSize(dto.getSize());
            //判断有没有超容量
            validTotalSize(dto.getSize());
            String fileSuffix = dto.getFileSuffix();
            resource = new Resource();
            resource.setMd5(dto.getMd5());
            resource.setName(dto.getName());
            resource.setPath(dto.getName());
            resource.setSize(dto.getSize());
            resource.setUrl(ConfigService.getResourceRootUrl()+dto.getName());
            resource.setState(0);
            resource.setCreateTime(new Date());
            resource.setPid(dto.getPid());
            resource.setType(fileSuffix.toLowerCase());
            resource.setDeleted(0);
            resource.setSeq(0d);
            resourceDAO.save(resource);
            return resource;
        }else{
            if(resource.getDeleted()==1){ //如果之前删除，但还没有清除数据，可以再启用
                resource.setDeleted(0);
                resource.setPid(dto.getPid());
                resource.setName(dto.getName());
                resource.setCreateTime(new Date());
            }
            return resource;
        }
    }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68

分片上传接口

@Getter
@Setter
public class ResourceUploadDTO {

    @NotNull
    private Long startSize;
    private MultipartFile file;
    @NotNull
    private Long id;

}


@Transactional
public Long upload(ResourceUploadDTO dto) {
    Resource resource = getById(dto.getId());
    if(resource.getState()==1){
        return resource.getSize();
    }
    String resourceUploadPath = SystemConfigService.getResourceDiskPath();
    File tempFolder = new File(resourceUploadPath);
    //如果文件存在
    File file = new File(tempFolder,resource.getPath());
    if(file.exists()){
        return file.length();
    }
    //如果缓存不存在，或缓存=开始上传的大小，否则就给当前缓存的大小让他重新上传
    File tempFile = new File(tempFolder,"temp/"+resource.getPath()+".temp");
    Long tempLen = tempFile.length(); // 缓存文件的大小
    if(!tempFile.exists() || tempLen.longValue() == dto.getStartSize().longValue()){
        FileUtil.append(tempFile,dto.getFile());
    }else {
        return tempLen;
    }
    //上传后的大小更新
    tempLen = tempFile.length();
    //如果上传完成，重名命名该文件，且更新记录
    if(resource.getSize().longValue() == tempLen.longValue()){
        tempFile.renameTo(new File(tempFolder,resource.getPath()));
        resource.setState(1);
    }
    return tempLen;
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

FileUtil工具类方法

public static void append(File file, MultipartFile multipartFile) {
 	try {
        append(file,multipartFile.getInputStream());
    } catch (IOException e) {
        throw new APIException(APICode.INTERNAL_SERVER_ERROR,"追加文件出错"+e.getMessage());
    }
}

public static void append(File file, InputStream inputStream) {
    BufferedOutputStream outputStream = null;
    FileOutputStream fileOutputStream = null;
    try {
        int bufSize = 1024;
        fileOutputStream = new FileOutputStream(file,true);
        outputStream = new BufferedOutputStream(fileOutputStream);
        byte[] buffer = new byte[bufSize];
        int temp;
        while ((temp = inputStream.read(buffer)) > 0) {
            outputStream.write(buffer, 0, temp);
        }
        if(inputStream!=null)inputStream.close();
        outputStream.flush();
    } catch (Exception e) {
        e.printStackTrace();
        throw new APIException(APICode.INTERNAL_SERVER_ERROR,"追加文件出错"+e.getMessage());
    }finally {
        try{
            if(fileOutputStream!=null)fileOutputStream.close();
            if(inputStream!=null)inputStream.close();
            if(outputStream!=null)outputStream.close();
        }catch (IOException e){

        }
    }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

原创文章未经本人许可，不得用于商业用途及传统媒体。转载请注明出处，否则属于侵权行为，谢谢合作！