ExoPlayer的缓存 三 SimpleCache的使用_exoplayer simplecache

ExoPlayer的缓存 – 三 Cache的使用

CacheDataSource 读取数据

创建 CacheDataSource

和 exoplayer 的其他 DataSource 一样，CacheDataSource 的生成也是通过 Factory 模式生成，

  private CacheDataSource createDataSourceInternal(
      @Nullable DataSource upstreamDataSource, @Flags int flags, int upstreamPriority) {
    Cache cache = checkNotNull(this.cache);
    @Nullable DataSink cacheWriteDataSink;
    if (cacheIsReadOnly || upstreamDataSource == null) {
      cacheWriteDataSink = null;
    } else if (cacheWriteDataSinkFactory != null) {
      cacheWriteDataSink = cacheWriteDataSinkFactory.createDataSink();
    } else {
      cacheWriteDataSink = new CacheDataSink.Factory().setCache(cache).createDataSink();
    }
    return new CacheDataSource(
        cache,
        upstreamDataSource,
        cacheReadDataSourceFactory.createDataSource(),
        cacheWriteDataSink,
        cacheKeyFactory,
        flags,
        upstreamPriorityTaskManager,
        upstreamPriority,
        eventListener);
  }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

CacheDataSource 构造函数的参数和读写有关的有

1. Cache cache 使用的缓存

2. @Nullable DataSource upstreamDataSource 根据url 生成的 DataSource, 可为null, 只能从Cache 中读取数据

3. cacheReadDataSource 从cache 中读取数据用的 DataSource

4. @Nullable DataSink cacheWriteDataSink 写缓存用的DataSink，可为null, 不可向Cache 中写数据

5. @Nullable CacheKeyFactory cacheKeyFactory 用来生成 Cache ID， 用来匹配Cache

private CacheDataSource(
    Cache cache,
    @Nullable DataSource upstreamDataSource,
    DataSource cacheReadDataSource,
    @Nullable DataSink cacheWriteDataSink,
    @Nullable CacheKeyFactory cacheKeyFactory,
    ......
) {
  ......
      this.upstreamDataSource = upstreamDataSource;
      this.cacheWriteDataSource =
          cacheWriteDataSink != null
              ? new TeeDataSource(upstreamDataSource, cacheWriteDataSink)
              : null;
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

在 CacheDataSource 的构造函数中 通过 upstreamDataSource 和cacheWriteDataSink 生成cacheWriteDataSource 类型为TeeDataSource。 缓存的保存正是通过TeeDataSource 完成。

TeeDataSource 写入缓存数据

在TeeDataSource 的read 函数中，先从upstream 中 读取数据，然后写入到dataSink 中。

public int read(byte[] buffer, int offset, int length) throws IOException {
  if (bytesRemaining == 0) {
    return C.RESULT_END_OF_INPUT;
  }
  int bytesRead = upstream.read(buffer, offset, length);
  if (bytesRead > 0) {
    // TODO: Consider continuing even if writes to the sink fail.
    dataSink.write(buffer, offset, bytesRead);
    if (bytesRemaining != C.LENGTH_UNSET) {
      bytesRemaining -= bytesRead;
    }
  }
  return bytesRead;
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14

CacheDataSource#open

在open 函数中，先根据cacheKeyFactory 生成的key 构建DataSpec， 然后调用openNextSource 函数

public long open(DataSpec dataSpec) throws IOException {
  try {
    String key = cacheKeyFactory.buildCacheKey(dataSpec);
    DataSpec requestDataSpec = dataSpec.buildUpon().setKey(key).build();
    this.requestDataSpec = requestDataSpec;


    if (bytesRemaining > 0 || bytesRemaining == C.LENGTH_UNSET) {
      openNextSource(requestDataSpec, false);
    }
    return dataSpec.length != C.LENGTH_UNSET ? dataSpec.length : bytesRemaining;
  }
}
1
2
3
4
5
6
7
8
9
10
11
12
13

openNextSource 中选择合适的DataSource

在openNextSource 中，首先通过

1. cache.startReadWriteNonBlocking 获取cache 中的CacheSpan 的nextSpan，
2. 如果nextSpan 为null, Cache 中无缓存数据但是数据被锁定，直接走 upstreamDataSource
3. 如果nextSpan 不为空而且已经缓冲完成，直接使用缓冲的nextSpan， 使用cacheReadDataSource
4. 如果nextSpan 不为空而且还在缓冲中，当前的实际位置开始读数据 ， 使用cacheWriteDataSource

private void openNextSource(DataSpec requestDataSpec, boolean checkCache) throws IOException {
  @Nullable CacheSpan nextSpan;
  String key = castNonNull(requestDataSpec.key);
  if (currentRequestIgnoresCache) {
     ...
  } else if (blockOnCache) {
     ...
  } else {
    nextSpan = cache.startReadWriteNonBlocking(key, readPosition, bytesRemaining);
  }

  DataSpec nextDataSpec;
  DataSource nextDataSource;
  if (nextSpan == null) {
    // The data is locked in the cache, or we're ignoring the cache. Bypass the cache and read
    // from upstream.
    nextDataSource = upstreamDataSource;
    nextDataSpec =
        requestDataSpec.buildUpon().setPosition(readPosition).setLength(bytesRemaining).build();
  } else if (nextSpan.isCached) {
    // Data is cached in a span file starting at nextSpan.position.
    Uri fileUri = Uri.fromFile(castNonNull(nextSpan.file));
    nextDataSpec =
        requestDataSpec
            .buildUpon()
            .setUri(fileUri)
            .setUriPositionOffset(filePositionOffset)
            .setPosition(positionInFile)
            .setLength(length)
            .build();
    nextDataSource = cacheReadDataSource;
  } else {
    // Data is not cached, and data is not locked, read from upstream with cache backing.

    nextDataSpec =
        requestDataSpec.buildUpon().setPosition(readPosition).setLength(length).build();
    if (cacheWriteDataSource != null) {
      nextDataSource = cacheWriteDataSource;
    } else {
    
    }
  }

  long resolvedLength = nextDataSource.open(nextDataSpec);
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

CacheDataSink 保存数据

在createDataSourceInternal 函数中通过

cacheWriteDataSink = new CacheDataSink.Factory().setCache(cache).createDataSink();
1

CacheDataSink 两个属性

1. fragmentSize 分片大小 数据可以分片保存，对应Cache中的CacheSpan
2. bufferSize 保存时使用的buffer 大小

  /** Default {@code fragmentSize} recommended for caching use cases. */
  public static final long DEFAULT_FRAGMENT_SIZE = 5 * 1024 * 1024;
  /** Default buffer size in bytes. */
  public static final int DEFAULT_BUFFER_SIZE = 20 * 1024;
  
public CacheDataSink(Cache cache, long fragmentSize, int bufferSize) {
1
2
3
4
5
6

CacheDataSink 写数据

首先检查当前文件的长度，如果长度等于分片大小，先关闭当前流，然后打开下一个分片流的

public void write(byte[] buffer, int offset, int length) throws CacheDataSinkException {
  @Nullable DataSpec dataSpec = this.dataSpec;
  try {
    int bytesWritten = 0;
    while (bytesWritten < length) {
      if (outputStreamBytesWritten == dataSpecFragmentSize) {
        closeCurrentOutputStream();
        openNextOutputStream(dataSpec);
      }
      int bytesToWrite =
          (int) min(length - bytesWritten, dataSpecFragmentSize - outputStreamBytesWritten);
      castNonNull(outputStream).write(buffer, offset + bytesWritten, bytesToWrite);
      bytesWritten += bytesToWrite;
      outputStreamBytesWritten += bytesToWrite;
      dataSpecBytesWritten += bytesToWrite;
    }
  } 
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

分片文件的 命名规则

在 openNextOutputStream 中调用cache.startFile 生成新的文件。

private void openNextOutputStream(DataSpec dataSpec) throws IOException {
  file =
      cache.startFile(
          castNonNull(dataSpec.key), dataSpec.position + dataSpecBytesWritten, length);
  FileOutputStream underlyingFileOutputStream = new FileOutputStream(file);
  ......
}
1
2
3
4
5
6
7

#SimpleCache
@Override
public synchronized File startFile(String key, long position, long length) throws CacheException {
  CachedContent cachedContent = contentIndex.get(key);
  
  File cacheSubDir = new File(cacheDir, Integer.toString(random.nextInt(SUBDIRECTORY_COUNT)));

  long lastTouchTimestamp = System.currentTimeMillis();
  return SimpleCacheSpan.getCacheFile(
      cacheSubDir, cachedContent.id, position, lastTouchTimestamp);
}
1
2
3
4
5
6
7
8
9
10
11

# SimpleCacheSpan
public static File getCacheFile(File cacheDir, int id, long position, long timestamp){
  return new File(cacheDir, id + "." + position + "." + timestamp + SUFFIX);
}
1
2
3
4

数据库中保存的文件

缓存 SimpleCache

public SimpleCache(File cacheDir, CacheEvictor evictor, DatabaseProvider databaseProvider)
1

1. File cacheDir 缓存目录
2. CacheEvictor evictor 缓存存储策略 Exoplayer 提供了两种 LeastRecentlyUsedCacheEvictor NoOpCacheEvictor
3. DatabaseProvider databaseProvider 数据库

startFile

public synchronized File startFile(String key, long position, long length) throws CacheException

参考 分片文件的 命名规则， 用于在当前缓存目录生成一个新的文件

commitFile

在文件下载完成后，用于向Cache 的CacheFileMetadataIndex fileIndex数据库提交信息。

首先生成一个SimpleCacheSpan，

public synchronized void commitFile(File file, long length) throws CacheException {


  if (fileIndex != null) {
    String fileName = file.getName();
    try {
      fileIndex.set(fileName, span.length, span.lastTouchTimestamp);
    } catch (IOException e) {
    }
  }
  addSpan(span)
}
1
2
3
4
5
6
7
8
9
10
11
12

addSpan

Add Splan 在内存中保存当前的缓存信息

同一个文件的缓存保存在一个 CachedContent， CachedContent 中的keyToContent 保存所有的span 文件。

// Adds a cached span to the in-memory representation.
// Params:
// span – The span to be added.
private void addSpan(SimpleCacheSpan span) {
  contentIndex.getOrAdd(span.key).addSpan(span);
  totalSpace += span.length;
  notifySpanAdded(span);
}

public CachedContent getOrAdd(String key) {
    CachedContent cachedContent = keyToContent.get(key);
}
1
2
3
4
5
6
7
8
9
10
11
12

startReadWriteNonBlocking

public synchronized CacheSpan startReadWriteNonBlocking(String key, long position) throws CacheException

从数据库中或者内存中查找 CacheSpan 文件

public synchronized CacheSpan startReadWriteNonBlocking(String key, long position)
    throws CacheException {
  SimpleCacheSpan span = getSpan(key, position);

  if (span.isCached) {
    // Read case.
    return touchSpan(key, span);
  }

  CachedContent cachedContent = contentIndex.getOrAdd(key);
  if (!cachedContent.isLocked()) {
    // Write case.
    cachedContent.setLocked(true);
    return span;
  }

  // Lock not available.
  return null;
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

cache 中的数据库

SimpleCache 中CacheFileMetadataIndex：

SimpleCache 中CachedContentIndex：

DownloadManager 中的DefaultDownloadIndex

DownloadManager 中用于记录下载的表

整体上的数据库设计

在Exoplayer 的默认实现中，使用了一个数据库，数据库中建了三张表。