CMU15-213 Malloc Lab实验记录_15213 malloc lab

Writeup翻译

原文和下载链接在这里

1 Introduction

In this lab you will be writing a dynamic storage allocator for C programs, i.e., your own version of the malloc, free and realloc routines. You are encouraged to explore the design space creatively and
implement an allocator that is correct, efficient and fast.

在这个lab中你将通过C语言，写一个动态内存分配器，写一个你自己版本的malloc、free以及realloc接口。我们鼓励你创造性地探索，实现一个正确、高效且快速的分配器。

2 Logistic

You may work in a group of up to two people. Any clarifications and revisions to the assignment will be posted on the course Web page.

关于组队的，与我们无关。

3 Hand Out Instructions

Start by copying malloclab-handout.tar to a protected directory in which you plan to do your
work. Then give the command: tar xvf malloclab-handout.tar. This will cause a number of
files to be unpacked into the directory. The only file you will be modifying and handing in is mm.c. The mdriver.c program is a driver program that allows you to evaluate the performance of your solution. Use the command make to generate the driver code and run it with the command ./mdriver -V. (The -V flag displays helpful summary information.)

首先下载malloclab-handout.tar，然后输入命令：

tar xvf malloclab-handout.tar
1

进行安装，我们需要写的只是一个mm.c文件。mdriver.c用于评估你的解决方案的表现。使用：

make
1

命令来生成driver的代码并用命令：

./mdriver -V
1

来运行它，输入*-V*标志会展示有帮助的总结信息。

Looking at the file mm.c you’ll notice a C structure team into which you should insert the requested
identifying information about the one or two individuals comprising your programming team. Do this right away so you don’t forget.

关于组队的，与我们无关。

4 How to Work on the Lab

Your dynamic storage allocator will consist of the following four functions, which are declared in mm.h and defined in mm.c.

你的动态内存分配器将由四部分组成，它们声明在mm.h文件中，并在mm.c文件中给出定义。

int mm_init(void);
void *mm_malloc(size_t size);
void mm_free(void *ptr);
void *mm_realloc(void *ptr, size_t size);
1
2
3
4

The mm.c file we have given you implements the simplest but still functionally correct malloc package that
we could think of. Using this as a starting place, modify these functions (and possibly define other private
static functions), so that they obey the following semantics:

mm.c文件中，我们已经给出了一些我们想到的，最简单的，但是在功能性上是正确的malloc包实现。使用这些作为起点，修改这些函数（也可以定义其他私有静态函数），所以他们遵循如下要求：

• mm_init:Before calling mm_malloc mm_realloc or mm_free,the application program (i.e.,
  the trace-driven driver program that you will use to evaluate your implementation)calls mm_init to
  perform any necessary initializations,such as allocating the initial heap area.The return value should
  be-1 if there was a problem in performing the initialization,0 otherwise.

• mm_init：在使用mm_malloc，mm_realloc以及mm_free应用程序之前（比如评估你的实现的driver），会调用mm_init来实现所有必要的初始化，比如生成初始的堆。如果在初始化过程中出现问题应该返回-1，否则返回0。

• mm_malloc: The mm_malloc routine returns a pointer to an allocated block payload of at least
  size bytes. The entire allocated block should lie within the heap region and should not overlap with
  any other allocated chunk.

• mm_malloc：该接口返回一个指向至少为size字节的分配块负载的指针。整个分配块须在堆区域内部，同时不能覆盖其他任何已分配的大块（chunk）。

• mm_free:The mm_free routine frees the block pointed to by ptr.It returns nothing.This rou-
  tine is only guaranteed to work when the passed pointer (ptr)was returned by an earlier call to
  mm_malloc or mm_realloc and has not yet been freed.

• mm_free：该接口释放一个被指针指向的块。这个接口，仅在传递的指针是由先前的生成函数返回，并且未被释放的情况下，保证有效。

• mm_realloc: The mm_realloc routine returns a pointer to an allocated region of at least size
  bytes with the following constraints.
  – if ptr is NULL, the call is equivalent to mm_malloc (size);
  – if size is equal to zero, the call is equivalent to mm-free (ptr);
  – if ptr is not NULL, it must have been returned by an earlier call to mm_malloc or mm_realloc. The call to mm-realloc changes the size of the memory block pointed to by ptr (the old block)to size bytes and returns the address of the new block. Notice that the address of the new block might be the same as the old block, or it might be different, depending on your implementation, the amount of internal fragmentation in the old block, and the size of the realloc request.
  – The contents of the new block are the same as those of the old ptr block, up to the minimum of
  the old and new sizes. Everything else is uninitialized. For example, if the old block is 8 bytes
  and the new block is 12 bytes, then the first 8 bytes of the new block are identical to the first 8
  bytes of the old block and the last 4 bytes are uninitialized. Similarly, if the old block is 8 bytes
  and the new block is 4 bytes, then the contents of the new block are identical to the first 4 bytes
  of the old block.

• mm_realloc：改接口返回一个至少size字节的被分配区域的指针，它有以下限制：
  – 如果指针为空，那么作用相当于mm_malloc(size)；
  – 如果size == 0，那么作用相当于mm_free(ptr)；
  – 如果指针非空，那么它必须是先前生成函数返回的指针。使用mm_realloc()，会将原指针指向的内存块大小，变为size字节，并返回新的块的地址。注意到新的块地址和原先的地址也许不同也许相同，这取决于你的实现，旧块的内部碎片大小，以及请求分配的size大小；
  – 新的块的内容与旧的块内容在公共的部分是一致的。其他的均未初始化。比如，如果一个8字节的旧块，想要转换成12字节的新块，那么转换后，前8字节与旧块保持一致，而后4字节为被初始化。相似的，如果一个8字节的旧块，想要转换成4字节的新块，那么新块前4字节内容与旧块保持一致。

These semantics match the the semantics of the corresponding libc malloc, realloc, and free routines. Type man malloc to the shell for complete documentation.

这些要求匹配了想要标准C库中这些接口的要求，通过在shell中输入：

man malloc
1

来查看完整的文档。

5. Heap Consistency Checker

Dynamic memory allocators are notoriously tricky beasts to program correctly and efficiently. They are difficult to program correctly because they involve a lot of untyped pointer manipulation. You will find it very helpful to write a heap checker that scans the heap and checks it for consistency.

众所周知，动态内存分配器是编程正确且高效的一大敌人。它们难以正确编程，因为它需要处理许多未定义类型的指针操作。写一个能够扫描堆，并检查它的一致性的堆检测器（heap checker），将会帮上你的大忙。

Some examples of what a heap checker might check are:

• Is every block in the free list marked as free?
• Are there any contiguous free blocks that somehow escaped coalescing?
• Is every free block actually in the free list?
• Do the pointers in the free list point to valid free blocks?
• Do any allocated blocks overlap?
• Do the pointers in a heap block point to valid heap addresses?

以下是堆检测器也许会检查的事情：

• 是否每个空闲链表中的块，都被标记为空闲了？
• 是否存在未合并的相邻空闲链表？
• 是否所有的空闲块都在空闲链表中？
• 空闲链表中的指针，是否指向了有效的空闲块？
• 是否有两个被分配的块重叠了？
• 指针是否都指向了有效的堆地址？

Your heap checker will consist of the function int_mm_check(void) in mm.c. It will check any invariants or consistency conditions you consider prudent. It returns a nonzero value if and only if your heap is consistent. You are not limited to the listed suggestions nor are you required to check all of them. You are encouraged to print out error messages when mm_check fails.

你的堆检测器将包含在*int_mm_check()*函数中，它将检查任何你认为重要的不变量和一致性条件。如果你的堆是一致的，那么返回一个非0的值。以上所列的只是建议，不必严格参考，但建议你能够在出错时，能够打印出错误信息。

This consistency checker is for your own debugging during development. When you submit mm.c, make sure to remove any calls to mm check as they will slow down your throughput. Style points will be given for your mm check function. Make sure to put in comments and document what you are checking.

关于提交的，pass。

6. Support Routines

The memlib.c package simulates the memory system for your dynamic memory allocator. You can invoke the following functions in memlib.c:

memlib.c包为你的动态内存分配器，模拟了内存系统。你可以援引以下函数：

• void *mem_sbrk(int incr): Expands the heap by incr bytes, where incr is a positive
  non-zero integer and returns a generic pointer to the first byte of the newly allocated heap area. The
  semantics are identical to the Unix sbrk function, except that mem sbrk accepts only a positive
  non-zero integer argument.
• void *mem_heap_lo(void): Returns a generic pointer to the first byte in the heap.
• void *mem_heap_hi(void): Returns a generic pointer to the last byte in the heap.
• size_t mem_heapsize(void): Returns the current size of the heap in bytes.
• size_t mem_pagesize(void): Returns the system’s page size in bytes (4K on Linux systems).

void *mem_sbrk(int incr): 将堆拓展[incr]大小，当incr是正数时，返回新分配的堆的第1字节的通用指针。
它的效果与Unix系统中的sbrk函数完全一样，除了这里的sbrk只能接收正整数
void *mem_heap_lo(void): 返回堆第一个字节的指针
void *mem_heap_hi(void): 返回堆最后一字节的指针
size_t mem_heapsize(void): 返回当前堆的大小
size_t mem_pagesize(void): 返回系统的页大小（单位是字节）
1
2
3
4
5
6

7 The Trace-driven Driver Program

The driver program mdriver.c in the malloclab-handout.tar distribution tests your mm.c package for correctness, space utilization, and throughput. The driver program is controlled by a set of trace files that are included in the malloclab-handout.tar distribution. Each trace file contains a sequence of
allocate, reallocate, and free directions that instruct the driver to call your mm_malloc, mm_realloc, and mm_free routines in some sequence. The driver and the trace files are the same ones we will use when we grade your handin mm.c file.

驱动程序mdriver.c会测试你的mm.c文件的正确性，空间利用率和吞吐量。你可以通过用一些列的trace file来操作驱动程序。每个trace file包含了一个生成和释放的序列，这会指引驱动器调用你写的函数。

The driver mdriver.c accepts the following command line arguments:

驱动器接收以下命令行参数：

• -t <tracedir>: Look for the default trace files in directory tracedir instead of the default
  directory defined in config.h.
• -f <tracefile>: Use one particular tracefile for testing instead of the default set of trace files.
• -h: Print a summary of the command line arguments.
• -l: Run and measure libc malloc in addition to the student’s malloc package.
• -v: Verbose output. Print a performance breakdown for each tracefile in a compact table.
• -V: More verbose output. Prints additional diagnostic information as each trace file is processed.
  Useful during debugging for determining which trace file is causing your malloc package to fail.

• -t <tracedir>：重新在tracedir文件夹中寻找默认的trace files。
• -f <tracefile>：使用一个特定的tracefile，来代替默认的trace files序列进行测试。
• -h：打印命令行参数的用途。
• -l：使用并衡量标准C库中的malloc，作为学生的malloc库的补充。
• -v：打印信息。在一张紧凑的表上打印每个trace file的性能指标。
• -V：打印更多信息。对每个运行的trace file，打印额外的诊断信息。当为决定哪个trace file导致你的malloc包运行失败时，非常有用。

8 Programming Rule

• You should not change any of the interfaces in mm.c.
• You should not invoke any memory-management related library calls or system calls. This excludes the use of malloc, calloc, free, realloc, sbrk, brk or any variants of these calls in your
  code.
• You are not allowed to define any global or static compound data structures such as arrays, structs, trees, or lists in your mm.c program. However, you are allowed to declare global scalar variables such as integers, floats, and pointers in mm.c.
  For consistency with the libc malloc package, which returns blocks aligned on 8-byte boundaries,
  your allocator must always return pointers that are aligned to 8-byte boundaries. The driver will
  enforce this requirement for you.

• 不允许改变mm.c的任何接口
• 不允许使用，任何内存管理相关的库调用和系统调用
• 不允许定义任何全局或者静态的复合数据结构。不过，你可以声明全局的标量变量，比如int，float或者指针
• 为了确保标准C库内存分配的一致性，标准的malloc总是返回能够被8整除的字节的块，所以你的分配器必须和malloc的一致。驱动器将按照这个要求运行。

9 Evaluation

You will receive zero points if you break any of the rules or your code is buggy and crashes the driver. Otherwise, your grade will be calculated as follows:

如果你的程序有bug，那么你就只能得0分了，如果没有，就按照以下规则计算：

• Correctness (20 points). You will receive full points if your solution passes the correctness tests
  performed by the driver program. You will receive partial credit for each correct trace.
• Performance (35 points). Two performance metrics will be used to evaluate your solution:
  – Space utilization: The peak ratio between the aggregate amount of memory used by the driver
  (i.e., allocated via mm malloc or mm realloc but not yet freed via mm free) and the size of the heap used by your allocator. The optimal ratio equals to 1. You should find good policies to minimize fragmentation in order to make this ratio as close as possible to the optimal.
  – Throughput: The average number of operations completed per second.

• 正确性（20分）。如果你的解决方案通过了驱动程序的正确性测试，那么你将获得满分。每过一个正确性检测，你都会得到部分分数。
• 性能（35分）。将通过下面两项性能指标来衡量你的解决方案：
  – 空间利用率：有效负载和堆空间的比率峰值。上限是1。你需要找到好的方法，来尽量减少碎片（fragmentation），使得你的比率尽量接近最优。
  – 吞吐量（Throughput）：每秒平均完成的操作数

The driver program summarizes the performance of your allocator by computing a performance index, P, which is a weighted sum of the space utilization and throughput

驱动程序计算你的性能指标，$P$，这是空间利用率和吞吐量的加权数：

where U is your space utilization, T is your throughput, and Tlibc is the estimated throughput of libc
malloc on your system on the default traces.1 The performance index favors space utilization over
throughput, with a default of w = 0.6.
Observing that both memory and CPU cycles are expensive system resources, we adopt this formula to encourage balanced optimization of both memory utilization and throughput. Ideally, the performance index will reach P = w + (1 − w) = 1 or 100%. Since each metric will contribute at most w and 1 − w to the performance index, respectively, you should not go to extremes to optimize either the memory utilization or the throughput only. To receive a good score, you must achieve a balance
between utilization and throughput.

$U$代表你的空间利用率，$T$代表你的吞吐量，而$T_{libc}$是标准C库的吞吐量。权重更青睐于空间利用率，而不是吞吐量，默认的权重$w=0.6$。
考虑到内存和CPU周期，是珍贵的系统资源，我们采用这个公式，来鼓励你平衡二者。理想情况下，$w$会接近100%。因为每项指标最多只能拿到$w$或$1-w$的分数，你不应该走极端。为了获得高分，你必须平衡好两者。

11 Hints

• Use the mdriver -f option. During initial development, using tiny trace files will simplify debugging and testing. We have included two such trace files (short1,2-bal.rep) that you can use for
  initial debugging.
• Use the mdriver -v and -V options. The -v option will give you a detailed summary for each
  trace file. The -V will also indicate when each trace file is read, which will help you isolate errors.
• Compile with gcc -g and use a debugger. A debugger will help you isolate and identify out of
  bounds memory references.
• Understand every line of the malloc implementation in the textbook. The textbook has a detailed
  example of a simple allocator based on an implicit free list. Use this is a point of departure. Don’t
  start working on your allocator until you understand everything about the simple implicit list allocator.
• Encapsulate your pointer arithmetic in C preprocessor macros. Pointer arithmetic in memory managers is confusing and error-prone because of all the casting that is necessary. You can reduce the complexity significantly by writing macros for your pointer operations. See the text for examples.
• Do your implementation in stages. The first 9 traces contain requests to malloc and free. The last 2 traces contain requests for realloc, malloc, and free. We recommend that you start by getting your malloc and free routines working correctly and efficiently on the first 9 traces. Only then should you turn your attention to the realloc implementation. For starters, build realloc on top of your existing malloc and free implementations. But to get really good performance, you will need to build a stand-alone realloc.
• Use a profiler. You may find the gprof tool helpful for optimizing performance.
• Start early! It is possible to write an efficient malloc package with a few pages of code. However, we can guarantee that it will be some of the most difficult and sophisticated code you have written so far in your career. So start early, and good luck!

• **使用mdriver -f选项。**在起步阶段，用简短的trace files将简化调试，我们提供了两个这样的文件（short1, 2-bal.rep）用于你的初始调试。
• 使用mdriver -v和-V选项。*-v将给予你每个trace file的细节。而-V*也会帮助你推测和避免错误。
• **用gcc -g编译并使用调试器。**调试器会帮你识别界外内存引用。
• **理解书上的malloc实现的每一行代码。**书上有一个细致的，基于隐式空闲链表的简单分配器的例子。把这个例子作为出发点。在没有理解这个分配器的所有内容之前，不要写这个lab！
• **在C的预处理宏汇总，封装你的指针算法。**出于必要的casting，内存管理中的指针算法，令人困惑且容易出错。你通过编写宏，来显著减少复杂度。
• **分阶段完成你的实现。**前9个traces包括了malloc和free的请求。而最后两个则增加了realloc的请求。我们建议你先写好malloc和free的接口，做到正确且高效地解决前9个traces。**只有在那种情况下，你才开始写realloc。开始时，你可以通过malloc和free来生成realloc。但是为了获得非常好的性能，你需要写一个独立版本的realloc。
• **使用*profiler。*你会发现gprof在优化性能时非常有用。
• **尽早开始！**也许不需要很长的代码，就能够写出一个高效的malloc包。**然而，我们保证，这将是你职业生涯至今，你写过的最为困难和复杂的代码之一。**所以尽早开始，祝好运！