MacOS 链接特性：Two-Level Namespace

在 MacOS 编译链接时，可能经常会报找不到符号的错误，本文所介绍的 two-level namespace 的链接特性可能就是报错原因之一。

本文为笔记附带翻译，想看详细内容的话，在文末附上了参考链接，由于篇幅不算大，所以我简单地翻译了一下，如有错误，帮忙指出，多多包涵。

1、Two-Level Namespace & Flat Namespace

When an executable file is loaded into a program, the dynamic linker (the part of the system responsible for loading code on demand) attempts to find all of the unresolved symbols (functions in other executable files). 【可执行文件被加载到程序内存当中时，动态链接器需要查找所有的未解析符号】

In order to do this, it needs to know which libraries contain those symbols. So when the program is built, you must specify the libraries that contain those functions to the static linker (the part of the build system responsible for linking object files together). The static linker places the names of those libraries into your program’s executable file. 【它得知道符号在什么库，所以编译程序时就需要给静态链接器指明包含这些函数的库。静态链接器将这些库的名字放入可执行文件当中】

The problem is that the static linker in Mac OS X 10.0 records only the names of the libraries, but not which functions are to be found in each of libraries. Therefore, it’s not possible to load two libraries containing a definition of the same symbol name, because it’s not possible to determine which library you really wanted the symbol from. 【问题来了，MacOS 10.0 只记录库的名字，而没有指明哪个函数来源于哪个库。因此带有同名函数的两个库将无法区分】

This isn’t usually a problem on Mac OS X 10.0, because the build system will give you a multiple-defined-symbols error message when you attempt to build your application using libraries that contain a symbol with the same name. But consider the following situations. 【所幸 MacOS 10.0 会报错，但还有一下情况：】

• Future versions of the system may export symbols which conflict with those implemented in your application. This will prevent users from being able to use your application. 【新版本系统库的符号和你程序里的库符号冲突】

• If your application supports third-party plugins or bundles, libraries used by your third-party developers may conflict with each other. 【第三方扩展冲突】

To solve this problem, the Mac OS X 10.1 runtime environment supports a new feature that records the names of symbols expected to be found in a library along with the library name. An executable file built with this feature enabled is called a two-level namespace executable. An executable file without this feature is called a flat namespace executable. 【MacOS 10.1 推出 two-level namespace 特性作为解决方案，即记录库名+符号名；没有这个特性的就叫 flat namespace 】

1.1、ld manual

这部分内容是我直接在命令行用 man ld 看的。

Two-level namespace

By default all references resolved to a dynamic library record the library to which they were resolved. At runtime, dyld uses that information to directly resolve symbols. The alternative is to use the -flat_namespace option. With flat namespace, the library is not recorded. At runtime, dyld will search each dynamic library in load order when resolving symbols. This is slower, but more like how other operating systems resolve symbols. 【默认会记录库名+符号，运行时 dyld 运用这个信息直接解析符号，这种方式会快很多。】

Alters how symbols are resolved at build time and runtime. 【改变的是编译时和运行时解析符号的方式】

With -two_levelnamespace (the default), the linker only searches dylibs on the command line for symbols, and records in which dylib they were found. 【默认的 two-level namespace，链接器只从参数中提供的库搜索符号，并记录符号在什么库被找到】

With -flat_namespace, the linker searches all dylibs on the command line and all dylibs those original dylibs depend on. The linker does not record which dylib an external symbol came from, so at runtime dyld again searches all images and uses the first definition it finds. In addition, any undefines in loaded flat_namespace dylibs must be resolvable at build time. 【flat namespace，链接器会从参数中搜索所有的库，以及它们所依赖的库。链接器不记录符号来自于哪个库，所以在运行时 dyld 又会搜索所有的映像，用它找到的第一个定义。此外，未定义符号必需在编译时解析完成。】

Indirect dynamic libraries

If the command line specifies to link against dylib A, and when dylib A was built it linked against dylib B, then B is considered an indirect dylib. When linking for two-level namespace, ld does not look at indirect dylibs, except when re-exported by a direct dylibs. On the other hand when linking for flat namespace, ld does load all indirect dylibs and uses them to resolve references. Even though indirect dylibs are specified via a full path, ld first uses the specified search paths to locate each indirect dylib. If one cannot be found using the search paths, the full path is used. 【如果参数指明链接 A，而 A 依赖 B，那么 B 就是间接依赖库。使用 two-level namespace 特性时，ld 不会去查找间接依赖库，除非其符号被直接依赖库重新导出。如果是 flat namespace，ld 会加载所有的间接依赖库并用它们解析符号。即使间接依赖库以绝对路径指明，ld 还是先用指明的搜索路径来定位间接依赖库。如果搜索路径找不到，采用绝对路径。】

Dynamic libraries undefines

When linking for two-level namespace, ld does not verify that undefines in dylibs actually exist. But when linking for flat namespace, ld does check that all undefines from all loaded dylibs have a matching definition. This is sometimes used to force selected functions to be loaded from a static library. 【当使用 two-level namespace 特性时，ld 不会检查库中未定义符号是否存在。但当使用 flat namespace 特性时，ld 会从所有已加载的匹配到函数定义的库中检查所有的未定义符号。有时用于强制从静态库中加载函数。】

1.2、如何判断一个文件使用的是哪种链接特性

otool -hV [file]
1

如果有 TWOLEVEL 字样的，即为 two-level namespace 特性，否则为 flat namespace。

2、使用 two-level namespace 的问题

• There can be no unresolved, undefined references in two-level namespace executables. Using the -undefined suppress option will cause this error:

/usr/bin/ld: -undefined error must be used when -twolevel_namespace is in effect

If you recieve this error message, you need to remove the -undefined suppress option and make sure you specify the path to the program you are linking against with the option *-bundle_loader pathnameToYourProgram*.

If you are linking against a framework, this path is already specified using the -framework option, so you just need to remove the -undefined suppress option. (To be really paranoid, specify the -undefined error option). See the next section for more information.

【two-level namespace 不能有未解析或未定义的引用，所以用了 -undefined suppress 会报错。

需要去掉这个参数并使用 -bundle_loader XXX 来指明链接路径。或者如果是链接 framework 去掉参数即可，因为 -framework 已经指明了路径。】

• When building a two-level namespace executable, you must link against all shared libraries containing the symbols you reference. If your program is currently a flat namespace executable, this may cause problems. When you build a flat namespace executable, the dynamic linker will search all libraries for the program’s undefined symbols, even libraries that your code did not explicitly link against.

For example, your application might link against ApplicationServices.framework, but not CoreFoundation.framework. Because ApplicationServices.framework links to CoreFoundation.framework, you can, as a flat namespace executable, use routines exported by CoreFoundation.framework. If you build the program as a two-level namespace executable, using routines exported by CoreFoundation.framework will result in undefined-symbol errors, so you must explicitly add CoreFoundation.framework to the list of libraries you link.

If you use a symbol from a library that you are not explicitly linking against, you will get a single error message for each such library of the form:

ld: object_file illegal reference to symbol: symbol defined in indirectly referenced dynamic library: library
1

If you see this error message, you must add the library library to your link command.

If library is a sub-framework of an umbrella (for example, HIToolbox.framework is a subframework of Carbon.framework), you will need to add the umbrella framework to your link objects (in Project Builder, just drag the framework into your project). Once you explicitly link against an umbrella framework, you may freely use all of the symbols it contains without referencing any sub-frameworks.

【two-level namespace 会导致间接链接的符号未定义的错误，你可能需要显式地指明间接链接的库。】

3、dyld 查找符号过程

The runtime symbol lookup routines released in Mac OS X 10.0 (located in the header <mach-o/dyld.h> and listed below) perform lookups in the flat, global symbol namespace, and thus, when you use them to find symbols in your plugins, may not return the intended symbols in for two-level namespace applications. 【之前 MacOS 10.0 查找符号的方法，不适用于 two-level namespace。】

Applications built as two-level namespace executables should instead use the following new routines. 【需要用到下面的方法操作】

• NSAddImage

• NSIsSymbolNameDefinedInImage

• NSLookupSymbolInImage

3.1、NSAddImage

const struct mach_header *
NSAddImage(
    char *image_name,
    unsigned long options);

// No options.
#define NSADDIMAGE_OPTION_NONE                  0x0

// If an error occurs and you have specified this option, NSAddImage returns NULL. 
// You can then use the function NSLinkEditError to retrieve information about the error.
#define NSADDIMAGE_OPTION_RETURN_ON_ERROR       0x1

// With this option the image_name passed for the library and all its dependents will be effected 
// by the various DYLD environment variables as if this library were linked into the program.
#define NSADDIMAGE_OPTION_WITH_SEARCHING        0x2

// With this option, NSAddImage will not load a shared library that has not already been loaded. 
// If the specified image_name is already loaded, NSAddImage will return its mach_header.
#define NSADDIMAGE_OPTION_RETURN_ONLY_IF_LOADED 0x4
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

NSAddImage loads the shared library specified by image_name into the current process, returning a pointer to the mach_header data structure of the loaded image. Any libraries that the specified library depends on are also loaded. 【把 image_name 加载到进程，返回被加载映像的 mach_header 指针。间接依赖也会被加载。】

If the shared library specified by image_name is already loaded, the mach_header already loaded is returned. 【如果已加载过了，则返回已有的。】

The image_name parameter is a pointer to a C string containing the pathname to the shared library on disk. For best performance, specify the full pathname of the shared library — do not specify a symlink. 【参数为绝对地址性能更好，不要用符号链接】

3.2、NSLookupSymbolInImage

extern NSSymbol
NSLookupSymbolInImage(
    const struct mach_header *image,
    const char *symbolName
    unsigned long options);

// Bind the non-lazy symbols of the module in the image that defines the symbolName 
// and let all lazy symbols in the module be bound on first call.
#define NSLOOKUPSYMBOLINIMAGE_OPTION_BIND            0x0

// Bind all the non-lazy and lazy symbols of the module in the image that defines the symbolName 
// and let all dependent symbols in the needed libraries be bound as needed.
#define NSLOOKUPSYMBOLINIMAGE_OPTION_BIND_NOW        0x1

// Bind all the symbols of the module that defines the symbolName and all if the dependent symbols of all needed libraries.
#define NSLOOKUPSYMBOLINIMAGE_OPTION_BIND_FULLY      0x2

// Return NULL if the symbol cannot be bound.
#define NSLOOKUPSYMBOLINIMAGE_OPTION_RETURN_ON_ERROR 0x4
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

NSLookupSymbolInImage returns the specified symbol (as an NSSymbol) from the specified image.Error handling for NSLookupSymbolInImage is similar to error handling for NSAddImage. 【返回指定映像的指定符号】

The image parameter is a pointer to a mach_header data structure. You can get this pointer from a shared library by calling NSAddImage. 【image 参数是 mach_header 的指针，可以配合 NSAddImage 使用。】

The symbolName parameter is a C string specifying the name of the symbol to retrieve. 【symbolName 参数是 C 字符串，指明要查找的符号。】

3.3、NSIsSymbolNameDefinedInImage

extern enum DYLD_BOOL
NSIsSymbolNameDefinedInImage(
    const struct mach_header *image,
    const char *symbolName);
1
2
3
4

NSIsSymbolNameDefinedInImage returns true if the specified image (or, if the image is a framework, any of its subframeworks) contains the specified symbol, false otherwise. 【用于判断指定的映像是否包含指定的符号】

3.4、获取 mach_header 的方法

The image parameter for NSLookupSymbolInImage and NSIsSymbolNameDefinedInImage is a pointer to the mach_header data structure of a Mach-O shared library. You can obtain a pointer to a mach_header data structure from:

• the NSAddImage function

• a linker defined symbol as defined in <mach-o/ldsym.h>, described on the ld(1) man page

• the dyld(3) function _dyld_get_image_header(3)

• the mach_header arguments to the callback functions called from _dyld_register_func_for_add_image(3).

参考&推荐阅读

Mac OS X Developer Release Notes:Two-Level Namespace Executables - http://mirror.informatimago.com/next/developer.apple.com/releasenotes/DeveloperTools/TwoLevelNamespaces.html