Pyinstaller 打包exe运行时找不到源码，函数 错误 OSError: could not get source code_oserror: could not get source code [7920] failed t

今天用pyinstaller将Detectron2的一个文件打包成了exe，可是在程序运行过程中出现了下面错误

Traceback (most recent call last):
  File "torch\_sources.py", line 21, in get_source_lines_and_file
    sourcelines, file_lineno = inspect.getsourcelines(obj)
  File "inspect.py", line 956, in getsourcelines
  File "inspect.py", line 787, in findsource
OSError: could not get source code

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "cell_analyse.py", line 8, in <module>
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "PyInstaller\loader\pyimod03_importers.py", line 495, in exec_module
  File "detectron2\model_zoo\__init__.py", line 8, in <module>
    from .model_zoo import get, get_config_file, get_checkpoint_url, get_config
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "PyInstaller\loader\pyimod03_importers.py", line 495, in exec_module
  File "detectron2\model_zoo\model_zoo.py", line 9, in <module>
    from detectron2.modeling import build_model
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "PyInstaller\loader\pyimod03_importers.py", line 495, in exec_module
  File "detectron2\modeling\__init__.py", line 2, in <module>
    from detectron2.layers import ShapeSpec
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "PyInstaller\loader\pyimod03_importers.py", line 495, in exec_module
  File "detectron2\layers\__init__.py", line 2, in <module>
    from .batch_norm import FrozenBatchNorm2d, get_norm, NaiveSyncBatchNorm, CycleBatchNormList
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "PyInstaller\loader\pyimod03_importers.py", line 495, in exec_module
  File "detectron2\layers\batch_norm.py", line 4, in <module>
    from fvcore.nn.distributed import differentiable_all_reduce
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "PyInstaller\loader\pyimod03_importers.py", line 495, in exec_module
  File "fvcore\nn\__init__.py", line 4, in <module>
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "PyInstaller\loader\pyimod03_importers.py", line 495, in exec_module
  File "fvcore\nn\focal_loss.py", line 52, in <module>
  File "torch\jit\_script.py", line 1307, in script
    ast = get_jit_def(obj, obj.__name__)
  File "torch\jit\frontend.py", line 233, in get_jit_def
    parsed_def = parse_def(fn)
  File "torch\_sources.py", line 95, in parse_def
    sourcelines, file_lineno, filename = get_source_lines_and_file(fn, ErrorReport.call_stack())
  File "torch\_sources.py", line 28, in get_source_lines_and_file
    raise OSError(msg) from e
OSError: Can't get source for <function sigmoid_focal_loss at 0x000002038751F9D8>. TorchScript requires source access in order to carry out compilation, make sure original .py files are available.
[75064] Failed to execute script 'cell_analyse' due to unhandled exception!
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61

这个错误是直接提示为sigmoid_focal_loss 这个函数的源代码没有找到，而添加源代码的程序是File “inspect.py”, line 787, in findsource，进一步追踪代码发现可以通过getfile这个函数可以找到程序具体要添加哪些模块，我在这个函数的代码里面添加了下面这行，打印出具体是哪个模块没有添加。

def getfile(object):
    """Work out which source or compiled file an object was defined in."""
    if ismodule(object):
        if getattr(object, '__file__', None):
            return object.__file__
        raise TypeError('{!r} is a built-in module'.format(object))
    if isclass(object):
        if hasattr(object, '__module__'):
            object = sys.modules.get(object.__module__)
            if getattr(object, '__file__', None):
                return object.__file__
        raise TypeError('{!r} is a built-in class'.format(object))
    if ismethod(object):
        object = object.__func__
    if isfunction(object):
        object = object.__code__
        **print(object)**     **#在这里添加这行代码，打印出到底是哪些源代码没有添加成功**
    if istraceback(object):
        object = object.tb_frame
    if isframe(object):
        object = object.f_code
    if iscode(object):
        return object.co_filename
    raise TypeError('module, class, method, function, traceback, frame, or '
                    'code object was expected, got {}'.format(
                    type(object).__name__))

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

个人感觉这个坑爹的错误是源代码加载有关的，Detectron2使用了inspect这个模块，加载源代码，导致Pyinstaller在分析文件依赖时，没有把相关的源代码添加进去。
添加好打印代码后，再次编译生成exe，重新运行，产生如下log

<code object norm at 0x0000020384365780, file "torch\functional.py", line 1309>
<code object sigmoid_focal_loss at 0x000002038750B390, file "fvcore\nn\focal_loss.py", line 7>
<code object sigmoid_focal_loss at 0x000002038750B390, file "fvcore\nn\focal_loss.py", line 7>
Traceback (most recent call last):
  File "torch\_sources.py", line 21, in get_source_lines_and_file
    sourcelines, file_lineno = inspect.getsourcelines(obj)
  File "inspect.py", line 956, in getsourcelines
  File "inspect.py", line 787, in findsource
OSError: could not get source code
1
2
3
4
5
6
7
8
9

由此可以看出来，是fvcore这个模块没有被添加到可行exe目录下。当我把源代码库下的fvcore拷贝到生成exe所在的目录后，再次运行，就不在报fvcore找不到错误了，可以报了其它模块源代码找不到错误，我以此拷贝过去，程序便可以顺利运行了。

当然正规的解决办法应该是通过修改.spec 文件，将需要依赖的文件在打包时自动添加到编译生成的文件里面，这一步，可以参考pyinstaller的官方解决方法