Android图像显示SurfaceFlinger总结

1 介绍

1.1 框架中位置

​​
上图为Android的图形显示系统框架图。
首先上层应用通过ViewRoot的scheduleTraversals函数发起绘制任务，并通过HWUI调用OpenGL接口将绘制数据传递给GPU处理；SF会接收所有应用更新的绘制数据，并根据Z-Order、透明度、大小、位置等参数计算出每个应用图层在最终合成图像中的位置；SF完成图层处理后会把所有的应用图层提供给HWC，由HWC来决定这些图层的合成策略并调用屏显驱动做合成；最后屏显驱动会将合成的最终画面送到硬件屏幕显示。

Android系统通过buffer缓冲区来保存图形信息，这个buffer的内存空间是有SF向图形内存分配器Gralloc申请的，而Gralloc是通过ION Driver从kernel中开辟出一块共享内存。在整个图形显示流程中，buffer缓冲区会在App、SF、HWC之间来回流转。

1.2 职责

1. surfaceFlinger是一个服务进程
2. 负责管理应用程序窗口的创建、显示、更新和销毁等操作
   3）向上和WindowManager交互，向下和HwComposer交互。
   SF会接收所有应用更新的绘制数据，并根据Z-Order、透明度、大小、位置等参数计算出每个应用图层在最终合成图像中的位置；
   SF完成图层处理后会把所有的应用图层提供给HWC，由HWC来决定这些图层的合成策略并调用屏显驱动做合成

负责显示系统界面和应用程序的图形渲染工作。

• 图像合成：SurfaceFlinger 接收来自多个应用程序和系统服务的图像缓冲区，根据它们的位置、大小、透明度、Z轴顺序等属性，将它们合成到一个最终的缓冲区中，然后发送到显示设备上 。
• 管理 Surface：SurfaceFlinger 管理所有 Surface 对象，包括视频、图片等，并为每个 Surface 对象分配 BufferQueue（缓冲区队列），确保每个 Surface 都能按时完成显示 。
• 三缓冲与 VSYNC 机制：SurfaceFlinger 采用双缓冲区机制，即前台和后台两个缓冲区，以及 VSYNC 机制，以提高渲染效率和质量，减少屏幕撕裂现象 。
• 硬件加速：SurfaceFlinger 提供多种硬件加速技术，如 OpenGL ES、Vulkan 等，使应用程序能够更快地渲染 UI 界面 。
• 窗口叠加与透明度支持：SurfaceFlinger 支持窗口叠加、透明度、混合模式等特性，以支持复杂的多层 UI 界面 。
• 与 Hardware Composer 通信：SurfaceFlinger 与硬件抽象层 Hardware Composer 进行通信，利用硬件加速的方式来合成 Surface，提高性能和节省电量 。
• 处理输入事件：SurfaceFlinger 中的 EventThread 方法负责处理输入事件，将输入事件分发给对应的窗口或 Surface 进行处理

2 源码分析

首先看下系统中SurfaceFlinger的进程

代码中入口：
SF是一个独立的可执行程序，路径是/system/bin/surfaceflinger，开机过程中init进程会解析surfaceflinger.rc，启动surfaceflinger进程。
frameworks/native/services/surfaceflinger/surfaceflinger.rc，

service surfaceflinger /system/bin/surfaceflinger
    class core animation
    user system
    group graphics drmrpc readproc
    capabilities SYS_NICE
    onrestart restart --only-if-running zygote
    task_profiles HighPerformance
    socket pdx/system/vr/display/client     stream 0666 system graphics u:object_r:pdx_display_client_endpoint_socket:s0
    socket pdx/system/vr/display/manager    stream 0666 system graphics u:object_r:pdx_display_manager_endpoint_socket:s0
    socket pdx/system/vr/display/vsync      stream 0666 system graphics u:object_r:pdx_display_vsync_endpoint_socket:s0

on property:vendor.debug.sf.restart=1
     restart surfaceflinger

on property:init.svc.zygote=restarting && property:debug.sf.toomanylayers=1
     restart surfaceflinger
     setprop debug.sf.toomanylayers "0"
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

2.1 SurfaceFlinger的初始化

init.rc中调用到SF的main函数，这里是SF进程的入口。
路径：frameworks/native/services/surfaceflinger/main_surfaceflinger.cpp

int main(int, char**) {
	startGraphicsAllocatorService(); //启动图形服务
	...
	sp<SurfaceFlinger> flinger = surfaceflinger::createSurfaceFlinger(); //1. 创建SF实例
	flinger->init(); //2. 对SF实例进行初始化
	// publish surface flinger
	//3. 将SF添加到serviceManager中。 这样客户端通过getService()+SurfaceFlinger的名称，
	// 就可以获取SF的远程代理对象，进而和SF服务跨进程通信
	//static char const* getServiceName() ANDROID_API { return "SurfaceFlinger"; } @SurfaceFinger.h
    sp<IServiceManager> sm(defaultServiceManager());
    sm->addService(String16(SurfaceFlinger::getServiceName()),  flinger,  false,
                   IServiceManager::DUMP_FLAG_PRIORITY_CRITICAL | IServiceManager::DUMP_FLAG_PROTO);

    // publish gui::ISurfaceComposer, the new AIDL interface
    //4. 新建SurfaceComposerAIDL对象，注册到serviceManager中
    sp<SurfaceComposerAIDL> composerAIDL = new SurfaceComposerAIDL(flinger);
    sm->addService(String16("SurfaceFlingerAIDL"), composerAIDL, false,
                   IServiceManager::DUMP_FLAG_PRIORITY_CRITICAL | IServiceManager::DUMP_FLAG_PROTO);
	// 启动Display服务
	startDisplayService(); // dependency on SF getting registered above
	...
	// run surface flinger in this thread
    flinger->run(); //5. 启动SF线程
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

2.2 SurfaceFlinger构造函数详解

先来看下SurfaceFlinger的类关系图
//frameworks/native/services/surfaceflinger/SurfaceFlinger.h

ISurfaceComposer的接口定义：
它的职责是：

class ISurfaceComposer: public IInterface {
public:
    DECLARE_META_INTERFACE(SurfaceComposer)
    static constexpr size_t MAX_LAYERS = 4096;
    ...
    //Create a connection with SurfaceFlinger.
    virtual sp<ISurfaceComposerClient> createConnection() = 0;
    ...
1
2
3
4
5
6
7
8

2.2.1 createSurfaceFlinger详解

接下来看下surfaceflinger::createSurfaceFlinger()具体做了什么？
新建一个SurfaceFlinger实例，将DefaultFactory 做为参数传入，其中DefaultFactory继承了surfaceflinger::Factory类。
主要做了什么？

• 创建CompositionEngine
• 添加了WindowInfosListenerInvoker回调

//frameworks/native/services/surfaceflinger/SurfaceFlingerFactory.cpp
sp<SurfaceFlinger> createSurfaceFlinger() {
    static DefaultFactory factory;
    return new SurfaceFlinger(factory);
}

SurfaceFlinger::SurfaceFlinger(Factory& factory, SkipInitializationTag)
      : mFactory(factory),
        mPid(getpid()),
        mInterceptor(mFactory.createSurfaceInterceptor()),
        mTimeStats(std::make_shared<impl::TimeStats>()),
        mFrameTracer(mFactory.createFrameTracer()),
        mFrameTimeline(mFactory.createFrameTimeline(mTimeStats, mPid)),
        mCompositionEngine(mFactory.createCompositionEngine()), //创建CompositionEngine
        ...
        //添加了WindowInfosListenerInvoker回调
        mWindowInfosListenerInvoker(sp<WindowInfosListenerInvoker>::make(*this)) {
    ALOGI("Using HWComposer service: %s", mHwcServiceName.c_str());
}
//frameworks/native/services/surfaceflinger/SurfaceFlinger.cpp
//SurfaceFlinger构造函数中对一些成员变量进行赋值初始化。
SurfaceFlinger::SurfaceFlinger(Factory& factory) : SurfaceFlinger(factory, SkipInitialization) {
    ALOGI("SurfaceFlinger is starting");
    hasSyncFramework = running_without_sync_framework(true);
    dispSyncPresentTimeOffset = present_time_offset_from_vsync_ns(0);
    useHwcForRgbToYuv = force_hwc_copy_for_virtual_displays(false);
    maxFrameBufferAcquiredBuffers = max_frame_buffer_acquired_buffers(2);
    ...
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

2.2.2 SurfaceFlinger的init详解

flinger->init()对SF实例进行初始化,具体做了什么？

void SurfaceFlinger::init() {
    ALOGI(  "SurfaceFlinger's main thread ready to run. "
            "Initializing graphics H/W...");
    Mutex::Autolock _l(mStateLock);
    //构造renderengine的builder
    auto builder = renderengine::RenderEngineCreationArgs::Builder()
                           .setPixelFormat(static_cast<int32_t>(defaultCompositionPixelFormat))
                           .setImageCacheSize(maxFrameBufferAcquiredBuffers)
                           .setUseColorManagerment(useColorManagement)
                           .setEnableProtectedContext(enable_protected_contents(false))
                           .setPrecacheToneMapperShaderOnly(false)
                           .setSupportsBackgroundBlur(mSupportsBlur)
                           .setContextPriority(
                                   useContextPriority
                                           ? renderengine::RenderEngine::ContextPriority::REALTIME
                                           : renderengine::RenderEngine::ContextPriority::MEDIUM);
    ...
    //创建renderEngine，并给mCompositionEngine设置RenderEngine
    mCompositionEngine->setRenderEngine(renderengine::RenderEngine::create(builder.build()));
    ...
    //调用mCompositionEngine来createHWComposer，给mCompositionEngine设置回调函数。思考：这里回调是什么接口？作用是？
	mCompositionEngine->setTimeStats(mTimeStats);
    mCompositionEngine->setHwComposer(getFactory().createHWComposer(mHwcServiceName));
    mCompositionEngine->getHwComposer().setCallback(*this);
    ...
    // Process any initial hotplug and resulting display changes.
    processDisplayHotplugEventsLocked(); //重要调用，第一次会做一次插拔检测
    const auto display = getDefaultDisplayDeviceLocked();
    LOG_ALWAYS_FATAL_IF(!display, "Missing primary display after registering composer callback.");
    const auto displayId = display->getPhysicalId();
    LOG_ALWAYS_FATAL_IF(!getHwComposer().isConnected(displayId),  "Primary display is disconnected.");
    // initialize our drawing state
    mDrawingState = mCurrentState; //更新drawState， 思考：其中共有几种状态？都是什么时候切换的？
    initializeDisplays();
    ...
    ALOGV("Done initializing");
｝
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

2.2.3 processDisplayHotplugEventsLocked详解

//frameworks/native/services/surfaceflinger/SurfaceFlinger.cpp
void SurfaceFlinger::processDisplayHotplugEventsLocked() {
    for (const auto& event : mPendingHotplugEvents) {
    	std::optional<DisplayIdentificationInfo> info =
                getHwComposer().onHotplug(event.hwcDisplayId, event.connection);
        if (!info) { continue; }
        //获取displayId和token 
        const auto displayId = info->id;
        const auto token = mPhysicalDisplayTokens.get(displayId);
    	if (event.connection == hal::Connection::CONNECTED) { //如果此时有显示屏幕连接
    		... //
    	} else { //如果此时没有显示屏幕连接
            ALOGV("Removing display %s", to_string(displayId).c_str());
            if (const ssize_t index = mCurrentState.displays.indexOfKey(token->get()); index >= 0) {
                const DisplayDeviceState& state = mCurrentState.displays.valueAt(index);
                mInterceptor->saveDisplayDeletion(state.sequenceId);
                mCurrentState.displays.removeItemsAt(index);
            }

            mPhysicalDisplayTokens.erase(displayId);
        }
         processDisplayChangesLocked(); //主要步骤，处理显示改变
    }

    mPendingHotplugEvents.clear();
}

void SurfaceFlinger::processDisplayChangesLocked() {
    const KeyedVector<wp<IBinder>, DisplayDeviceState>& curr(mCurrentState.displays);
    const KeyedVector<wp<IBinder>, DisplayDeviceState>& draw(mDrawingState.displays);
    if (!curr.isIdenticalTo(draw)) {
        mVisibleRegionsDirty = true;

        // find the displays that were removed (ie: in drawing state but not in current state)
        // also handle displays that changed (ie: displays that are in both lists)
        for (size_t i = 0; i < draw.size(); i++) {
            const wp<IBinder>& displayToken = draw.keyAt(i);
            const ssize_t j = curr.indexOfKey(displayToken);
            if (j < 0) {
                // in drawing state but not in current state
                processDisplayRemoved(displayToken);
            } else {
                // this display is in both lists. see if something changed.
                const DisplayDeviceState& currentState = curr[j];
                const DisplayDeviceState& drawingState = draw[i];
                processDisplayChanged(displayToken, currentState, drawingState); //核心步骤，调用processDisplayAdded
            }
        }
		...
	}
    mDrawingState.displays = mCurrentState.displays;
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52

2.2.4 processDisplayAdded详解

主要做了几件事：
1）通过CompositionEngine创建display
2）创建BufferQueue 和 DisplaySurface对象
3）调用initscheduler来初始化整个Vsync系统

void SurfaceFlinger::processDisplayAdded(const wp<IBinder>& displayToken,
                                         const DisplayDeviceState& state) {
	compositionengine::DisplayCreationArgsBuilder builder; 
	builder.setPixels(resolution);
    builder.setIsSecure(state.isSecure);
    builder.setPowerAdvisor(mPowerAdvisor.get());
    builder.setName(state.displayName);
    // 1. 通过CompositionEngine创建display，具体display参数通过builder设置
    auto compositionDisplay = getCompositionEngine().createDisplay(builder.build());
    compositionDisplay->setLayerCachingEnabled(mLayerCachingEnabled);

	sp<compositionengine::DisplaySurface> displaySurface;
    sp<IGraphicBufferProducer> producer; //为什么有producer 和BqProducer，两者有什么区别？
    sp<IGraphicBufferProducer> bqProducer;
    sp<IGraphicBufferConsumer> bqConsumer;
    //2. 创建BufferQueue
    getFactory().createBufferQueue(&bqProducer, &bqConsumer, /*consumerIsSurfaceFlinger =*/false);
	//3. 创建DisplaySurface对象
    if (state.isVirtual()) {
        const auto displayId = VirtualDisplayId::tryCast(compositionDisplay->getId());
        auto surface = sp<VirtualDisplaySurface>::make(getHwComposer(), *displayId, state.surface,
                                                       bqProducer, bqConsumer, state.displayName);
        displaySurface = surface;
        producer = std::move(surface);
    } else {
        const auto displayId = PhysicalDisplayId::tryCast(compositionDisplay->getId());
        displaySurface = sp<FramebufferSurface>::make(getHwComposer(), *displayId, bqConsumer,
                                             state.physical->activeMode->getResolution(),
                                             ui::Size(maxGraphicsWidth, maxGraphicsHeight));
        producer = bqProducer;
    }
	...
    auto display = setupNewDisplayDeviceInternal(displayToken, std::move(compositionDisplay), state,
                                                 displaySurface, producer);
    if (display->isPrimary()) {
        initScheduler(display); //主要步骤，调用initscheduler来初始化整个Vsync系统
    }
   
    mDisplays.try_emplace(displayToken, std::move(display));
｝
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

2.2.5 initScheduler详解

initscheduler初始化整个Vsync系统，主要做几件事：
1）如果没有mScheduler ，创建mScheduler，有的话直接使用
2）创建APP连接
3）创建SF连接
4）初始化Vsync

void SurfaceFlinger::initScheduler(const sp<DisplayDevice>& display) {
    if (mScheduler) {
        // If the scheduler is already initialized, this means that we received
        // a hotplug(connected) on the primary display. In that case we should
        // update the scheduler with the most recent display information.
        ALOGW("Scheduler already initialized, updating instead");
        mScheduler->setRefreshRateConfigs(display->holdRefreshRateConfigs());
        return;
    }
    //如果没有mScheduler ，创建mScheduler
    mScheduler = std::make_unique<scheduler::Scheduler>(static_cast<ICompositor&>(*this),
                                                        static_cast<ISchedulerCallback&>(*this),
                                                        features);
	{
        auto configs = display->holdRefreshRateConfigs();
        if (configs->kernelIdleTimerController().has_value()) {
            features |= Feature::kKernelIdleTimer;
        }
        mScheduler->createVsyncSchedule(features);
        mScheduler->setRefreshRateConfigs(std::move(configs));
    }
    setVsyncEnabled(false);
    mScheduler->startTimers();

    const auto configs = mVsyncConfiguration->getCurrentConfigs();
    const nsecs_t vsyncPeriod = currRefreshRate.getPeriodNsecs();
	//创建APP连接
    mAppConnectionHandle =  mScheduler->createConnection("app", mFrameTimeline->getTokenManager(),
                                         /*workDuration=*/configs.late.appWorkDuration,
                                         /*readyDuration=*/configs.late.sfWorkDuration,
                                         impl::EventThread::InterceptVSyncsCallback());
	//创建SF连接
    mSfConnectionHandle = mScheduler->createConnection("appSf", mFrameTimeline->getTokenManager(),
                                         /*workDuration=*/std::chrono::nanoseconds(vsyncPeriod),
                                         /*readyDuration=*/configs.late.sfWorkDuration,
                                         [this](nsecs_t timestamp) {
                                             mInterceptor->saveVSyncEvent(timestamp);
                                         });
	//初始化Vsync
    mScheduler->initVsync(mScheduler->getVsyncDispatch(), *mFrameTimeline->getTokenManager(),
                          configs.late.sfWorkDuration);
    mFpsReporter = new FpsReporter(*mFrameTimeline, *this); 
    mScheduler->onPrimaryDisplayModeChanged(mAppConnectionHandle, display->getActiveMode());
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

思考几个问题：
1）创建mScheduler的用处？
2）mScheduler->createConnection()具体做了什么？
3）mScheduler->initVsync()做了什么？

2.2.6 mScheduler详解

2.3

3 小结