当前位置:   article > 正文

开发自己的操作系统(Hobby OS-deving)_为什么自己开发系统总感觉像在做娃娃

为什么自己开发系统总感觉像在做娃娃

Hobby OS-deving 1: Are you ready?


       自从开始开发我自己的操作系统以来,已经有一年了,这期间我经常停下来,回头看看我已经完成了什么,并好奇当初是什么原因使得项目的开头这么困难。我得出的一个结论是,虽然有大量的讲述操作系统开发的文档,但这些文档大部分都是从技术角度出发,在项目管理方面着墨不多。也就是说,如何从一个模糊的想法(“我想编写自己的操作系统”),到一个精确的规划(我希望实现什么),或干脆在撞上南墙之前放弃它。本系列文章除了旨在使那些对 OS 开发感兴趣的朋友能够走上正轨外,也希望能兼顾如上所述项目管理的一些方面。
       您正阅读的本文将回答一个相当简单,却非常重要的问题:您真的准备好要进入 OS 开发了吗?很少有像编写 OS 这样耗时长,要求高还收获晚的业余爱好。大部分的个人 OS 开发项目最后都进入了一个死角,因为事先没有很好的理解要进入的领域,因此编写了大量很难维护并且设计糟糕的代码,导致项目无疾而忠;或者因为仅仅实现一个能工作的 malloc()就耗费了大量的时间而意志消沉,并最终被击倒。

一些不好的想法:

       ”我想引导下一次计算革命“:简单的说,不可能。详细点说,如果设想你自己是一个小的研发团队的话,你也许可以把某个东西做的很优雅,但也就是在一个小的范围内。如果没有一个大的组织或社区在背后支持,你的研究成果无法更大规模的普及。你也不可能通过实现所有 OS 的模拟来运行它们之上的程序以吸引更多用户,因为这实在是太困难了。

       ”我想精通某一门编程语言,我需要一些大的挑战来练习“:有些人会尝试通过开发一个 OS 来学习一门新的编程语言,他们认为通过完成某些挑战性的工作,可以真正的理解这门语言。坦白说,这并不是一个好的想法。首先,这会让原本已经非常困难的 OS 开发更加困难,最好还是不要高估自己的能力,尤其在面对挫折和疲惫时。其次,因为使用一门新的编程语言,你将不可避免的犯更多错误。而这些错误通常都发生在底层,你将无法使用那些用户空间程序可以使用的 GDB 与 Valgrind 等诊断工具。再者,即使如此,你学到的也不是这门语言的全部,而只是其中一个很小的子集。除了极少数语言以外(比如专为编写 OS 设计的 C),大部分语言提供的标准库将无法使用,大部分语言都有某些运行时功能,是你在开发 OS 的过程中至少某段时间内无法提供的,比如 C++的异常与vtables,C#与 Java 的垃圾收集与实时指针检查,Pascal 对象的动态数组与字符串,等等。更多相关细节可以看这里(1, 2)。总之,你将不得不有一段艰难的时间来编写一些丑陋的代码,而这些代码无法让你去学习某门语言,还是放弃这个想法吧。  

       ”实现一个新的内核复杂的可怕,不过我有更简单的方式!我只需将 Linux源代码拿来,按我的需求做一些修改,这样做应该不会那么难。“:让我通过一个练习例题来告诉你 Linux源代码(比如 init/main.c)实际上是什么样的,你的第一个目标是找出真正的 main 函数;第二个目标是试着理解它所调用的函数的定义位置,以及它们可能用来做什么;第三个目标则是估算一下完全理解这些函数所需要的时间。你现在应该知道,如果对即将进入的领域没有一个非常好的技术上的理解的话,在已有的代码基础上进行修改将是一个十分困难的选择。代码重用当然是一个相当不错的可选之路,并且有许多优势,但并不会因此使你的生活更加轻松,相反还会更加困难,因为你不光要全面的学习内核的开发,还得深入细节的学习这些已有的代码。

       ”我不喜欢 Linux/Windows/Mac OS X 里实现的某个功能 X,这个功能应该重写“:所有现代的操作系统都有一些自己独特的怪癖,尤其是桌面系统,毫无疑问。不过,如果你的抱怨牵涉到的问题并不大的话,你应该考虑试着去修复它,并贡献给相应的项目,除非这样做失败了,即使重写最好也只重写这个功能牵涉到的功能模块。

       ”我正在寻找一个令人兴奋的商机“:参见上面第一条。除非你工作一个实验室里,并且专门研究操作系统,否则只要你开发的 OS 没有什么实际用途,那么你很难从它身上赚到一毛钱(至少很长一段时间内是如此)。


一些好的看法:

       ”我想尝试一个全新的 OS 设计“:你有自己独到的关于 OS 应该如何设计的想法,并且这些想法在当前的 OS 中找不到?那么开发自己的 OS 可以用来验证你的想法是否实际,并进一步完善你的想法。

       ”我已经熟练掌握了我现在使用的语言,我想找点编程方面的挑战,或者更深入的理解我的电脑在底层是如何工作的“:OS 开发当然能满足你的想法,甚至能提供更多你想不到的。你将学会如何在没有其它诊断工具的帮助下(除了文本输出)去调试代码;如何在没有标准库的环境下完成相应的功能;你将了解为什么有这么多人在抱怨x86 体系架构,即使你的x86 架构的电脑运行的相当不错;你将了解硬件生产商提供的文档到底有多烂…

       ”我已经在某个内核上工作了很长时间了,我非常清楚它的代码结构,以及那些令人头大的地方“:如果这些问题需要相当大的改动,或者可能带来不兼容,那么你除了在它的基础上另起分支外,别无选择。这样做当然也是一种正确的方法,甚至比从头重写一个 OS 要更加值得赞赏,不过本系列文章并不针对这种情况。

       ”我想对这个 OS 所做的改动太大,无法通过简单的补丁或开一个分支就能实现“:这当然是从头开发自己 OS 的最佳选项,如果你的想法确实如此,那么欢迎你来到这儿。

       ”我正在寻找一个新的令人兴奋的爱好“:这要看你如何定义“兴奋”二字,如果你准备好了的话,OS 开发绝对可以称得上是这个地球上最令人兴奋的爱好之一。打一个不太恰当的比方,有点类似于培养一个孩子,或者养活一颗树。这个过程无疑是漫长的,有时候甚至令人疲惫烦躁,以至于你感觉不值得。当然如果你能挺过去,那只会让你的成就感更强烈。此外你还能在这个过程中找到不一样的乐趣,你可以完完全全控制你的电脑。


       那么,你准备好了吗?


       在开始项目之前,除了上面的问答阶段以外,为了能够成功上路,你还必须知道或迅速学习如下的这些东西:
       普通计算机科学:你必须非常清楚并且能够熟练的操作二进制数与16进制数,布尔逻辑,数据结构(数组,链表,哈希表,等等),以及排序算法。
       计算机架构:你应该知道一个桌面电脑的整体架构。The Words and expressions ALU,中断,内存与 PCI 总线,DMA 等应该不至于对你来说太陌生。了解其它一些更高级的技术比如:TLB,多级缓存,或者 branch prediction 将在你走的更远时大有帮助,比如你开始通过优化你的代码提高效率时。
       编译器与可执行文件的内部结构:你必须大体上知道在预编译,编译以及链接阶段都发生了什么,并且能够在需要时从 toolchain 中找出你可以控制的地方。你还必须学会 linker scripting 以及可执行文件的内部结构。
       汇编:你必须知道什么是汇编以及它是如何工作的,你必须能够编写一些简单的汇编代码(操作栈以及寄存器,加减数字,实现 if-else 逻辑…)。你可以用其它语言来实现大部分的 OS 功能,但至少有极少数任务,你只能通过汇编来实现。
       C 或 C++:即使你打算使用其它编程语言来开发你自己的 OS,你也需要了解 C 或者 C++,因为你需要参考的其它 OS 实现,或某些 OS 开发教程基本上都是使用 C 或 C++完成的。
       你将要使用的语言:不管你是用 C#, Java,汇编, BASIC, C, C++, Pascal, Erlang, REBOL 还是什么其它语言来开发你的 OS,你都必须熟练掌握它。实际上,你甚至应该阅读这门语言的内部实现的文档。举个例子,你应该知道使用你选择的语言进行函数调用最终在汇编那儿是如何实现的,你应该知道你的语言需要什么样的运行时支持并且如何来实现它。
       (以上部分基于 OSdevwiki 上的一些内容,并做了一些修改。)

       这就是本系列文章的第一篇所要讲述的内容。如果你已经完成本文中所要求的,那么恭喜你。你应该具备正常的心智与足够的知识来开始你自己的 OS 开发之旅了。在下一篇文章里,我将讲述如何为你的 OS 项目设置一个正确的目标以及期望值,这将是你最终到达目的地的至关重要的一步。


Hobby OS-deving 2: Setting Goals and Expectations?


       现在你应该已经参加了之前的测试,并且做好了开发操作系统的准备了是吗?这个时候,许多 OS-deving 的业余爱好者都会试图去寻找一份教程,并且按照教程的指示,手把手地完成二进制启动,文本输入/输出,或者其他“简单”的功能。如果说他们的开发有一个计划的话,那十之八九是这样的:他们会不时想到一些很酷的功能,然后把它实现。渐渐的,如果设想的没错的话,他们的 OS 系统就这样一个功能又一个功能的组建起来,并且慢慢的超过其它系统。我认为,这并不是获得成就最好的方法(如果你的目标是有所成就的话)。在本文中,我会用我的观点阐述在这个阶段你应该做什么并且为什么要这么做。


设计的重要性:

       首先,过于依赖教程有损创造性。不管教程作者怎样事先提出警告,大多数情况下我们都是在盲目的跟着教程走,而没有完全理解那些描述一个个步骤的冗长文字。为什么会有这种情况呢?因为很多网络上的低水平教程都只是描述如何去创建一个用 C 语言实现的,单内核结构的,具有一个简单 shell 的 UNIX 克隆系统。想到什么了吗?这太普通了,我们身边这样的东西已经泛滥成灾。如果你花费了大把的时间用于操作系统的开发,却仅仅只是实现了又一个 UNIX 系统,并且比现有的系统支持的硬件还少,那真是一种悲哀,当然这只是我的看法,也许你并不这么想。

       不用这种方法的另外一个原因是,过早的将代码塞满大脑会让我们变得鼠目寸光。如果在编码前多花些时间来设计我们的项目,很多陷阱可以就此避免。比如异步 I/O 和多进程。如果在项目的早期你就决定要使用它们,并且能坚持下去,考虑到这方面已经有足够的文档,那么实现这些功能并没有什么本质上的困难。这只是换了一种思考方式而已。但是,如果你从一开始就缺失了这一步,而之后又想在你的代码中添加这些功能,转换地过程将非常痛苦。

       最后,我认为设计非常重要(即使是针对业余项目)的最后一个原因是他们的队伍都太小。他们普遍只有一两个开发人员。当然这并不见得是件坏事情:在操作系统项目的早期阶段,你肯定不希望有无止境的争吵和成堆的各种风格的代码片段。不过缺点也很明显,除非你能专注于非常有限的事情并很好的完成它,否则你无法走得更远。这也是为什么我认为在这个阶段,比起直接编写代码,你更应该先决定好这个项目的目标和你将要做到什么样的程度。

探索动机:

       首先,忘记你是针对某台特定的电脑,甚至忘记你是正在编写代码。作为一个只有1-2人(或者更多人)的小组中的一员,你正在设计的是可以为他人(也许只是小组中的另一员)提供服务的某个东西。所有的一切都将从这里开始。你想提供什么样的服务?何时能说你的项目是成功的?何时又能说你并没有选对方法呢?就像许多其他个人项目一样,要想在一个业余操作系统开发中取得成功,第一步就是给它一个清晰的定义。

       那么继续,拿出一张纸或者用其它的文本方式(我是用我的博客),开始探究你的动机吧。如果你编写操作系统只是因为对现存的某些操作系统感到不满的话,不如花些时间想想你不喜欢它们什么。试着理解这些地方为什么会这样编写,探索他们的演变过程,阅读在这方面你能找到的书。即使这些地方的实现看起来非常随意,你也能获得些什么:那就是实现某样东西不能太随意,并且要特别注意它们的设计。如果你是想通过编写一个操作系统来学些什么的话,那就要更加详细的明确你想学些什么。也许你是想知道过程抽象是如何得来的?通过怎样的过程使一个只是执行运算的复杂而无声的电子器件,到那些没有专业知识的人也能够使用的某个东西。

       所有这些中,在不影响你思考的前提下,提取出尽量多的笔记。这样做有两个原因:首先,晚些时候你能回顾这些笔记。其次,如果你能坚持那些太模糊以至于不能直接用英语表达的想法(或者任何一种你能流利表达的语言),把思想转化为文字能使你想得更加严密。试着让这些笔记尽量的精确,就像这些文字将会被发表或者被你意想不到的人阅读一样。

定义目标受众:

       明确你为什么要这么做之后,尽量快速的定义你的目标客户是谁。操作系统是硬件,终端用户和第三方软件开发者之间的接口,所以你必须在一定程度上将他们全部定义:你的操作系统将在哪种硬件上运行?谁可能会使用它?谁会为它编写软件?编写哪种软件?在这个阶段你不用过于具体的去定义它们,但有些事情你还是需要现在就下决定。

       在硬件方面:你至少应该知道计算机终端用户是如何与硬件进行互动的(键盘?移动定位设备?分别是哪个种类?),你想要覆盖多大的屏幕范围,你所支持的最老的 CPU 具有多强运算能力和应该保证的最小内存容量是多少。你也许还想立刻知道,你的操作系统是否需要连接网络才能正常工作,你打算使用哪种大容量存储器,以及如何安装第三方软件(如果有的话)。

       迅速检查一下你准备为其编写代码的硬件类型是否是对自制 OS 友好的的。大多数的台式机和笔记本电脑都是(可能即将到来的 Chrome OS 上网本是个例外),视频游戏控制器则是参差不齐的(任天堂的通常是,其他的通常不是),苹果和索尼的产品除非快过气了通常也不是。关于手机的警告:尽管听起来是一个相当诱人的编码平台,但就一般而言,他们的硬件文档通常都极其差劲,并且你应当谨记,即使你找到这样一个平台,它能够运行自制的操作系统,并且网上能找到足够的文档,它的下一代产品通常也不能保证让你足够舒坦地运行你的 OS。

       终端用户大概是最难去准确定义的了,除非开发人员只为他们自己设计。你或许应该针对你的目标人群的特定特征建立起形象的编号。举几个例子:“70 岁的老奶奶在圣诞节收到孙子送的电脑。她从来没用过,并且有视力障碍。”,“40 岁留着胡子的网络系统管理员,不会去使用没有提供 bash 风格的命令行界面的电脑,希望通过自动化操作完成他的一些工作,总想着超级给力的脚本引擎。需要多用户支持,以及一个只能由他管理,并且能够实现所有一切的特别账号。”,“20岁使用平板和自由软件画画的富有创造性的女孩。电脑同所有其他事物一样,只是个工具,它不应该比路上的一卷纸还碍事。”

       第三方软件开发人员:首先记住如果你想让他们对你的计划有兴趣的话,你得先像吸引终端用户一样吸引他们。人们不会为他们不会使用的平台编写代码,除非是有报酬的。另一方面,开发人员有着特殊的地位,说明白点,他们创造软件。因此你的一项任务便是明确你想让他们创造什么样的软件。

       从为你编写操作系统的重要部分(经常性的发生在自由和开源软件世界里的桌面操作系统),到那些只能在内部开发并且不存在第三方开发人员的软件(最近已经不流行,除非是某种嵌入式设备)。在这两种最极端的情况之间,有很多其它可能。这里就是一些…

       操作系统部分的设计/管理是版权所有的,但是规格说明是开放的,所以第三方开放人员可以以他们想要的方式自由实现或者重新实现系统的一部分。
       大多数的操作系统是以专利的和闭源的方式制作,但是你需要对第三方驱动和底层工具开放(Windows 9x,Mac OS)。
       同上,但是你要求那些需要访问底层或者其他“危险”的功能的软件通过一些许可或者签名过程(最近的 Windows 和 Symbian 操作系统发行版)。
       你不能忍受第三方的底层应用,并且用一切方法阻止他们的存在,但是你为那些需要经常调用系统 API 的应用提供原生开发工具。
       同上,但是被认为危险的原生第三方用户应用必须通过签名/认证的过程(iOS)。
       没有原生第三方应用,所有程序通过应用程序管理获得有限的系统权限(Android,Windows Phone 7,以及大多数手机)。

       有了这些信息,你应该对于从哪儿开始有了很好的主意,并为下一步做好了准备。

目标和边界:

       既然想法已经在这儿了,尝试把它写到纸上。描述你的操作系统应该达到怎样的目标,谁来使用,运行在哪种硬件上,谁为它开发什么样的程序,以及何时你可以说根据最初的目标项目是成功的,也就是完成了版本 1.0 的发布。通过定义项目的目标,你就定义了一个可以客观判断你的项目是成功还是失败的标准,这将在后面被证明是非常有价值的资源,这也是避免功能无限制膨胀,以及其他浪费你宝贵开发资源的唯一的方法。

       最后,从整体上回顾你现在的项目,问问自己这个简单的问题:“我能成功吗?”如果你觉得你也许应该稍稍降低下期望值,现在就是最好的时候,因为越晚改变,造成的损失就越大。好好研究你现在收集到的一切,仔细琢磨每一件事情直到你对你制定的项目非常满意了(或者完全放弃编写操作系统的想法,如果你已经不再有兴趣),那么你就可以开始下一步了,也就是设计你的内核。


Hobby OS-deving 3: Designing a Kernel


       既然你已经想好你的操作系统项目的总体规划,现在就是具体实施的时候了。如果你要从头开始的话,你需要设计的第一个操作系统组件就是内核,因此这篇文章的目的就是快速引导你进入内核设计,描述你需要思考的主要方面,并且指导你在哪里能找到关于这些主题的更多内容。

内核是什么,它又是做什么的?

       我认为最好在这里做个定义。内核是操作系统的核心部分,它的作用是以一种可控制的方式将硬件资源分配给其他软件。有很多因素使得集中式的硬件资源管理如此引人入胜,包括可靠性(每个应用程序具备的控制能力越小,它在运行异常时造成的损失就越小),安全性(完全相同的理由,只是这次是在有目的地致使应用程序运行发狂时),以及一些需要系统范围内的硬件管理策略的底层系统服务(抢占多任务,电源管理,内存分配…)。

       除了这些通常的考虑之外,大多数现代内核的主要目的实际上是管理进程和线程抽象。进程是被隔离的一个软件实体(isolated software entity),它能够以独占的方式访问有限的硬件资源,目的是防止并发灾难。线程是可以与其他任务并发同时执行的任务(task)。这两个概念彼此独立,虽然现代多任务操作系统中的每个进程至少拥有一个专属线程。

硬件资源

       到目前为止,我还没有就“硬件资源”这个概念做深入的解释,当你读到这种表述时,你最先想到的可能是一些彼此完全不同的硬件实体,譬如鼠标,键盘,存储设备等等…

       然而,你应该知道这些外围设备并不是直接与 CPU 相连的。他们都是通过总线到达某一个 CPU 端口。所以如果你想保证每个进程只能访问某些外围设备的话,内核就得必须控制总线。或者你也许决定将总线也视作一种进程必须请求才能获得的硬件资源。依赖于你的硬件资源模型的精细度,你在进程隔离以及内核复杂性之间的位置也会有所不同。

       使事情更加复杂的是,现代操作系统还管理着一些非常有用的,尽管从纯硬件角度考虑并不存在的硬件资源。考虑内存分配的例子。从硬件的角度看,只有一个 RAM。你的计算机也许有多个 RAM 模块,但是你的 CPU 仍把它们视为一个单独的,大块的 RAM。而你通常想把它的一部分分给一个进程,而另一部分分给另一个进程。

       为了让它工作,内核不得不根据每个进程的需要,拿出它最好的菜刀,把这块连续的内存切成更小的,能安全分配给不同进程的小块。这里还需要一些机制来防止不同的进程互相窥探各自使用的内存,这可以由不同的方式实现,但是最普遍的实现是使用一个嵌入在 CPU 中的特殊的硬件,也就是内存管理单元(MMU)。这个硬件使得内核可以控制让每个进程只能访问属于自己的那块内存,并且当内核从一个进程切换到另一个时,能够在不同进程的内存访问权限间快速切换。

       另一个典型的硬件资源抽象的例子是 CPU 时间。我假定你早就注意到在多核芯片出现之前,桌面操作系统就能让你同时运行多个应用程序。操作系统保证进程能够以一定的方式共享 CPU 时间,通过使 CPU 频繁地从一个进程切换到另一个,使得这一切看起来就像在正常使用条件下同时执行一样。

       当然,我们无法告诉 CPU:“嘿,小伙子,你能让 A 进程以 15 的优先权运行;让 B 进程以 8 的优先权运行吗?” CPU 是非常愚蠢的,他们只是简单的读取一条二进制指令,执行它,然后读取下一条指令,除非有中断使他们从当前的任务转移。所以在现代交互式操作系统中,正是内核保证中断正常的发生,并且当中断发生时,进程的切换也同时发生。整个过程通常称做抢占式多任务处理(pre-emptive multitasking)。

       最后,通常也不能让进程直接访问存储设备,而是让他们访问文件系统的某个地方。当然,同分配的内存一样,文件系统也是一个虚拟的结构,它并没有物理成分在硬盘驱动器或固态硬盘中,并且在某些时刻必须由操作系统完全管理。

       总之,你需要定义哪些硬件资源是内核管理的,并且可以让进程访问他们。通常是否允许进程访问某个硬件并不是件难事,折中地想,内核总是要完成相当一部分这样的管理工作,并且有时候内核必须无中生有出一些硬件资源,例如内存分配,抢占式多任务管理和文件系统操作。

进程间通信与线程同步

       通常来讲,进程彼此间越独立越好。先前说到,封闭的沙盒环境让恶意软件难成大业,并且使系统可靠性大幅的提高了。另一方面,有些场合也需要进程彼此间能方便的交换信息。

       典型的例子是客户端/服务器架构:在系统某处,一个休眠的“服务”进程正在等待命令。“客户”进程能够唤醒它并且使之受控地工作。在某个时刻,“服务”进程完成并把结果返回给“客户”进程。在 UNIX 的世界里,这是非常普遍的方式。另一个进程间通信的例子是有多个交互式进程组成的应用程序。

       有几种进程间通信的方法,比如这些:

       信号:进程间通信最基本的方式,类似中断。可以看做是进程 A 在进程 B 中“响铃”。这里的铃,称之为信号,有一个唯一的数字与之对应,除此之外,别无其它。进程 B 在那一时刻也许正在等待信号的到来,也许定义了一个与之关联的函数,以便当进程收到信号时,该函数能在内核生成的一个新线程里调用。
       管道和其他数据流:进程也经常需要交换各种类型的数据。大多数的现代操作系统都提供这样的功能,尽管因为某些遗留问题,不少操作系统只允许进程以字节为单位交换数据。
       远程过程调用:一旦我们能够从一个进程向另一个进程发送数据,以及发送信号来调用它的某个方法,那么结合这两项,允许一个进程调用另一个进程的方法(明显的在受控制的情况下)则是非常诱人的。这种方式使得我们可以像使用共享链接库那样使用进程,再加上与共享链接库不同的独特优点,调用进程也许能访问那些调用者本不能访问的资源,这也是一种使调用者获得受控制的资源访问的方法。
       共享内存:尽管大多数情况下进程彼此独立更好,但有时候让两个进程共享一段 RAM 也是具有实际意义的,这使得它们能够在内核不介入的情况下做任何它们想做的事。这种方法通常用在内核内部以加快数据传输,以及当需要使用共享链接库时避免重复载入多次,但有些内核也把这样的功能公开给其他需要使用的进程。


       另一件与进程间通信有关的问题是同步,也就是线程必须协调工作的情况。

       要着手于这个问题,注意到在多任务环境中,在某些情况下,我们有时必须保证每次只有一定数量的线程(一般来说只有一个)能够访问所给的资源。举个例子,想象一下以下情景:两个字处理进程同时打开不同的文件,然后用户野蛮地决定打印所有东西,并且迅速在两个窗口中按下了“打印”。

       如果没有机制去预防这种情况,接下来将会发生:两个字处理进程同时向打印机输出数据,互相干扰并且使打印机输出混乱,一般来说输出将混合了这两个文件的内容,可以说惨不忍睹。为了防止这种情况,我们必须在打印机驱动里的某处内置一个机制,使得一次只有一个线程能打印文件。或者,如果我们有两个打印机并且无论使用哪个都可以的话,我们可以使用一种机制来保证一次只有两个线程能打印文档。

       通常的机制称作信号量,一般是这样工作的:在内部,信号量拥有一个计数器来表示一个资源可以被访问多少次。每当一个线程试图访问受信号量保护的资源时,这个计数器就会被检查。如果它的值非零,它就自减一,同时线程被准许访问这个资源。如果它的值为零,那么线程就不能访问这个资源了。请注意要完全确保该机制的健壮性,这个机制必须保证信号量的值在单个处理器指令下被检查和修改,而不是同时运行在多处理器核心上,这需要一些硬件上的帮助。这并不像检查和修改一个整数值那么简单,但是具体怎么去实现就不是现在我们这个阶段的事情了。

       除了信号量机制,另一个较少使用却仍然闻名的同步机制是栅栏(barrier)。它使得 N 个线程必须等待所有线程都完成相应的工作才继续执行。这个机制在某些情况下特别有用,尤其是一个任务被分为几块并行执行,并且不一定能同时完成的时候(想象一下将一个三维图片分为几组进行渲染,然后用单独的线程分别计算它们的情况)。

       总之,定义完你的进程模型,你需要定义它们之间如何进行通信,以及线程如何同步了。

主要关注点与折中

       你也许注意到我已经尽最大努力去关注那些基本的概念,并且专门展示有很多种方法来完成某一件事情。这并不是因为有趣才那么做。在设计内核时,有许多地方需要折中处理,依赖于你设定的目标的不同,你也许需要用不同的方式来考虑它们。

       这里是你在设计内核时需要关注的主要问题的一个综述(尽管每项的重要性根据你的操作系统的不同而不同):

       性能(Performance):相对而言有两个意思,一个是硬件资源的使用,一个是用户察觉到的性能。它们并不完全相同。举个例子,在桌面操作系统中,通过设置优先级,就可以提高用户所能感知到的性能而不需要太多的优化工作(比如,将与用户直接交互的进程的优先级设置的高于提供服务的后台进程)。在实时系统中,硬件资源的使用情况并不像会议的最后期限一样重要。
       可维护性(Maintainability):内核的编码相当困难,所以你通常希望用很长一段时间来完成它。为此,相关代码的查找以及修改必须要非常容易,以便于以后修复某个缺陷或者添加一个新的功能。代码应尽量精简,具有良好的注释,文档和组织,并在不破坏应用程序接口的前提下,为以后的调整留下空间。
       可移植性(Portability):在这个快速演变的硬件世界中,操作系统内核应当很容易被移植到一个新的架构之中。在嵌入式设备领域尤其如此,它们的硬件在复杂性和变化上都大于桌面领域。
       可扩展性(Scalability):可移植性的另一面是你的内核应当自我适应该体系中未来硬件的演化。很明显,这意味着你应该对多核芯片作优化:你应当尽量限制内核中只能有部分线程才能访问的代码的数量,并为多线程的使用好好地优化你的内核。
       可靠性(Reliability):进程是不应该崩溃的。但一旦进程崩溃了,就应该将它们所造成的影响最小化,将系统的可恢复能力最大化。这也是为什么最大化进程隔离,减少松散隔离的进程的代码量,寻求备份进程数据的方法,以及在不重启操作系统的前提下能够重新运行哪怕是最关键的服务进程,等等做法的可贵之处。
       安全性(Security):既然未被认证的第三方程序能在我们的平台上运行,就应该有一些对抗恶意程序的保护措施。你现在必须理解像开源,反病毒,防火墙,经由大量测试人员验证软件等方式都远远不够,也不有效。这些方法应该只是为那些偏执狂所准备的工具,以及当系统安全性失败后的回退方法,并且系统安全性失败应该尽可能的减少。最大化的隔离进程才是根本之道,但你也得减少底层代码渗透系统组件的可能性。
       模块化(Modularity):通常你会希望让内核组件彼此越独立越好。当达到一定的模块化级别时,除了可以改进可维护性,甚至提高可靠性外,你可以在运行的系统不遭受损失的前提下即时重启失败的内核模块。此外,它也能让你使某些内核功能成为可选的,这又是一项很好的功能,尤其是应用在包含在内核里的硬件驱动时。
       开发人员友好性(Developer goodies):在 DOS 的黑暗时代,程序员在它们的软件中加入硬件驱动代码被认为是可以的,因为操作系统几乎什么也不提供。这种情况已经不再了。对于你声称所要支持的所有东西,你必须提供一个友好而统一的接口,该接口应该是一个隐藏了底层复杂硬件细节的好而简单的抽象。
       酷毙了(Cool factor):如果你的内核和其它内核一样,只是使用方式更优越的话,谁会用呢?所以,让我们引进节能的调度机制,别出心裁的崩溃界面,高亮和格式化的日志输出,以及其他有趣而独特的东西吧!


       现在,让我们看看它们是怎样互相冲突的…

       过分追求性能将与除了可扩展性外的所有特性冲突(比如,用汇编写所有代码,不使用函数调用,为追求速度把所有东西放在内核之中,将抽象的级别放到最低…)。
       可维护性与可扩展性冲突,也与任何使代码更加复杂的特性相冲突,特别是这些额外的复杂性无法放到分离的模块里。
       可移植性与那些需要访问硬件架构特有功能的特性相冲突,特别是这些依赖于某个硬件架构功能的代码分散在各处,而没有紧凑放置在一个易于寻找的角落里(比如可能是为了性能优化的方面)。
       可扩展性与任何不能同时运行在 65536 个 CPU 核上的功能或者结构相冲突。除了明显的对可维护性与可靠性的妥协外,这些妥协最好避免让那些难于编码以及难于调试的线程分散在各处,可扩展性与程序员友好性之间也需要做些平衡(一个明显的例子是 libc 以及它多达上百个可阻塞的系统调用)。
       可靠性是任何需要增加复杂度的特性的梦魇,越多的代码意味着越多的 bug,特别是如前所述代码还难以调试时。可靠性与性能有着严重的冲突,许多性能优化需要你提供额外的访问硬件特性的代码。可靠性也是条目中唯一与自己有冲突的设计标准,有些系统功能本身可以改进可靠性。
       就代码的复杂性而言,安全性和可靠性是对兄弟,因为 bug 很容易被利用,从而造成安全隐患。但不同于那些不单独检查操作(指针运算,C 语言风格数组…)的底层代码,它更容易被利用从而导致系统失败。
       模块化通常不喜欢那些必须放在 RAM 中同一个地方的大块代码,这意味着与性能的严重冲突,因为相近的代码位置能优化 CPU 高速缓存的使用。模块化与可维护性的关系也有些模糊:把系统组件彼此分开自然帮助提高可维护性,但是如果走入极端(比如把调度程序也就是内核中管理多任务的部分放在一个进程中),则让人感到困惑。
       我们之前已经看到程序员友好性和那些很酷的功能与其它大部分特性因为各种各样的原因相冲突。还要注意的是,新功能和可维护性的冲突:增加功能是容易的,但是想去除它却很困难,而在此之前你不会知道它们有多么有用。如果你不小心,这将会导致通常所说的功能膨胀(feature bloat),也就是一大堆没用的功能开始随着时间按指数增长的现象。预防这个现象的一个方法是在第一个发布的版本中尽量保证功能集最小,然后检查用户的反馈来发现什么功能是真正需要添加的。但是也要注意“二次系统现象(second-system-effect)”,当你把用户要求的所有东西都放在第二个版本中时,甚至有可能会造成比预先放置大量功能更加严重的功能膨胀现象。


内核设计的一些例子

       当前存在有多种系统内核,虽然它们并没有都取得相同意义上的成功。这里是一些经常遇见的已经定型的内核设计方法(这个列表并不是详尽无遗的)。


       单内核(Monolithic Kernels)

       这是早期所有内核都采用的实现方法。因为性能的原因,这种方法仍然在当今的内核设计中占统治地位。单内核模型由于设计上的简单仍然相当有吸引力。基本上,内核可以看作是一个拥有最高权限,试图包括一切功能的巨大的单独进程。举个例子,桌面系统的单内核通常包括了用来渲染 GPU 加速显卡的图形机制,以及管理所有已存在的文件系统的机制。

       单内核在高性能需求下表现突出,因为所有东西都是内核这个进程的一部分。它们也更容易设计,因为硬件资源模型可以被设计得更简单一些(只有内核能直接访问硬件,用户进程只能访问由内核精心提供的抽象),并且一直到开发的后期,用户空间都不需要你去太过关注。另一方面,采用这种方法也会面临那些不好的编码实践的诱惑,结果将是不可维护,不可移植,没有模块化的代码。由于单内核巨大的代码库,以及对硬件的完全访问,单内核中的 bug 往往更多,造成的影响也比那些隔离度较高的内核设计更大。

       使用单内核设计的例子包括 Linux 以及它的 Android 分支,大多数的 BSD 内核,Windows NT 和 XNU(没错,我也知道后面这两个声称它们是混合内核【hybrid kernels】,但那不过是标榜罢了。如果你把大多数服务放在同一个地址空间,拥有对硬件的完全访问,并且彼此之间没有任何形式的隔离,那么这就是一个单内核,具有单内核的所有优缺点)。


       微内核(Micro-kernels)

       微内核从隔离性来说,是单内核的反面。内核能够访问硬件资源的部分被尽量最小化(MINIX 3 中只有几千行可执行代码,与之对比的是 Linux和 Windows 这样的单内核拥有数百万行计的代码),并且大多数内核服务被移到分离的单独的服务中去,这些服务对硬件的访问被精心设计为只需满足它们特定的目的。

       微内核自然的具有高度的模块化,这种隔离设计也使得它偏好那些良好的编码实践。进程隔离与硬件资源访问的精确控制同样也确保了最优的可靠性和安全性。另一方面,微内核难于编写却易于维护,并且由于经常需要从一个进程切换到另一个进程,使得大部分比较直接的微内核实现在性能上都表现糟糕:通常需要更多的优化工作才能使微内核达到高性能,特别是在 IPC 方面(因为 IPC 是微内核的一个关键的机制)。

       商业级的微内核实现的例子包括:QNX,µ-velOSity 以及 PikeOS。在研究领域,值得一提的是 MINIX 3,GNU Hurd,L4 家族,以及 EROS 家族(KeyKOS,EROS,Coyotos,CapROS)。


       基于 VM 的内核

       这是最近才出现的一种方式,还没完全走出实验室和概念模型的阶段。也许你就是那个能成功实现它的人。想法是既然大多数 bug 和漏洞都来自于原生代码的错误(缓冲区溢出,悬空指针【dangling pointers】,内存泄露…),那么原生代码是邪恶的,应该被淘汰掉。也因此这里的挑战在于如何用 C#或 JAVA 这样的可管理的语言去编写包括内核在内的整个操作系统。

       这个方法的优点首先在于它很酷,以及对可靠性和安全性的提高。基于 VM 的内核提供与微内核相似的隔离性,通过纯软件的机制来隔离进程,有可能在遥远的未来达到比微内核更好的性能(因为指针以及所有对硬件的访问都会被虚拟机检查,没有进程可以访问不被允许的资源)。从另一方面来说,内核开发的世界里没有任何东西是免费的,基于 VM 的内核有几个主要的缺点抵消了这些诱人的优点:

       内核需要包括一个功能齐全的相应语言的解释器,这意味着庞大的代码库,难于设计,以及在实现时那些令人不快的虚拟机 bug。
       编写一个足够快,也适合于运行内核代码(能完全访问硬件以及具有指针操作)的虚拟机是一个巨大的编程挑战。
       比起运行在操作系统用户空间里的虚拟机,直接运行在硬件层上的虚拟机难于编写,也更容易产生 bug 和被利用。与此同时,漏洞也会造成更加严重的后果。当前的 JAVA 虚拟机就是当今个人桌面系统中最易受攻击的来源之一,因此我们必须改变实现虚拟机的方式,才有可能使得它们被包含进内核之中。

       基于虚拟机内核的例子包括:Singularity,JNode,PhantomOS 和 Cosmos。也有一些很有意思却不再活跃的项目,如 JX 和 SharpOS(它的开发人员现在为 MOSA 项目服务)。


参考书目,链接和朋友

       在定义了你必须要考虑的基本概念之后,我相信你希望了解更多细节。事实上你也理应如此。这里有一些比本文的介绍更为深入的关于内核设计的材料。

       Modern Operating Systems (Andrew S. Tanenbaum):如果你只想阅读一本这个主题方面的书,我强烈推荐这本。它在这方面有非常好的启发性和大量描述,覆盖了内核设计的相当多的内容,并且你在开发自己的操作系统的时候都能用得上。

       你可以在 OSdev 的wiki上找到其他相关书籍的列表,也有评论。

       当你浏览上述wiki时,你也应该查看下它的主页,更精确的说是在左边栏(向下移点)的“Design Considerations”链接。你最好把它加为书签,因为当你开始实现你的 OS 后,你将发现你会迫切的需要它。简而言之,这是我知道的关于这个主题方面的最好的资源。

       当你碰到问题时,可以到论坛里面去问他们。他们每月都会面对“我应该怎么办?”和“我困住了,我需要做什么?”这类实现上的问题数百次,所以带一点理论的讨论能得到他们的青睐。但请注意一定要避免那些wiki 上已经回答过的愚蠢问题,否则准备好面对那些喋喋不休的嘲讽吧。

       操作系统设计者的问题读起来颇有意思,尽管它没有涉及非常具体的细节。我应该将它链接到我的上一篇文章中。


       目前为之就是这些了。下次,我们将会进入平台有关的细节,我将描述最基本的x86 体系特色,这在本教程的余下部分将用得到(我将会在显著的地方解释为什么)。


(源地址:http://news.cnblogs.com/n/106914/

==========================================================================================================


【附录】作者原文


posted by  Hadrien Grasland on Fri 28th Jan 2011 20:37 UTC
IconIt's recently been a year since I started working on  my pet OS project, and I often end up looking backwards at what I have done, wondering what made things difficult in the beginning. One of my conclusions is that while there's a lot of documentation on OS development from a technical point of view, more should be written about the project management aspect of it. Namely, how to go from a blurry "I want to code an OS" vision to either a precise vision of what you want to achieve, or the decision to stop following this path before you hit a wall. This article series aims at putting those interested in hobby OS development on the right track, while keeping this aspect of things in mind.

The article which you're currently reading aims at helping you answering a rather simple, but very important question: are you really ready to get into OS development? Few hobbies are as demanding, time-consuming and late-rewarding at the same time. Most hobby OS projects end up reaching a dead-end and slowly being abandoned by their creators because they didn't understand well enough what they were getting into and wrote insufficiently maintainable and ill-designed code or were overwhelmed and depressed by the amount of time it takes to get a mere malloc()working.


Some bad reasons for trying hobby OS development

  • "I'll bring the next revolution of computing": Simply put: no. Long answer: think of yourself as a small research team. You can make something very neat, sure, but only at a small scale. Without massive backing, your research won't make it to mass distribution. And no, you can't emulate an existing OS' software in order to appeal to a wider audience.

  • "I want to become proficient with a programming language, and need some challenging exercises": Some people try to learn a new programming language through OS development, thinking that they need something difficult to do in order to really understand the language. That's not a good idea either.

    • First, because it makes things even harder than they already are for the rest of the OS-deving world, and you shouldn't overestimate your ability to withstand frustration and strain.

    • Second, because being new to the language, you'll make more mistakes and those mistakes are much harder to debug when done at a low level than they are at the user application level where you have neat tools like GDB and Valgrind to help you.

    • Third, because you won't actually learn the language itself, but only a subset of it. The whole standard library will be unavailable and except for a few languages designed for the job like C, most programming languages have features which require a runtime support you won't be able to provide for some time. A few examples: C++ exceptions and vtables, C# and Java's garbage collectors and real-time pointer checks, Pascal Object's dynamic arrays and nice strings... More details here andthere.

    In short, you'll have a hard time coding things. When you do, you'll write poor code and you won't actually learn the language. Don't do it.

  • "Implementing a new kernel is horribly complicated, but there's a much easier path! I'll just take the Linux sources and tweak them till I get an operating system which fits my needs, it shouldn't be so hard": May I be the first to introduce you to what Linux's sources actually look like by showing you its main.c file. Your first mission will be to locate where the "actual" main function is. Your second mission will be to understand where each mentioned function is located and what it's supposed to do. Your third mission will be to estimate the time it'll take you to understand all of that.

    As you should have understood by now, modifying an existing, old codebase without a good knowledge of what you're getting into is really, really hard. Code reuse is a very noble goal and it has many advantages, but you must understand now that it won't make your life simpler but rather make it much worse because you'll not only have to learn kernel development as a whole but also all the specifics of the codebase you're looking at.

  • "I don't like detail X in Linux/Windows/Mac OS X, these should totally be re-done": All modern operating systems have their quirks, especially the desktop ones, no question about that. On the other hand, if your gripes against them remain relatively minor, you really should consider trying to contribute patches which fix them to the relevant projects and only rewriting the incriminated component(s) if this approach fails.

  • "I'm looking for an exciting job": cf. "I'll bring the next revolution of computing". Except if you're in a research lab which works on operating systems, you'll hardly get any money from your OS as long as it has no practical uses (and it can remain the case for a very long time).


Some good reasons for trying hobby OS development

  • "I want to experiment with new OS design": You have your own idea of what an operating system should be like and you can't find it in the existing OS market ? Hobby OS development can be a way to try designing and implementing what you've envisioned and see if it actually works in practice.

  • "I'm now quite experienced with a language and looking for a programming challenge, or want to know how my computer works at the lowest levels": OS development certainly provides both, and to a great extent. You'll learn how to debug software with hardly anything more than text output facilities, how to do without a standard library or any sort, why so many people complain about this x86 architecture which works so well on your computers, what bad documentation from the vendor truly is...

  • "I've been working on an operating system for some time, know its codebase well, and have many gripes with it": If addressing these gripes would imply major changes, the kind of which makes software incompatible, you'll probably no choice but to fork, which while not described in this article is a totally valid path for OS development (and one which is certainly more rewarding than writing a new OS from scratch).

  • "The set of changes I want to bring to an OS is too big for simply patching or forking part of it": There sure is a point where starting a new project becomes the best option. If you think you have reached it, then welcome to the club!

  • "I'm looking for an exciting hobby": Well, depends a lot on what your definition of "exciting" is, of course, but OS development can be one of the most exciting hobbies on Earth if you're sufficiently prepared. It's somewhat akin to raising a child or growing plants: sure, it's slow, and sometimes it gets on your nerves and feels highly unrewarding. But this just makes success feel even better. Plus it's incredibly fun to be in full control of what's happening and have absolute power on your hardware.


So, are you ready?

Aside from the questioning phase, here are a few things which you must know or learn quickly in order to take this road successfully:

  • General computer science: You must know well and be able to fluently manipulate binary and hexadecimal numbers, boolean logic, data structures (arrays, linked links, hash tables, and friends), and sorting algorithms.

  • Computer architecture: You should know the global internal organisation of a desktop computer. The words and expressions ALU, interrupts, memory and PCI buses, and DMA should not sound mysterious to you. Knowledge of more automated mechanisms like the TLB, multilevel caching, or branch prediction, would be a very valuable asset in the long run, once you try to optimize code for performance.

  • Compiler and executable files internals: You must roughly know what happens during the preprocessing, compilation, and linking steps of binary generation, and be able to find everything you need to control your toolchain in its manual. You should also learn about linker scripting and the internal structure of executable files.

  • Assembly: You must know what Assembly is and how it works, and you must be able to code simple Assembly programs (playing with the stack and the registers, adding and subtracting numbers, implementing if-like branching...). You can write most of your operating system in other programming languages, but at some point you will need assembly snippets to do some tasks.

  • C or C++: Even if you plan to use another programming language to write your OS in, you'll need to know C or C++ in order to understand the code you'll find on the web when looking at other operating systems' source or other OS-deving tutorials.

  • Your system programming language: No matter whether you plan to write your OS in C#, Java, Assembly, BASIC, C, C++, Pascal, Erlang or REBOL, you should be very familiar with it. In fact, you should even have access to a description of its internals. As an example, you should know how to call a function written in your language of choice from Assembly code, what kind of runtime support your language requires and how it can be implemented.

(List based on this page of the OSdev wiki and tweaked based on personal experience.)

This ends the "Are you ready ?" part of this OS-deving tutorial. If you have reached this point, congratulations ! You should have the right mindset and sufficient knowledge to begin the hobby OS development adventure. The next article will explain to you how to set goals and expectations for your OS project, a truly vital step if you actually want it to get somewhere in the end.


(源地址:http://www.osnews.com/story/24270/Hobby_OS-deving_1_Are_You_Ready_


posted by  Hadrien Grasland on Sat 5th Feb 2011 10:59 UTC
IconSo you have  taken the test and you think you are ready to get started with OS development? At this point, many OS-deving hobbyists are tempted to go looking for a simple step-by-step tutorial which would guide them into making a binary boot, do some text I/O, and other "simple" stuff. The implicit plan is more or less as follow: any time they'll think about something which in their opinion would be cool to implement, they'll implement it. Gradually, feature after feature, their OS would supposedly build up, slowly getting superior to anything out there. This is, in my opinion, not the best way to get somewhere (if getting somewhere is your goal). In this article, I'll try to explain why, and what you should be doing at this stage instead in my opinion.

The importance of design

First, heavy reliance on tutorials is bad for creativity. More often than not we end up blindly following the thing without fully understanding the verbose paragraphs describing what we're doing, no matter which big red warning the author has put against this practice. What do we get this way? Many low-level tutorials on the web describe how to create a UNIX clone mainly written in C with a monolithic kernel and a simple console shell. Reminds you of something? That's normal, we already have tons of these around. It's quite sad to spend all this time in OS development only to get yet another UNIX-like kernel with more limited hardware support than the existing ones, in my opinion, though you might think otherwise.

Another reason for not following this approach is that having our head stuck in code makes us short-sighted. There are many pitfalls which can be avoided through some time spent designing our thing before coding. Simple examples : asynchronous I/O and multiprocessing. There's nothing intrinsically hard about supporting those, given sufficient doc on the subject, if you've made the decision to use them early enough and stick with it. It's just a different way of thinking. However, if you started without those and try to add them to your codebase later, the switching process will be quite painful.

Finally, the last reason why I think that design is important even for hobby projects is that their teams are small. Generally one or two developers. This is not necessarily a bad thing : in early stages of an OS project, you generally don't want endless arguments and heaps of code snippets written by different persons in different ways. The obvious drawback, however, is that unless you focus on a limited set of things and try to do these well, you're not going very far. This is why I think at this stage you should be not be writing code already but rather defining what the project's goals are, where you want to go.


Exploring motivations

To begin with, forget that you're targeting a specific computer, and even that you're writing code. You, as a team of 1-2, maybe more human beings, are designing something which will provide some kind of service to other human beings (which may simply be the members of the team). It all begins here. What is this service which you want to provide? When will you be able to say that your project is successful? When will you be able to say that you're not on the right track? To encounter success in a hobby OS project, like in many other kinds of personal projects, the first step is to define it clearly.

So go ahead, take a piece of paper or some other mean of text storage (in my case it was a blog), and explore your motivations. If you're writing an OS because you have some gripe with the existing ones you know of, spend some time finding out what you don't like in them. Try to understand why they are so, explore their evolution in time, read books on the subject if you can find them. Even if it turns out to be a random choice, you've found something : the importance of not doing this part randomly, and to pay particular attention to its design. If you're writing an OS in order to learn something, try to define what you want to learn in more details. Maybe you want to know where the process abstraction comes from? Through which process does one goes from a complex and dumb electrical appliance which does calculations to something which can actually be used by human beings without a high level of qualification?

In all the cases, take as much notes as possible without disturbing your thinking process. There are two reasons for this : first you can review said notes later, second translating thoughts in text forces you to be more rigorous than you would be if you could stick with ideas so vague that they can't even be directly expressed in English (or whatever language you may fluently speak). Try to be as precise as if these notes were to be published and read by other people who are not in your head.


Defining a target audience

After having defined why you're doing this, try to quickly define what your target audience is. An operating system is an interface between hardware, end users, and third-party software developers, so you have to define all three to some extent : what hardware is your operating system supposed to run on? Who is supposed to use it? Who is supposed to write software for it, and what kind of software? You don't have to be overly precise at this stage, but there are some things which you'd like to decide now.

On the hardware side: you should at least know how the users will interact with the hardware (Keyboard? Pointing devices? Which sort of each?), what range of screen areas you wish to cover, how powerful the oldest CPU which you want to target is and how much RAM is guaranteed to be there. You might also want to know right away if your OS requires an internet connection to operate properly, what sorts of mass storage devices you're going to use, and how third-party software (if there is some) will be installed.

Quickly check that the kind of hardware you want to code for is homebrew-friendly. Most desktops and laptops are (with maybe the upcoming ChromeOS notebooks as an exception), video games consoles are a mixed bag (Nintendo's ones often are, others generally aren't), products from Apple and Sony aren't until they are deprecated. A word of warning about cellphones: though they may sound like an attractive platform to code for, their hardware is generally-speaking extremely poorly documented, and you should keep in mind that even when you find one which does allow homebrew OSs to run and has sufficient documentation about it available on the web, the next generation is not guaranteed in any way to offer such comfort at all.

The end user is perhaps the one thing which is hardest to define precisely, except when the developers work only for themselves. You should probably keep around the description of a number of stereotypes representing the kind of people which you want to target. Some examples: "70-year old grandma who has been offered a computer by her grandchildren for Christmas. Never used one. Pay special attention to her sight problems.", "40-year old bearded sysadmin on a company's network. Won't use a computer if it can't be operated in CLI mode with a bash-like syntax. Wants to automate many tasks which are part of his job, think about powerful scripting engines. Needs multi-user support, and a special account to rule them all which only him may access.", "20-year old creative girl who draws things using a pen tablet and various free software. A computer is a tool like every single other for her, it shouldn't get in the way more than a block of paper."

Third-party developers; first thing to note is that if you want them to get interested in your project at some point, you'll first have to attract them as end users anyway. People don't write code for a platform which they wouldn't like to use, except when they are paid for it. On the other hand, devs have a special status in that, to state the obvious, they create software. One of your tasks will hence be to define which kind of software you let them create.

It can range from them writing vital parts of your operating system for you (frequently happens in the world of FOSS desktop operating systems) to every software being written in-house and third-party devs being non-existent (has grown out of fashion these days, except in some categories of embedded devices). Between these extreme situations, there's a wide range of possibilities. Here are a few...

  • The design/management of OS parts is proprietary, but the spec is disclosed, so that third-party developers are free to implement or re-implement parts of it the way they want

  • Most of the operating system itself is made in a proprietary and undisclosed fashion, but you're open to third-party drivers and low-level tools (Windows 9x, Mac OS)

  • Same as above, but you ask that applications needing access to some low-level or otherwise "dangerous" functionality go through some approval or signing process (Recent releases of Windows, Symbian)

  • You don't tolerate third-party low-level apps and will do everything you can to prevent their very existence, but you offer a native development kit for user applications with generous access to the system APIs

  • Same as above, but native third-party user apps are considered dangerous and must go through some signing/approval process (iOS)

  • No native third-party app, everything goes through managed apps which only have restricted access to system capabilities (Android, Windows Phone 7, most feature phones)

With all this information in mind, you should start to have a fairly good idea of where you're heading, and be ready for the next step.


Goals and frontiers

Now that the idea is there, try to put it on paper. Describe what your OS is supposed to achieve, for who, on which hardware, who will code what, and when you'll be able to tell that the project is successful according to its initial goals, that it has reached the 1.0 release, so to speak. By defining the project's goals you define criteria for objectively measuring its success and failure, which will prove to be a very valuable resource later, be it only as a mean to avoid feature bloat and other ways of wasting your very precious development resources.

In the end, look at this global view of the project you have now and ask yourself this simple question : "Do I think I can make it?". If you think you should maybe lower your expectations a little bit, now is the right time, because the later you do it the higher the damage done. Play with all this stuff you have gathered together, polish everything until you have a project you're satisfied with (or maybe drop this idea of writing an OS altogether if you don't feel like it any more), and then you'll be ready for the next step, namely designing your kernel.


(源地址:http://www.osnews.com/story/24341/Hobby_OS-deving_2_Setting_Goals_and_Expectations


posted by  Hadrien Grasland on Sun 20th Feb 2011 13:20 UTC
IconNow that you have an idea of  where your OS project is heading as a whole, it's time to go into specifics. The first component of your OS which you'll have to design, if you're building it from the ground up, is its kernel, so this article aims at being a quick guide to kernel design, describing the major areas which you'll have to think about and guiding you to places where you can find more information on the subject.

What is a kernel and what does it do?

I think it's best to put a definition there. The kernel is the central part of an operating system, whose role is to distribute hardware resources across other software in a controlled way. There are several reasons why such centralized hardware resource management is interesting, including reliability (the less power an application has, the less damage it can cause if it runs amok), security (for the very same reasons, but this time the app goes berserk intentionally), and several low-level system services which require a system-wide hardware management policy (pre-emptive multitasking, power management, memory allocation...).

Beyond these generic considerations, the central goal of most modern kernels is in practice to manage the process and thread abstractions. A process is an isolated software entity which can hold access to a limited amount of hardware resources, generally in an exclusive fashion, in order to avoid concurrency disasters. A thread is a task which may be run concurrently from other tasks. Both concepts are independent from each other, although it is common for each process to have at least one dedicated thread in modern multitasking OSs.

Hardware resources

So far, I've not given much depth to the "hardware resource" concept. When you read this expression, the first thing which you're thinking about is probably some pieces of real hardware which are actually fully independent from each other: mice, keyboards, storage devices, etc...

However, as you know, these peripherals are not directly connected to the CPU. They all are accessed via the same bus, through one single CPU port. So if you want to make sure that each process only has access to some peripherals, the kernel must be the one in control of the bus. Or you may also decide that the bus is the hardware resource which processes must request access to. Depending on how fine-grained your hardware resource model is, your position in the process isolation vs kernel complexity scale will vary.

To make things even more complex, modern OSs also manage some very useful hardware resources which do not actually exist from a pure hardware point of view. Consider, as an example, memory allocation. From a hardware point of view, there is only one RAM. You may have several RAM modules in your computer, but your CPU still sees them as one single, contiguous chunk of RAM. Yet you regularly want to allocate some part of it to one process, and another part of it to another process.

For this to work, the kernel has to take its finest cleaver and virtually slice the contiguous RAM in smaller chunks which can safely be allocated separately to various processes, based on each one's needs. There also has to be some mechanism for preventing different processes from peeking into each other's memory, which can be implemented in various ways but most frequently implies use of special hardware bundled in the CPU, the Memory Management Unit (MMU). This hardware allows the kernel to only give each process access to his limited region of memory, and to quickly switch between memory access permissions of various processes while the kernel is switching from one process to another.

Another typical example of abstract hardware resource is CPU time. I assume you have all noticed that desktop operating systems did not wait for multicore chips to appear before letting us run several applications at once. They all made sure that processes would share CPU time in some way, having the CPU frequently switching from one process to another, so that as a whole it looks like simultaneous execution under normal usage conditions.

Of course, this doesn't work by calling the CPU and telling it "Hi, fella, can you please run process A with priority 15 and process B with priority 8?". CPUs are fairly stupid, they just fetch an instruction of a binary, execute it, and then fetch the next one, unless some interrupt distracts them from their task. So in modern interactive operating systems, it's the kernel which will have to make sure that an interrupt occurs regularly, and that each time this interrupt occurs, a switch to another process occurs. This whole process is called pre-emptive multitasking.

Finally, it is common not to let processes access storage devices directly, but rather to give them access to some places in the file system. Of course, like allocated RAM, the file system is a purely virtual construct, which has no physical basis in HDDs/SSDs, and must be fully managed by the OS at some point.

In short, you'll have to define which hardware resources your kernel manages and gives processes access to. It is generally not a matter of just giving processes access to hardware x or not, there is often a certain amount of management work to be done in the kernel, compromises to be considered, and sometimes hardware resources must just be created by the kernel out of nowhere, as an example in the case of memory allocation, pre-emptive multitasking, and filesystem operation.

Inter-process communication and thread synchronization

Generally-speaking, the more isolated from each other processes are the better. As said earlier, malware can't do much in a tightly sandboxed environment, and reliability is greatly improved too. On the other hand, there are several occasions where it is convenient for processes to exchange information with each other.

Typical use case for this is a client-server architecture: somewhere in the depths of the system, there's a "server" process sleeping, waiting for orders. "Client" processes can wake it up and give it some work to do in a controlled way. At some point, "server" process is done and returns the result to "client" process. This way of doing things is especially common in the UNIX world. Another use case for inter-process communications is apps which are themselves made of several interacting processes.

There are several ways through which processes may communicate with each other. Here are a few:

  • Signals: The dumbest form of inter-process communication, akin to an interrupt. Process A "rings a bell" in process B. Said bell, called a signal, has a number associated to it, but nothing more. Process B may be waiting for this signal to arrive at the time, or have defined a function attached to it which is called in a new thread by the kernel when the process receives it.

  • Pipes and other data streams: Processes also frequently want to exchange data of various types. Most modern OSs provide a facility for doing this, although several ones only allow processes to exchange data on a byte-per-byte basis, for legacy reasons.

  • Remote/Distant procedure calls: Once we are able to both send data from one process to another and to send signals to other processes so that one of their methods gets called, it's highly tempting to combine both and allow one process to call methods from another process (in a controlled way, obviously). This approach allows one to use processes like shared libraries, with the added advantage that contrary to shared libraries, processes may hold access to resources which the caller doesn't have access to, giving the caller access to these resources in a controlled way.

  • Shared memory: Although in most cases processes are better isolated from each other, it may sometimes be practical for two processes to share a chunk of RAM and just do whatever they want in it without having the kernel going in the way. This approach is commonly used under the hood by kernels to speed up data transfers and to avoid loading shared libraries several times when several processes want to use them, but some kernels also make this functionality publicly available to processes which may have other uses for it.

Another issue, related to inter-process communication, is synchronization, that is situations where threads must act in a coordinated manner.

To get started to this problem, notice that in a multitasking environment, there are several occasions where you want to make sure that only a limited amount of threads (generally one) may access a given resource at a time. Imagine, as an example, the following scenario: two word processors are opened simultaneously, with different files inside of them. The user then brutally decides to print everything, and quickly clicks the "print" button of both windows.

Without a mechanism in place to avoid this, here's what would happen: both word processors start to feed data to the printer, which gets confused and prints garbled output, basically a mixture of both documents. Not a pretty sight. To avoid this, we must put somewhere in the printer driver a mechanism which ensures that only one thread may be printing a document at the same time. Or, if we have two printers available and if which one is used does not matter, we can have a mechanism which ensures that only two threads may be printing a document at the same time.

The usual mechanism for this is called a semaphore, and typically works as follows: on the inside, the semaphore has a counter which represents how much time a resource can still be accessed. Each time a thread tries to access a resource protected by a semaphore, this counter is checked. If its value is nonzero, it is decreased by one and the thread is permitted to access the resource. If its value is zero, the thread may not access the resource. Notice that to be perfectly bullet-proof, the mechanism in use must ensure that the value of the semaphore is checked and changed in a single processor instruction that can't be run on several CPU cores at once, which requires a bit of help from the hardware. It's not as simple as just checking and changing an integer value. But how exactly this is done is not our business at this design stage.

Apart from semaphores, another, less frequently used but still well-known synchronization mechanism, is the barrier. It allows N threads to wait until each one has finished its respective work before moving on. This is particularly useful in situations where a task is parallelized in several chunks that may not necessarily take the same time to be completed (think as an example about rendering a 3D picture by slicing it in a number of parts and having each part be computed by a separate thread).

So in short, having defined your process model, you'll have to define how they will communicate with each other, and how threads' actions will be synchronized.

Major concerns and compromises

You may have noticed that I've done my best to stick to generic concepts and put some care in showing that there are several ways to do a given thing. That's not just for fun. There are several compromises to play with when designing a kernel, and depending on what your OS project's goals are, you'll probably have to consider them in different ways.

Here's an overview of some major concerns which you should have in mind when designing a kernel (though the importance of each varies depending on what your OS project is):

  • Performance: Both in terms of hardware resource use and in terms of performance perceived by the user. These are not totally the same. As an example, in a desktop operating system, prioritizing software which the user is directly interacting with over background services improves perceived performance without requiring much optimization work. In a real-time operating system, hardware resource usage does not matter as much as meeting deadlines. And so on...

  • Maintainability: Kernels are pretty hard to code, therefore you generally want them to last a fairly long time. For this to happen, it must be easy to grab their code and tinker with it as soon as a flaw is found or a feature is to be added. The codebase must thus be kept as short as possible, well-commented, well-documented, well-organized, and leave room for tweaking things without breaking the API.

  • Portability: In our world of quickly-evolving hardware, operating system kernels should be easily portable to a new architecture. This is especially true in the realm of embedded devices, where the hardware is much more complex and ever-changing than it is on the desktop.

  • Scalability: The other side of the portability coin is that your kernel should adapt itself to future hardware evolutions in a given architecture. This noticeably implies optimizing for multi-core chips: you should strive to restrict the parts of the kernel which can only be accessed by a limited number of threads to a minimal amount of code, and aggressively optimize your kernel for multi-threaded use.

  • Reliability: Processes should not crash. But when they do crash, the impact of their crash should be minimized, and recoverability should be maximized. This is where maximizing process isolation, reducing the amount of code in loosely-isolated processes, and investigating means of backing up process data and restarting even the most critical services without rebooting the OS really shines.

  • Security: On platforms which allow untrusted third-party software to run, there should be some protection against malware. You must understand right away that things like open-source, antiviruses, firewalls, and having software validated by a bunch of testers, are simply neither enough nor very efficient. These should only be tools for paranoids and fallback methods when system security has failed, and system security should fail as infrequently as possible. Maximal isolation of processes is a way to reach that result, but you must also minimize the probability that system component can be exploited by low-privilege code.

  • Modularity: You'll generally want to make kernel components as independent from each other as possible. Aside from improving maintainability, and even reliability if you reach a level of modularity where you can restart failing kernel components on the fly without having the live system take a big hit, it also permits you to make some kernel features optional, a very nice feature, especially when applied to hardware drivers in kernels which include them.

  • Developer goodies: In the dark ages of DOS, it was considered okay to ask from developers that they literally code hardware drivers in their software, as the operating system would do nearly nothing for them. This is not the case anymore. For everything which you claim to support, you must provide nice and simple abstractions which hide the underlying complexity of hardware behind a friendly, universal interface.

  • Cool factor: Who'll use a new kernel if it's just the same as others, but in a much superior way? Let's introduce power-efficient scheduling, rickrolling panic screens, colored and formatted log output, and other fun and unique stuff!

Now, let's see how they end up in conflict with each other...

  • The quest for performance conflicts with everything but scalability when taken too far (writing everything in assembly, not using function calls, putting everything in the kernel for better speed, keeping the level of abstraction minimal...)

  • Maintainability conflicts with scalability, along with anything else that makes the codebase more complex, especially if the extra complexity can't be put in separate modules.

  • Portability is in conflict with everything that requires using or giving access to architecture-specific features, particularly when arch-specific code ends up being spread all over the place and not tightly packed in a known corner (as in some forms of performance optimizations).

  • Scalability is in conflict with any feature or construct which can't be used on 65536 CPU cores at the same time. Aside from the obvious compromise with maintainability and reliability which are better without hard-to-code and hard-to-debug threads spread all over the place, there's also a balance with some developer goodies (an obvious example being the libc and its hundreds of blocking system calls).

  • Reliability is the fiend of anything which adds complexity, as more code statistically means more bugs, especially when said code is hard to debug. The conflict with performance is particularly big, as many performance optimizations require to provide code with more access to hardware than it actually requires. It is also the sole design criteria in this list to have the awesome property of conflicting with itself, as some system features can improve reliability.

  • Security is a cousin of reliability as far as code complexity is concerned, since bugs can be exploitable. It also doesn't like low-level code where every single action is not checked (pointers arithmetic, C-style arrays...), which is more prone to exploitable failures than the rest.

  • Modularity doesn't like chunks of code which must be put at the same place of RAM. This means a serious conflict with performance, since code locality allows optimal use of CPU caches. The relationship between modularity and maintainability is ambiguous: separating system components from each other initially helps maintainability a lot, but extreme forms of modularity like putting the scheduler (part of the kernel which manages multitasking) in a process can make the code quite confusing.

  • We've previously seen that developer goodies and other cool stuff conflict with a large part of the rest for a number of reasons. Notice also an extra side of the feature vs maintainability conflict: it's easy to add features, but hard to remove them, and you don't know in advance how useful they will be. If you're not careful, this results in the phenomenon of feature bloat where the number of useless features grows exponentially with time. A way to avoid this is to keep the feature set minimal in the first release, then examine user feedback to see what is actually lacking. But beware of the "second-system effect", where you just put everything you're asked for in the second release, resulting in even worse feature bloat than if you had put a more extensive feature set to start with.

Some examples of kernel design

There are many operating system kernels in existence, though not all meet the same level of success. Here are a few stereotypical designs which tend to be quite frequently encountered (this list is by no mean exhaustive):

Monolithic kernels

The way all OS kernels were written long ago, for performance reasons, and still the dominant kernel design as of today. The monolithic kernel model remains quite attractive due to the extreme simplicity of its design. Basically, the kernel is a single big process running with maximum privileges and tending to include everything but the kitchen sink. As an example, it is common for desktop monolithic kernels to include facilities for rendering GPU-accelerated graphics and managing every single filesystem in existence.

Monolithic kernels shine especially in areas where high performance is needed, as everything is part of the same process. They are also easier to design, since the hardware resource model can be made simpler (only the kernel has direct access to hardware, user-space processes only have access to kernel-crafted abstractions), and since user-space is not a major concern until late in the development process. On the other hand, this way of doing things highers the temptation of using bad coding practices, resulting in unmaintainable, non-portable, non modular code. Due to the large codebase and the full access to hardware, bugs in a monolithic kernel are also more frequent and have a larger impact than in more isolated kernel designs.

Examples of monolithic kernel include Linux and its Android fork, most BSDs' kernels, Windows NTand XNU (Yes, I know, the two latter call themselves hybrid kernels, but that's mostly marketing. If you put most services in the same address space, with full access to the hardware, and without any form of isolation between each other, the result is still a monolithic kernel, with the advantages and drawbacks of monolithic kernels).

Micro-kernels

This is the exact opposite of a monolithic kernel in terms of isolation. The part of the kernel which has full access to the hardware is kept minimal (a few thousands of lines of executable code in the case of MINIX 3, to be compared with the millions of lines of code of monolithic kernels like Linux or Windows NT), and most kernel services are moved in separate services whose access to hardware is fine-tuned for their specific purpose.

Microkernels are highly modular by their very nature, and the isolated design favors good coding practices. Process isolation and fine-tuned access to hardware resources also ensure optimal reliability and security. On the other hand, microkernels are harder to write as much as they are easier to maintain, and the need to constantly switch from one process to another makes the most straightforward implementation perform quite poorly: it takes more optimization work to have a microkernel reach high performance, especially on the IPC side (as IPC becomes a critical mechanism).

Examples of commercial-grade microkernels include QNXµ-velOSity and PikeOS. On the research side, one can mention MINIX 3GNU Hurd, the L4 family, and the EROS family (KeyKOSEROS,CoyotosCapROS).

VM-based kernels

A fairly recent approach, which at the time has not fully gotten out of research labs and proof-of-concept demos. Maybe you'll be the one implementing it successfully. The idea here is that since most bugs and exploits in software come from textbook mistakes with native code (buffer overflows, dangling pointers, memory leaks...), native code is evil and should be phased out. The challenge is thus to code a whole operating system, including its kernel, in a managed language like C# or Java.

Benefits of this approach include obviously a very high cool factor and increased reliability and security. It could also potentially reach better performance than microkernels while providing similar isolation in a distant future, by isolating processes through a purely software mechanism (since all pointers and accesses to the hardware are checked by the virtual machine, no process may access resources which it's not allowed to access). On the other hand, nothing is free in the world of kernel development, and VM-based kernels have several major drawbacks to compensate for these high promises.

  • The kernel must include a full featured interpreter for the relevant language, which means that the codebase will be huge, hard to design, and that very nasty VM bugs are to be expected during implementation.

  • Making a VM fast enough that it is suitable for running kernel-level code full of hardware access and pointer manipulation is another big programming challenge.

  • A VM running on top of bare hardware will be harder to write, and thus more buggy and exploitable, than a VM running in the user space of an operating system. At the same time, exploits will have even worse consequences. Currently, the Java Virtual Machine is one of the biggest sources of vulnerabilities in existence on desktop PCs, so clearly something must change in the way we write VMs before they are suitable for inclusion in operating system kernels.

Examples of active VM-based kernel projects include SingularityJNodePhantomOS and Cosmos. There are also some interesting projects that are not active anymore, like JX and SharpOS (whose developers are now working in the MOSA project).

Bibliography, links, and friends

Having defined what the general concepts which you'll have to care about are, I bet you want to get into more details. In fact you should. So here is some material for going deeper than this introductory article on the subject of kernel design:

  • Modern Operating Systems (Andrew S. Tanenbaum): Should you only read one book on the subject, I strongly recommend this one. It is an enlightening, extensive description of the subject, covering a lot of aspects of kernel design and which you may also use in much more parts of your OS development work.

  • You may also find a list of other books, along with some reviews, on the OSdev wiki.

  • While you're on said wiki, you might also want to have a look at its main page, more precisely at the "Design Considerations" links in the left column (scroll down a bit). Globally, you should bookmark this website, because you'll have a vital need for it once you start working on implementation. It's, simply put, the best resource I know of on the subject.

  • And when you have question, also consider asking them in their forum. They are being asked hundreds of "how do I?" and "I'm stuck, what should I do?" implementation questions per month, so a bit of theoretical discussions would really please them. But beware of stupid questions which are answered in the wiki, otherwise prepare to face Combuster's sharp tongue.

  • Questions for an OS designer is also an interesting read, although it doesn't go too deeply into specifics. I should have linked to it in my previous article.

And that's all for now. Next time, we're going to go a bit more platform-specific, as I'm going to describe basic characteristics of the x86 architecture, which will be used for the rest of this tutorial (I'll noticeably explain why).


(源地址:http://www.osnews.com/story/24405/Hobby_OS-deving_3_Designing_a_Kernel


声明:本文内容由网友自发贡献,不代表【wpsshop博客】立场,版权归原作者所有,本站不承担相应法律责任。如您发现有侵权的内容,请联系我们。转载请注明出处:https://www.wpsshop.cn/w/小丑西瓜9/article/detail/139512
推荐阅读
相关标签
  

闽ICP备14008679号