openai-gpt_GPT-3不会承担您的编程工作

TL;DR; GPT-3 will not take your programming job (Unless you are a terrible programmer, in which case you would have lost your job anyway)

TL; DR; GPT-3不会接任您的编程工作(除非您是一个糟糕的程序员，否则您将失去工作)

Once again the hype of artificial intelligence has broken in the news. This time under the name of GPT-3, the successor of GPT-2 (of course), a model that is so large and so powerful that is making people think we finally made AGI, artificial general intelligence, possible (AGI is the kind of stuff that charlatans like Ben Goertzel keep claiming since a decade already).

新闻再次炒作了人工智能。 这次以GPT-3(当然是GPT-2的后继产品)的名义命名，该模型如此之大，功能如此强大，以至于人们认为我们终于使人工智能通用人工智能AGI成为可能(AGI是那种像Ben Goertzel这样的流氓已经自10年以来就一直宣称拥有这种东西)。

For those who are new to the topic, GPT-2 was a model in the NLP (Natural Language Processing) field of research that can generate text from an input sample. Basically, given a bunch of words or a structured sentence in English or another language, it will continue generating text that is consistent with the input. <sarcasm> Such an impressive result! Such an amazing artificial intelligence! </sarcasm>

对于那些刚接触该主题的人，GPT-2是NLP(自然语言处理)研究领域的模型，可以从输入样本中生成文本。 基本上，给定一堆用英语或另一种语言表示的单词或结构化句子，它将继续生成与输入内容一致的文本。 <sarcasm> Such an impressive result! Such an amazing artificial intelligence! </sarcasm>

However, the way such a model is trained is not magic nor mysterious at all. Let me be clear on this: it is not magic nor mysterious. In fact, it is very simple. Given a bunch of words, the model is asked to predict the next word that makes the most sense in that particular context. That is it. An old good statistician would do this with a Markov process or other probabilistic approaches that could lead to similar results. GPT-3 does this with 175 billion parameters (yep, no typos). The input text is nothing more than whatever is publically available from the Internet: discussion forums, Reddit threads, digitized books, websites, Wikipedia, you name it.

但是，训练这种模型的方式根本不是魔术也不是神秘的。 让我清楚一点：这既不是魔术也不是神秘。 实际上，这很简单。 给定一堆单词，要求模型预测在特定上下文中最有意义的下一个单词。 这就对了。 一个老的好统计学家会用马尔可夫过程或其他可能导致相似结果的概率方法来做到这一点。 GPT-3使用1750亿个参数(是，没有错别字)来完成此操作。 输入的文本无非是Internet上公开提供的内容：讨论论坛，Reddit主题，数字化书籍，网站，维基百科，您都可以命名。

如何使用GPT-3？ (How can one use GPT-3?)

Using GPT-3 is very simple (provided one has a really really fat machine packed with GPUs). It can be done in three easy steps, given the trained model (that would cost you several thousand dollars):

使用GPT-3非常简单(前提是其中有一台配备GPU的非常胖的机器)。 给定训练有素的模型，这可以通过三个简单的步骤来完成(这将花费您数千美元)：

1. provide a description of the task e.g. “translate English to french”
   提供任务说明，例如“将英语翻译为法语”
2. provide an example (optional) eg. the chair => la chaise
   提供示例(可选)，例如。 椅子=> la chaise
3. provide an “unseen” case e.g. the table =>…
   提供“看不见的”情况，例如表格=>…

and GPT-3 will magically make a translation for you because you know, <sarcasm> that's what artificial intelligence can do</sarcasm>.

和GPT-3会神奇地为您翻译，因为您知道<sarcasm> that's what artificial intelligence can do</sarcasm> 。

In fact, GPT-3 behaves just like a massive lookup-table — because that’s what it is — and search for something it already encountered in the input (during training). It does not even perform back-propagation due to the massive amount of parameters it is equipped with. Does it sound intelligent now? It certainly does not to me. The most intelligent component in GPT-3 is the transformer architecture that I have discussed extensively in a podcast episode

实际上，GPT-3的行为就像一个庞大的查找表 —因为它就是它—并在输入中(在训练过程中)搜索它已经遇到的东西。 由于配备了大量参数，它甚至不执行反向传播。 现在听起来很聪明吗？ 当然对我来说不是。 GPT-3中最智能的组件是我在播客中广泛讨论的变压器架构

人们如何看待GPT-3？ (What do people think of GPT-3?)

Among the many demos practitioners have been creating since the release of such a large lookup-table, there is one skill that GPT-3 seems it has acquired during training: computer programming. This has clearly alarmed several developers who are not really familiar with machine learning (though they know what lookup tables are). Not only can GPT-3 write computer code. Apparently, one can provide a description of how a web application should look like in plain English and GPT-3 would magically generate the source code that implements such an app.

自从发布如此大的查找表以来，从业人员一直在创建许多演示程序，其中GPT-3似乎在训练过程中获得了一项技能：计算机编程。 这显然使一些并不真正熟悉机器学习的开发人员感到震惊(尽管他们知道什么是查找表)。 GPT-3不仅可以编写计算机代码。 显然，可以用简单的英语描述Web应用程序的外观，而GPT-3会神奇地生成实现该应用程序的源代码。

<sarcasm> Finally we got rid of developers and saved a lot of $$ </sarcasm>

<sarcasm> Finally we got rid of developers and saved a lot of $$ </sarcasm>

Now let’s be serious and put things in perspective. We have a model that is great at looking at a bunch of words and predicting the next most appropriate word. Since each word is converted to a numeric vector — because guess what? computers are good with numbers — there is no way such a model would understand what that text is about (except under the terms of topic classification), nor how it is generated. Again, the only task that GPT-3 can perform is guessing one word, given a certain context.

现在，让我们认真点，将事情放在透视中。 我们有一个模型，可以很好地查看一堆单词并预测下一个最合适的单词。 由于每个单词都被转换为数字矢量-因为您猜怎么着？ 计算机具有良好的数字功能-这样的模型无法理解文本的含义(主题分类下除外)，也不知道其生成方式。 同样，GPT-3可以执行的唯一任务是在给定上下文的情况下猜测一个单词。

Specifically to the coding task, but this can be easily generalized, coding is the result of several skills and capabilities that go beyond language syntax. Code can be elegant, short, abstract, highly maintainable and it usually follows certain engineering principles (I am clearly referring to proper programming, no spaghetti here). While all this might be observed directly from the source code, it cannot be easily separated from it.

专门针对编码任务，但是可以很容易地将其概括，编码是超越语言语法的多种技能和能力的结果。 代码可以是优雅，简短，抽象，高度可维护的，并且通常遵循某些工程原理(我显然是指适当的编程，此处不涉及意大利面条)。 尽管所有这些都可以直接从源代码中观察到，但不能轻易将其与源代码分开。

To explain this in the language of real engineers, looking at millions of buildings can tell a lot about their material and shape. But very little about the construction principles and the geology that are definitely required for a building to be safe, durable, and resilient.

为了用实际工程师的语言解释这一点，查看数以百万计的建筑物可以说明很多有关其材料和形状的信息。 但是，对于建筑物安全，耐用和有弹性的绝对必需的构造原理和地质条件，知之甚少。

The biggest problem of machine learning algorithms is that they can only learn from data. When such data is biased, incomplete, or simply inaccurate, an observation of the phenomenon that will be extrapolated will also be biased, incomplete, and inaccurate.

机器学习算法的最大问题是它们只能从数据中学习。 当此类数据有偏差，不完整或仅是不准确时，对将要推断的现象的观察也将有偏差，不完整和不准确。

Moreover, GPT-3 needs an enormous amount of unbiased data. The Internet is exactly the place that lacks such a requirement.

此外，GPT-3需要大量的无偏数据。 互联网正是缺乏这种要求的地方。

In addition, good developers (and humans) do not need to read about pointers in C/C++ or lifetimes in Rust millions of times, for them to master such concepts. A model that learns the way GPT-3 does is, without loss of generality, a probabilistic model. Developers do not write code on a probabilistic basis (not even those who copy&paste from Stackoverflow).

此外，优秀的开发人员(和人员)无需阅读C / C ++中的指针或Rust中的生存期数百万次的知识，因此他们可以掌握这些概念。 在不失一般性的情况下，学习GPT-3方式的模型是概率模型。 开发人员不会以概率为基础编写代码(甚至不是从Stackoverflow复制粘贴的人员)。

To be clear one more time, when it comes to coding skills, GPT-3 is similar to the developer who has some familiarity with the syntax of a programming language, without knowing any type of abstraction behind it, and who is constantly referring to a massive dictionary of coding snippets. Rather than being concerned of killing his job, I’d be more concerned such a coder was in my team.

再说一次，在编码技能方面，GPT-3与开发人员相似，他对编程语言的语法有些熟悉，却不了解其背后的任何抽象类型，并且不断地引用大量的代码片段字典。 与其担心杀死他的工作，不如说我的团队中有这样的编码员。

Listen to the podcast version of this post.

听这篇文章的播客版本 。

Originally published at https://codingossip.github.io on July 24, 2020.

最初于 2020年7月24日 发布在 https://codingossip.github.io 。