当前位置:   article > 正文

用 Python 调用 GPT-3 API_pycharm引入gpt-3

pycharm引入gpt-3

用 Python 调用 GPT-3 API

GPT-3 是去年由 Open AI 推出的语言机器学习模型。它因其能够写作、写歌、写诗,甚至写代码而获得了广泛的媒体关注!该工具免费使用,只需要注册一个电子邮件即可。

GPT-3 是一种叫 transformer 的机器学习模型。具体来说,它就是 Generative Pre-training Transformer,因此叫做“GPT”。Transformer 架构使用自我注意和强化学习来模拟会话文本。通常,它一次处理一个单词,并使用前面的单词预测序列中的下一个单词。

GPT-3 具有广泛的应用场景,涵盖科学、艺术和技术等所有领域。它可以用来回答有关科学和数学的基本问题。甚至可以准确回答研究生级别的数学和科学概念相关的问题。更令人惊讶的是,我询问了一些与我的物理化学博士研究有关的问题,它能够提供较好的解释。不过,它也有其局限性。当我询问 GPT-3 有关物理化学中更新奇的研究方法时,它无法提供明确的答案。因此,在作为教育和研究的搜索引擎使用时,应该谨慎使用 GPT-3。GPT-3 没有事实核查功能。随着事实核查功能的提高,我可以想象 GPT-3 在研究生阶段甚至在研究领域将非常有用。

此外,除了我个人的经验外,我还看到了其他很多很酷的工具应用。例如,一个开发人员使用 GPT-3 来编排完成复杂任务的云服务。其他用户使用 GPT-3 生成了工作的 python 和 SQL 脚本,以及其他语言的程序。在艺术领域,用户请 GPT-3 写一篇比较现代和当代艺术的文章。GPT-3 的潜在应用几乎在任何领域都是丰富的。

GPT-3 在回答有准确内容的基本问题方面表现得很好。例如,它可以对光合作用做出相当不错的解释。它不能很好地回答关于光合作用的前沿研究问题,例如,它不能描述光合作用的机理和涉及的量子概念。它可以给出体面的回应,但不太可能提供大多数研究问题的技术细节。同样,GPT-3 可以编写一些简单的工作代码,但是随着任务的复杂度增加,生成的代码就越容易出错。它也不能生成政治观点、伦理价值观、投资建议、准确的新闻报道等通常是由人类生成的内容。

尽管 GPT-3 有其局限性,但其广泛适用性令人印象深刻。我认为提出一些有趣的数据科学和机器学习提示,以看看它们是否可以补充数据科学工作流程的部分是有趣的。

首先,我们将根据一些简单的提示生成一些与数据科学有关的文本。一旦我们对该工具有了一些了解,就可以询问一些可以帮助解决数据科学任务的问题。有几个有趣的数据科学和机器学习问题,我们可以向 GPT-3 询问。例如,是否可以使用 GPT-3 源自公开可用的数据集?GPT-3 的训练数据有多少等。另一个有趣的应用是问题框架。 GPT-3 可以帮助用户构建良好的机器学习研究问题吗?虽然它难以给出具体的技术答案,但也许它可以很好地构建出未解决的研究问题。

另一个很酷的应用是使用 GPT-3 来决定用于特定应用程序的 ML 模型。这很好,因为对于在线文献丰富的经过验证的技术,它应该能够很好地帮助用户选择模型,并解释为什么选定的模型最适合。最后,我们可以尝试使用GPT-3 编写一些数据科学任务的 Python 代码。例如,我们将看看是否可以使用它来编写生成特定用例的合成数据的代码。

注意:GPT-3 API 的结果是不确定的。因此,您获得的结果可能与此处显示的输出略有不同。此外,由于 GPT-3 没有事实核查机制,建议您对计划用于工作,学校或个人项目的任何事实结果进行双重核查。

在这项工作中,我将在 Deepnote 中编写代码,它是一个协作数据科学笔记本,使得运行可再现实验非常简单。

安装 GPT-3

首先,让我们到 Deepnote 并创建一个新项目(如果您还没有账户,可以免费注册)。

创建一个名为“GPT3”的项目以及该项目中的一个名为“GPT3_ds”的 notebook。

接下来,我们在第一个单元中使用 pip 安装 OpenAI:

  1. %pip install openai
  2. %pip install catboost
  3. 复制代码

将密钥保存在 openAI 对象的 api_key 属性:

  1. import openai
  2. openai.api_key = "your-key"
  3. 复制代码

接下来就可以提问了,比如问“什么是 Pandas 库”,GP3 会给反馈:

  1. completion = openai.Completion.create(engine="text-davinci-003", prompt="What is the pandas library?", max_tokens=1000)
  2. print(completion.choices[0]['text'])
  3. # output
  4. Pandas is an open source software library written in Python for data manipulation and analysis. Pandas is widely used in data science, machine learning and many other fields. It provides high-level data structures and tools for handling and manipulating data, including data frames, series, plotting tools and more.
  5. 复制代码

我们甚至可以询问更具体的问题,例如“Pandas 的一些常见用途是什么?”。它给出了合理的答案,列出了数据整理、数据可视化、数据聚合和时间序列分析:

  1. completion = openai.Completion.create(engine="text-davinci-003", prompt="what are some common Pandas use cases?", max_tokens=240)
  2. print(completion.choices[0]['text'])
  3. # output
  4. 1. Data Cleaning and Transformation
  5. 2. Data Analysis and Exploration
  6. 3. Time Series Analysis
  7. 4. Data Visualization
  8. 5. Statistical Modeling
  9. 6. Predictive Modeling
  10. 7. Machine Learning
  11. 8. Web Scraping
  12. 复制代码

询问“最常见的深度学习库是什么?”:

  1. #what are the most common deep learning libraries?
  2. completion = openai.Completion.create(engine="text-davinci-003", prompt="what are the most common deep learning libraries?", max_tokens=240)
  3. print(completion.choices[0]['text'])
  4. # output
  5. . TensorFlow
  6. 2. PyTorch
  7. 3. Keras
  8. 4. Caffe
  9. 5. CNTK
  10. 6. MXNet
  11. 7. Theano
  12. 8. Deeplearning4j
  13. 9. Gensim
  14. 10. LUNA
  15. 复制代码

甚至可以询问“什么是深度神经网络?”:

  1. completion = openai.Completion.create(engine="text-davinci-003", prompt="What is a deep neural network?", max_tokens=1000)
  2. print(completion.choices[0]['text'])
  3. # output
  4. A deep neural network (DNN) is a type of artificial neural network (ANN) with multiple layers of neurons between the input and output layers. DNNs are designed to learn complex non-linear relationships from data, and have been successfully applied in a wide range of areas such as image recognition, natural language processing, and financial forecasting.
  5. 复制代码

使用 GPT-3 进行事实搜索应该谨慎进行,因为没有任何事实核查机制。在大多数情况下,如果您想获得关于数据科学和机器学习的众多在线博客和论坛中已知概念的表面理解,GPT-3 应该能够很好地工作。

特征工程

GPT-3 的另一个有趣的应用案例是用作指导数据科学工作的辅助工具。例如,GPT-3 能否给出一些改进模型性能的特征变换的想法?让我们询问 GPT-3:

  1. completion = openai.Completion.create(engine="text-davinci-003", prompt="give some ideas on feature transformations that can improve model performance", max_tokens=1000)
  2. print(completion.choices[0]['text'])
  3. # output
  4. 1. Standardization/Normalization: A common feature transform used to ensure features are on the same scale, standardizing or normalizing variables can help limit the severity of outliers and improve the overall model performance.
  5. 2. Feature Binning: Binning is a process of transforming numerical variables into categorical ones. This can be useful when working with variables that have too many levels and can have a significant effect on the model performance.
  6. 3. Polynomial Expansion: When a nonlinear relationship is expected between features and the output variable, a polynomial expansion feature transformation can help improve model performance.
  7. 4. Feature Selection: Removing redundant or irrelevant features from the dataset can help improve the model performance as these features may lead to overfitting.
  8. 5. Ensemble: Combining different types of models (or different versions of the same model) can often improve performance due to their combined capabilities.
  9. 复制代码

我们看到它给出了一些很好的特征变换建议以及每个变换的解释。

让我们看看是否可以更进一步。让它写一些 Python 代码:

  1. completion = openai.Completion.create(engine="text-davinci-003", prompt="Write example python code that performs data standardization", max_tokens=1000)
  2. print(completion.choices[0]['text'])
  3. #output
  4. # Import the necessary libraries
  5. import numpy as np
  6. # Define the data
  7. data = np.array([[-3, 9, 0, 8],
  8. [ 4, 6, 5, 12],
  9. [20, 2, 3, 15]])
  10. # Calculate mean and standard deviation
  11. mean = np.mean(data, axis=0)
  12. std = np.std(data, axis=0)
  13. # Perform data standardization
  14. standardized_data = (data - mean) / std
  15. # Print the results
  16. print(standardized_data)
  17. 复制代码

复制并粘贴到一个新单元格中并运行它:

  1. # Import the necessary libraries
  2. import numpy as np
  3. # Define the data
  4. data = np.array([[-3, 9, 0, 8],
  5. [ 4, 6, 5, 12],
  6. [20, 2, 3, 15]])
  7. # Calculate mean and standard deviation
  8. mean = np.mean(data, axis=0)
  9. std = np.std(data, axis=0)
  10. # Perform data standardization
  11. standardized_data = (data - mean) / std
  12. # Print the results
  13. print(standardized_data)
  14. # output
  15. [[-1.03881504 1.16247639 -1.29777137 -1.27872403]
  16. [-0.31164451 0.11624764 1.13554995 0.11624764]
  17. [ 1.35045955 -1.27872403 0.16222142 1.16247639]]
  18. 复制代码

接下来,让我们对特征标准化变换做同样的操作:

  1. completion = openai.Completion.create(engine="text-davinci-003", prompt="Write example python code that performs data normalization on fake data", max_tokens=1000)
  2. print(completion.choices[0]['text'])
  3. # output
  4. # Normalizing data will rescale features in the range [0,1]
  5. data = [3, 7, 10, 13] # Sample data
  6. # Calculate the maximum and minimum of the data
  7. max_data = max(data)
  8. min_data = min(data)
  9. # Normalize the data
  10. normalized_data = [(x-min_data)/(max_data-min_data) for x in data]
  11. # Print first value to check
  12. print(normalized_data[0]) # Prints 0.2
  13. 复制代码

执行返回的代码:

  1. # Normalizing data will rescale features in the range [0,1]
  2. data = [3, 7, 10, 13] # Sample data
  3. # Calculate the maximum and minimum of the data
  4. max_data = max(data)
  5. min_data = min(data)
  6. # Normalize the data
  7. normalized_data = [(x-min_data)/(max_data-min_data) for x in data]
  8. # Print first value to check
  9. print(normalized_data)
  10. 复制代码

模型选择

另一个酷的应用是模型选择。让我们看看 GPT-3 是否可以指导我们选择机器学习模型。询问“如何选择时间序列模型?”

  1. completion = openai.Completion.create(engine="text-davinci-003", prompt="How do i select a time series model?", max_tokens=1000)
  2. print(completion.choices[0]['text'])
  3. # output
  4. The selection of a time series model depends on the type of data that is being analyzed. If there are long-term trends or cycles in the data, then an autoregressive integrated moving average (ARIMA) model or a exponential smoothing (ETS) model may be the best option. If the data are seasonal or there are outliers present, then a Holt-Winters model may be a better choice. If the data do not contain any long-term trends or seasonality, a Box-Jenkins model may be suitable. Regardless of the model selected, it is important to consider the type of data that is being modeled when selecting a time series model.
  5. 复制代码

答案是合理的,列出了平稳性、季节性、趋势以及模型性能评估的检查。让我们询问更具体的问题,例如“何时使用 ARIMA vs Prophet?”:

  1. completion = openai.Completion.create(engine="text-davinci-003", prompt="When to use ARIMA vs FB prophet?", max_tokens=1000)
  2. print(completion.choices[0]['text'])
  3. # output
  4. ARIMA and FB Prophet are both used for time series forecasting. ARIMA is a linear model and is better for predicting long-term trends with stationary data. FB Prophet is a non-linear model and is better for predicting short-term trends with non-stationary data. So, you would use ARIMA when you are mainly concerned with predicting long-term trends, and you would use FB Prophet when you are mainly concerned with predicting short-term trends.
  5. 复制代码

生成合成数据

能用 GPT-3 生成合成数据吗?让我们看一看!我们将生成一个众所周知的数据:医院再入院和交易数据。

  • 医院再入院数据

询问 GPT-3:“编写 python 代码,生成合成医院再入院数据”。

  1. completion = openai.Completion.create(engine="text-davinci-003", prompt="Write example python code that generates synthetic healthcare readmission data stored in a dataframe", max_tokens=1000)
  2. print(completion.choices[0]['text'])
  3. # output
  4. import pandas as pd
  5. import numpy as np
  6. # Create Dataframe
  7. df = pd.DataFrame(columns=['Patient_ID', 'Age', 'Admission_Type', 'Readmitted'])
  8. # Generate Data
  9. np.random.seed(0)
  10. for i in range(10):
  11. admission_type = np.random.choice(['Urgent', 'Scheduled', 'Emergency'])
  12. patient_age = np.random.randint(18, 80)
  13. readmission = np.random.choice([0, 1])
  14. df.loc[i] = [i+1, patient_age, admission_type, readmission]
  15. # Print Dataframe to Console
  16. print(df)
  17. 复制代码

执行此代码:

  1. import pandas as pd
  2. import numpy as np
  3. # Create Dataframe
  4. df = pd.DataFrame(columns=['Patient_ID', 'Age', 'Admission_Type', 'Readmitted'])
  5. # Generate Data
  6. np.random.seed(0)
  7. for i in range(10):
  8. admission_type = np.random.choice(['Urgent', 'Scheduled', 'Emergency'])
  9. patient_age = np.random.randint(18, 80)
  10. readmission = np.random.choice([0, 1])
  11. df.loc[i] = [i+1, patient_age, admission_type, readmission]
  12. # Print Dataframe to Console
  13. df
  14. 复制代码

输出结果:

让我们看看是否可以用这个合成数据构建一个分类模型,预测重新入院的人,并评估性能。

  1. completion = openai.Completion.create(engine="text-davinci-003", prompt="Write example python code that generates synthetic healthcare readmission data stored in a dataframe. From this write code that builds a catboost model that predicts readmission outcomes. Also write code to calculate and print performance", max_tokens=3000)
  2. print(completion.choices[0]['text'])
  3. # output
  4. metrics
  5. ## Generate Synthetic Healthcare Readmission Data
  6. import pandas as pd
  7. import numpy as np
  8. # set the seed for reproducibility
  9. np.random.seed(1)
  10. # create dataframe
  11. df = pd.DataFrame(np.random.randint(0, 100, size=(100, 10)), columns=['age','gender','length_of_stay','diagnosis','NIV','laboratory','past_hospitalizations','medications','bmi','readmission'])
  12. # add labels to data frame
  13. df['age'] = np.random.randint(20, 80, size=(100))
  14. df['gender'] = np.random.randint(1, 2, size=(100))
  15. df['length_of_stay'] = np.random.randint(2, 14, size=(100))
  16. df['diagnosis'] = np.random.randint(1, 5, size=(100))
  17. df['NIV'] = np.random.randint(0, 2, size=(100))
  18. df['laboratory'] = np.random.randint(1, 6, size=(100))
  19. df['past_hospitalizations'] = np.random.randint(0, 10, size=(100))
  20. df['medications'] = np.random.randint(1, 6, size=(100))
  21. df['bmi'] = np.random.randint(18, 35, size=(100))
  22. df['readmission'] = np.random.randint(0, 2, size=(100))
  23. # print the dataframe
  24. print(df)
  25. ## Build a CatBoost Model
  26. from catboost import CatBoostClassifier
  27. from sklearn.metrics import confusion_matrix
  28. # separate X and y
  29. X = df.iloc[:, 0:9]
  30. y = df.iloc[:, 9]
  31. # initialize catboost classifier
  32. cat_clf = CatBoostClassifier(iterations=50,
  33. learning_rate=0.3,
  34. depth=8,
  35. eval_metric='Accuracy',
  36. random_seed=42)
  37. # fit the model
  38. cat_clf.fit(X, y)
  39. # predict values
  40. y_pred = cat_clf.predict(X)
  41. # print confusion matrix
  42. conf_mat = confusion_matrix(y, y_pred)
  43. print(conf_mat)
  44. ## Calculate and Print Performance Metrics
  45. from sklearn.metrics import accuracy_score, precision_score, recall_score
  46. # calculate performance metrics
  47. acc = accuracy_score(y, y_pred)
  48. precision = precision_score(y, y_pred)
  49. recall = recall_score(y, y_pred)
  50. # print performance metrics
  51. print("Accuracy: {}".format(acc))
  52. print("Precision: {}".format(precision))
  53. print("Recall: {}".format(recall))
  54. 复制代码

复制上面的代码,并执行:

  1. ## Generate Synthetic Healthcare Readmission Data
  2. import pandas as pd
  3. import numpy as np
  4. # set the seed for reproducibility
  5. np.random.seed(1)
  6. # create dataframe
  7. df = pd.DataFrame(np.random.randint(0, 100, size=(100, 10)), columns=['age','gender','length_of_stay','diagnosis','NIV','laboratory','past_hospitalizations','medications','bmi','readmission'])
  8. # add labels to data frame
  9. df['age'] = np.random.randint(20, 80, size=(100))
  10. df['gender'] = np.random.randint(1, 2, size=(100))
  11. df['length_of_stay'] = np.random.randint(2, 14, size=(100))
  12. df['diagnosis'] = np.random.randint(1, 5, size=(100))
  13. df['NIV'] = np.random.randint(0, 2, size=(100))
  14. df['laboratory'] = np.random.randint(1, 6, size=(100))
  15. df['past_hospitalizations'] = np.random.randint(0, 10, size=(100))
  16. df['medications'] = np.random.randint(1, 6, size=(100))
  17. df['bmi'] = np.random.randint(18, 35, size=(100))
  18. df['readmission'] = np.random.randint(0, 2, size=(100))
  19. # print the dataframe
  20. print(df)
  21. ## Build a CatBoost Model
  22. from catboost import CatBoostClassifier
  23. from sklearn.metrics import confusion_matrix
  24. # separate X and y
  25. X = df.iloc[:, 0:9]
  26. y = df.iloc[:, 9]
  27. # initialize catboost classifier
  28. cat_clf = CatBoostClassifier(iterations=50,
  29. learning_rate=0.3,
  30. depth=8,
  31. eval_metric='Accuracy',
  32. random_seed=42)
  33. # fit the model
  34. cat_clf.fit(X, y)
  35. # predict values
  36. y_pred = cat_clf.predict(X)
  37. # print confusion matrix
  38. conf_mat = confusion_matrix(y, y_pred)
  39. print(conf_mat)
  40. ## Calculate and Print Performance Metrics
  41. from sklearn.metrics import accuracy_score, precision_score, recall_score
  42. # calculate performance metrics
  43. acc = accuracy_score(y, y_pred)
  44. precision = precision_score(y, y_pred)
  45. recall = recall_score(y, y_pred)
  46. # print performance metrics
  47. print("Accuracy: {}".format(acc))
  48. print("Precision: {}".format(precision))
  49. print("Recall: {}".format(recall))
  50. # output
  51. 复制代码
  • 交易数据

询问 GPT-3:“编写 Python 代码,生成交易数据”。

  1. completion = openai.Completion.create(engine="text-davinci-003", prompt="Write example python code that generates synthetic transaction data stored in a dataframe", max_tokens=1000)
  2. print(completion.choices[0]['text'])
  3. # output
  4. import pandas as pd
  5. import numpy as np
  6. #create randomly generated customer data
  7. customer_id = np.arange(1,101)
  8. customer_names = [f'John Doe {x}' for x in range(1,101)]
  9. #create randomly generated transaction data
  10. transaction_id = np.arange(1,101)
  11. dates = [f'2020-07-{x}' for x in range(1,101)]
  12. amounts = np.random.randint(low=1, high=1000, size=(100,))
  13. #create dataframe with randomly generated data
  14. transaction_data = pd.DataFrame({'Customer ID': customer_id,
  15. 'Customer Name': customer_names,
  16. 'Transaction ID': transaction_id,
  17. 'Date': dates,
  18. 'Amount': amounts})
  19. print(transaction_data)
  20. 复制代码

拷贝代码,并执行:

  1. import pandas as pd
  2. import numpy as np
  3. #create randomly generated customer data
  4. customer_id = np.arange(1,101)
  5. customer_names = [f'John Doe {x}' for x in range(1,101)]
  6. #create randomly generated transaction data
  7. transaction_id = np.arange(1,101)
  8. dates = [f'2020-07-{x}' for x in range(1,101)]
  9. amounts = np.random.randint(low=1, high=1000, size=(100,))
  10. #create dataframe with randomly generated data
  11. transaction_data = pd.DataFrame({'Customer ID': customer_id,
  12. 'Customer Name': customer_names,
  13. 'Transaction ID': transaction_id,
  14. 'Date': dates,
  15. 'Amount': amounts})
  16. transaction_data
  17. 复制代码

(部分输出结果)

现在有物品 ID、客户和购买金额。让我们看看是否可以更具体一点。再增加年龄、性别和邮政编码。

  1. completion = openai.Completion.create(engine="text-davinci-003", prompt="Write example python code that generates synthetic transaction data stored in a dataframe. Include customer ID, transaction amount, item ID, item name, age, gender, and zipcode", max_tokens=2000)
  2. print(completion.choices[0]['text'])
  3. # output
  4. import pandas as pd
  5. import numpy as np
  6. rows = ['customer_ID', 'transaction_amnt', 'item_ID', 'item_name', 'age', 'gender', 'zipcode']
  7. data = pd.DataFrame(columns=rows)
  8. for i in range(1,100):
  9. customer_ID = int( np.random.uniform(100,600-100))
  10. transaction_amnt = np.random.uniform(1.25, 10.00)
  11. item_ID = int( np.random.uniform(1,35))
  12. item_name = np.random.choice(["phone", "tablet", "laptop", "smartwatch"])
  13. age = int( np.random.uniform(17,75))
  14. gender = np.random.choice(["male", "female"])
  15. zipcode = np.random.choice(["98101", "98200", "98469", "98801"])
  16. data.loc[i] = [customer_ID, transaction_amnt, item_ID, item_name, age, gender, zipcode]
  17. print (data)
  18. 复制代码

执行代码:

  1. import pandas as pd
  2. import numpy as np
  3. rows = ['customer_ID', 'transaction_amnt', 'item_ID', 'item_name', 'age', 'gender', 'zipcode']
  4. data = pd.DataFrame(columns=rows)
  5. for i in range(1,100):
  6. customer_ID = int( np.random.uniform(100,600-100))
  7. transaction_amnt = np.random.uniform(1.25, 10.00)
  8. item_ID = int( np.random.uniform(1,35))
  9. item_name = np.random.choice(["phone", "tablet", "laptop", "smartwatch"])
  10. age = int( np.random.uniform(17,75))
  11. gender = np.random.choice(["male", "female"])
  12. zipcode = np.random.choice(["98101", "98200", "98469", "98801"])
  13. data.loc[i] = [customer_ID, transaction_amnt, item_ID, item_name, age, gender, zipcode]
  14. data
  15. 复制代码

(部分输出结果)

公共数据集的询问提示

另一种应用是询问 GPT-3 关于公共数据集。让我们询问 GPT-3 列出一些公共数据集:

  1. completion = openai.Completion.create(engine="text-davinci-003", prompt=" list some good public datasets", max_tokens=1000)
  2. print(completion.choices[0]['text'])
  3. # output
  4. 1. US Census Data
  5. 2. Enron Email Dataset
  6. 3. Global Open Data Index
  7. 4. Air Quality Monitoring Data
  8. 5. New York City Taxi Trip Data
  9. 6. IMF Data
  10. 7. World Bank Open Data
  11. 8. Google Books Ngrams Dataset
  12. 9. Amazon Reviews Dataset
  13. 10. UCI Machine Learning Repository
  14. 复制代码

让我们看看是否可以找到根据 Apache 2.0 许可的公共数据。还询问源链接:

  1. completion = openai.Completion.create(engine="text-davinci-003", prompt=" list some good public datasets under apache 2.0 license. provide links to their source", max_tokens=1000, temperature=0)
  2. print(completion.choices[0]['text'])
  3. # output
  4. 1. OpenStreetMap: https://www.openstreetmap.org/
  5. 2. US Census Data: https://www.census.gov/data.html
  6. 3. Google Books Ngrams: https://aws.amazon.com/datasets/google-books-ngrams/
  7. 4. Wikipedia: https://dumps.wikimedia.org/enwiki/
  8. 5. US Government Spending Data: https://www.usaspending.gov/
  9. 6. World Bank Open Data: https://data.worldbank.org/
  10. 7. Common Crawl: http://commoncrawl.org/
  11. 8. Open Images: https://storage.googleapis.com/openimages/web/index.html
  12. 9. OpenFlights: https://openflights.org/data.html
  13. 10. GDELT: http://data.gdeltproject.org/
  14. 复制代码

虽然并不是所有这些链接都是正确的,但它在寻找源链接方面做得相当不错。Google Ngrams、Common Crawl和 NASA 数据都相当出色。如果不提供数据的确切位置,在大多数情况下,它提供了一个可以找到数据的网页链接。

再请求对数据进行描述。请注意,虽然结果可能重叠,但它们在每次运行时略有不同。据我所知,结果并不总是可以相同的:

  1. completion = openai.Completion.create(engine="text-davinci-003", prompt=" list some good public datasets under apache 2.0 license. provide links to their source and descriptions", max_tokens=1000, temperature=0)
  2. print(completion.choices[0]['text'])
  3. # output
  4. 1. OpenStreetMap: OpenStreetMap is a free, editable map of the world, created and maintained by volunteers and available for use under an open license. It contains millions of data points, including roads, buildings, and points of interest. Source: https://www.openstreetmap.org/
  5. 2. Google Books Ngrams: Google Books Ngrams is a dataset of over 5 million books from Google Books, spanning from 1500 to 2008. It contains word counts for each year, allowing researchers to track the usage of words over time. Source: https://aws.amazon.com/datasets/google-books-ngrams/
  6. 3. Wikipedia: Wikipedia is a free, open-source encyclopedia with millions of articles in hundreds of languages. It is available for use under the Creative Commons Attribution-ShareAlike license. Source: https://www.wikipedia.org/
  7. 4. Common Crawl: Common Crawl is a large-scale web crawl that collects data from over 5 billion webpages. It is available for use under the Apache 2.0 license. Source: https://commoncrawl.org/
  8. 5. Open Images Dataset: The Open Images Dataset is a collection of 9 million images annotated with labels spanning over 6000 categories. It is available for use under the Apache 2.0 license. Source: https://storage.googleapis.com/openimages/web/index.html
  9. 复制代码

机器学习问题整理

最后一个示例,将看看 GPT-3 是否可以帮助我们整理机器学习问题。

  • 询问用例

虽然 GPT-3 中的数据仅截至2021年,但它仍然可以帮助我们构建与今天仍然相关的 ML 用例。让我们询问“社交媒体中有哪些新兴的机器学习用例?”:

  1. completion = openai.Completion.create(engine="text-davinci-003", prompt="What are some emerging machine learning use-cases in social media?", max_tokens=1000, temperature=0)
  2. print(completion.choices[0]['text'])
  3. # output
  4. 1. Automated Content Curation: Automatically curating content from social media platforms to create personalized content feeds for users.
  5. 2. Sentiment Analysis: Analyzing user sentiment from social media posts to gain insights into customer opinions and preferences.
  6. 3. Social Media Monitoring: Using machine learning algorithms to monitor social media conversations and detect potential issues or trends.
  7. 4. Social Media Advertising: Leveraging machine learning to optimize social media advertising campaigns and target the right audience.
  8. 5. Social Media Recommendations: Using machine learning to recommend content to users based on their interests and preferences.
  9. 复制代码

我们也可以看到在医疗领域的回复:

  1. completion = openai.Completion.create(engine="text-davinci-003", prompt="What are some emerging machine learning use-cases in healthcare?", max_tokens=1000, temperature=0)
  2. print(completion.choices[0]['text'])
  3. # output
  4. 1. Automated Diagnosis: Machine learning algorithms can be used to analyze patient data and medical images to detect and diagnose diseases.
  5. 2. Personalized Medicine: Machine learning algorithms can be used to analyze patient data and medical images to create personalized treatment plans for each patient.
  6. 3. Drug Discovery: Machine learning algorithms can be used to analyze large datasets of chemical compounds to identify potential new drugs.
  7. 4. Clinical Decision Support: Machine learning algorithms can be used to analyze patient data and medical images to provide clinicians with real-time decision support.
  8. 5. Predictive Analytics: Machine learning algorithms can be used to analyze patient data and medical images to predict future health outcomes.
  9. 复制代码

询问研究项目

现在让我们看看它是否可以提出一些研究问题。让我们询问:“使用深度学习进行图像检测有哪些好的研究项目?”

  1. completion = openai.Completion.create(engine="text-davinci-003", prompt="What are some good research questions on using deep learning for image detection?", max_tokens=1000, temperature=0)
  2. print(completion.choices[0]['text'])
  3. # output
  4. 1. What are the most effective deep learning architectures for image detection?
  5. 2. How can deep learning be used to improve the accuracy of image detection?
  6. 3. What are the most effective methods for training deep learning models for image detection?
  7. 4. How can deep learning be used to detect objects in images with varying levels of complexity?
  8. 5. How can deep learning be used to detect objects in images with varying levels of illumination?
  9. 6. How can deep learning be used to detect objects in images with varying levels of noise?
  10. 7. How can deep learning be used to detect objects in images with varying levels of resolution?
  11. 8. How can deep learning be used to detect objects in images with varying levels of occlusion?
  12. 9. How can deep learning be used to detect objects in images with varying levels of background clutter?
  13. 10. How can deep learning be used to detect objects in images with varying levels of rotation?
  14. 复制代码

再问一下 NLP 方向:

  1. completion = openai.Completion.create(engine="text-davinci-003", prompt="What are some good research questions related to NLP transformer models?", max_tokens=1000, temperature=0)
  2. print(completion.choices[0]['text'])
  3. # output
  4. 1. How can transformer models be used to improve the accuracy of natural language processing tasks?
  5. 2. What are the most effective methods for training transformer models for natural language processing tasks?
  6. 3. How can transformer models be used to improve the efficiency of natural language processing tasks?
  7. 4. What are the most effective methods for optimizing transformer models for natural language processing tasks?
  8. 5. How can transformer models be used to improve the interpretability of natural language processing tasks?
  9. 6. What are the most effective methods for deploying transformer models for natural language processing tasks?
  10. 7. How can transformer models be used to improve the scalability of natural language processing tasks?
  11. 8. What are the most effective methods for combining transformer models with other natural language processing techniques?
  12. 9. How can transformer models be used to improve the robustness of natural language processing tasks?
  13. 10. What are the most effective methods for evaluating transformer models for natural language processing tasks?
  14. 复制代码

本文所有代码都发布在 GitHub 上。

作者:老齐Py
链接:用 Python 调用 GPT-3 API - 掘金
来源:稀土掘金
著作权归作者所有。商业转载请联系作者获得授权,非商业转载请注明出处。

声明:本文内容由网友自发贡献,不代表【wpsshop博客】立场,版权归原作者所有,本站不承担相应法律责任。如您发现有侵权的内容,请联系我们。转载请注明出处:https://www.wpsshop.cn/w/小惠珠哦/article/detail/970615
推荐阅读
相关标签
  

闽ICP备14008679号