在SageMaker上部署Hugging Face模型_sagermaker部署huggingface模型

在SageMaker上部署Hugging Face模型

初次编辑时间：2024/2/19；最后编辑时间：2024/2/19

在本地运行sagemaker

1. 在aws上创建访问密钥：

   1. 右上角用户名-安全
   2. 向下拉，创建访问密钥
   3. 复制访问密钥，私密密钥

2. 本地安装awscli

   1. python -m pip install awscli

3. 配置asw

   1. aws configure，配置完结果如下：

   (.venv) PS C:\Users\HP\PycharmProjects\pythonProject> aws configure
   AWS Access Key ID [None]: your_key
   AWS Secret Access Key [None]: you_secret_key
   Default region name [None]: your_region
   Default output format [None]: json
   1
   2
   3
   4
   5

4. 创建角色，步骤如下(转换成英文进行修改)：

   1. Open the IAM console at https://console.aws.amazon.com/iam/.
   2. Choose Roles and then choose Create role.
   3. Keep AWS service as the Trusted entity type and then use the down arrow to find SageMaker in Use cases for other AWS services.
   4. Choose SageMaker – Execution and then choose Next.
   5. The IAM managed policy, AmazonSageMakerFullAccess, is automatically attached to the role. To see the permissions included in this policy, choose the plus (+) sign next to the policy name. Choose Next.
   6. Enter a Role name and a Description.
   7. (Optional) Add additional permissions and tags to the role.
   8. Choose Create role.
   9. On the Roles section of the IAM console, find the role you just created. If needed, use the text box to search for the role using the role name.
   10. On the role summary page, make note of the ARN.

5. 设置本地环境的代码：

import sagemaker.huggingface
import sagemaker
import boto3
sess = sagemaker.Session()
# sagemaker session bucket -> used for uploading data, models and logs
# sagemaker will automatically create this bucket if it not exists
sagemaker_session_bucket=None
if sagemaker_session_bucket is None and sess is not None:
    # set to default bucket if a bucket name is not given
    sagemaker_session_bucket = sess.default_bucket()

try:
    role = sagemaker.get_execution_role()
except ValueError:
    iam = boto3.client('iam')
    role = iam.get_role(RoleName='your_role_name')['Role']['Arn'] # 第一个参数是你的角色名称

sess = sagemaker.Session(default_bucket=sagemaker_session_bucket)

print(f"sagemaker role arn: {role}")
print(f"sagemaker bucket: {sess.default_bucket()}")
print(f"sagemaker session region: {sess.boto_region_name}")
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

如果上述步骤均成功，会得到输出：

sagemaker role arn: your_role_arn
sagemaker bucket: your_bucket
sagemaker session region: your_region
1
2
3

一个可以直接运行的例子，使用huggingface transformers直接运行：

import sagemaker
import boto3

sess = sagemaker.Session()
# sagemaker session bucket -> used for uploading data, models and logs
# sagemaker will automatically create this bucket if it not exists
sagemaker_session_bucket = None
if sagemaker_session_bucket is None and sess is not None:
    # set to default bucket if a bucket name is not given
    sagemaker_session_bucket = sess.default_bucket()

try:
    role = sagemaker.get_execution_role()
except ValueError:
    iam = boto3.client('iam')
    role = iam.get_role(RoleName='segamaker-remote1')['Role']['Arn']  # 第一个参数是你的角色名称

sess = sagemaker.Session(default_bucket=sagemaker_session_bucket)

print(f"sagemaker role arn: {role}")
print(f"sagemaker bucket: {sess.default_bucket()}")
print(f"sagemaker session region: {sess.boto_region_name}")

from sagemaker.huggingface.model import HuggingFaceModel

# Hub model configuration <https://huggingface.co/models>
hub = {
    'HF_MODEL_ID': 'distilbert-base-uncased-distilled-squad',  # model_id from hf.co/models, onnx format
    'HF_TASK': 'question-answering'  # NLP task you want to use for predictions
}

# create Hugging Face Model Class
huggingface_model = HuggingFaceModel(
    env=hub,  # configuration for loading model from Hub
    role=role,  # IAM role with permissions to create an endpoint
    transformers_version="4.26",  # Transformers version used
    pytorch_version="1.13",  # PyTorch version used
    py_version='py39',  # Python version used
)

# deploy model to SageMaker Inference
predictor = huggingface_model.deploy(
    initial_instance_count=1,
    instance_type="ml.m5.xlarge"  # 用于实时推理，暂时是免费的
)

# example request: you always need to define "inputs"
data = {
    "inputs": {
        "question": "What is used for inference?",
        "context": "My Name is Philipp and I live in Nuremberg. This model is used with sagemaker for inference."
    }
}
# request
result = predictor.predict(data)
print(result)  # {'score': 0.9987204670906067, 'start': 68, 'end': 77, 'answer': 'sagemaker'}
print(result['answer'])  # 'sagemaker'
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57

这只是一个简单的例子，更多的用法还在学习中，亟待更新…