当前位置:   article > 正文

bert模型训练,加载保存的模型Can‘t load tokenizer for ‘/content/drive/MyDrive/Colab Notebooks/classification_mode_can't load tokenizer for 'bert-base-uncased'. if y

can't load tokenizer for 'bert-base-uncased'. if you were trying to load it
  1. # 5.保存训练好的模型参数
  2. import os
  3. model.save_pretrained("/content/drive/MyDrive/Colab Notebooks/test/classification_models_2/space1/")
  4. tokenizer.save_pretrained("/content/drive/MyDrive/Colab Notebooks/test/classification_models_2/space1/")
  5. torch.save(args,os.path.join("/content/drive/MyDrive/Colab Notebooks/test/classification_models_2/space1/","training_args.bin"))
  6. # 1. 载入训练好的模型
  7. args_eval={"model_name_or_path": "/content/drive/MyDrive/Colab Notebooks/test/classification_models_2/space1/",
  8. "config_name": "/content/drive/MyDrive/Colab Notebooks/test/classification_models_2/space1/",
  9. "tokenizer_name": "/content/drive/MyDrive/Colab Notebooks/classification_models_2/space1/",
  10. }
  11. config_class, tokenizer_class = MODEL_CLASSES["bert"]
  12. model_class=BertForClassification
  13. config = config_class.from_pretrained(
  14. args_eval["config_name"],
  15. finetuning_task="",
  16. cache_dir=None,
  17. )
  18. tokenizer = tokenizer_class.from_pretrained(
  19. args_eval["tokenizer_name"],
  20. do_lower_case=True,
  21. cache_dir=None,
  22. )
  23. model = model_class.from_pretrained(
  24. args_eval["model_name_or_path"],
  25. from_tf=bool(".ckpt" in args_eval["model_name_or_path"]),
  26. config=config,
  27. cache_dir=None,
  28. )
  29. model.to(device)
  1. OSError Traceback (most recent call last)
  2. <ipython-input-57-4658f3d30e04> in <module>()
  3. 17 args_eval["tokenizer_name"],
  4. 18 do_lower_case=True,
  5. ---> 19 cache_dir=None,
  6. 20 )
  7. 21 model = model_class.from_pretrained(
  8. /usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py in from_pretrained(cls, pretrained_model_name_or_path, *init_inputs, **kwargs)
  9. 1766 if all(full_file_name is None for full_file_name in resolved_vocab_files.values()):
  10. 1767 raise EnvironmentError(
  11. -> 1768 f"Can't load tokenizer for '{pretrained_model_name_or_path}'. If you were trying to load it from "
  12. 1769 "'https://huggingface.co/models', make sure you don't have a local directory with the same name. "
  13. 1770 f"Otherwise, make sure '{pretrained_model_name_or_path}' is the correct path to a directory "
  14. OSError: Can't load tokenizer for '/content/drive/MyDrive/Colab Notebooks/classification_models_2/space1/'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure '/content/drive/MyDrive/Colab Notebooks/classification_models_2/space1/' is the correct path to a directory containing all relevant files for a BertTokenizer tokenizer.

哈喽!

我用Bert预测评论分数,训练好模型保存到文件夹后,再一次加载它出现了上述错误,不太明白为什么,请教各位!

这是第二次训练这模型,也就是说我训练了一次之后,再把第一次训练的模型用新的数据训练,提高它的精确度,是可以的把,为什么第一次训练没有报错,第二次就报错了呢,我只改了保存的路径

声明:本文内容由网友自发贡献,不代表【wpsshop博客】立场,版权归原作者所有,本站不承担相应法律责任。如您发现有侵权的内容,请联系我们。转载请注明出处:https://www.wpsshop.cn/w/知新_RL/article/detail/347528
推荐阅读
相关标签
  

闽ICP备14008679号