Here you may find information on how to save/load fine-tuned model like BERT: https://huggingface.co/transformers/v1.0.0/model_doc/overview.html#serialization-best-practices. I hope this helps.