philschmid/aws-neuron-samples

Training and Inference Examples for Hugging Face Transformers and AWS Neuron (Inf1, Inf2, Trn1)

This repository contains instructions/examples/tutorials using the neuron-sdk for running inference and training Hugging Face libraries like transformers, datasets with AWS Accelerators including AWS Inferentia & AWS Trainium.

Inference

• Speed up BERT inference with Hugging Face Transformers and AWS Inferentia
• Accelerate Hugging Face Transformers with AWS Inferentia 2 | Benchmarking Transformers for AWS Inferentia 2

Training

Resources

• Github: aws-neuron-sdk
• Neuron Documentation
• Getting Started Neuron

Requirements

Before we can start make sure you have met the following requirements

• AWS Account with quota for Inf1, Inf2, Trn1 instances.
• AWS CLI installed
• AWS IAM user configured in CLI with permission to create and manage ec2 instances

Start environment

inf2:

source ../aws_neuron_venv_pytorch_p37/bin/activate