awslabs/aws-ai-solution-kit

Machine Learning APIs for common use cases, include: General OCR (Simplified/Traditional Chinese), Custom OCR, Image Similarity, Object Recognition, Face Detection, Face Comparison, Human Image Segmentation, Human Attribute Recognition, Pornography Detection, Image Super Resolution, Text Similarity, Car License Plate, etc.

English | 简体中文

AI Solution Kit

Common Machine Learning Features Kit

Documentation · Changelog · Feature List

Introduction

This repository contains several pre-trained deep learning models based on AWS Lambda and Amazon SageMaker, for example: general OCR, text similarity, face detection, human image segmentation, image similarity, object recognition, image super resolution (see full list below). By this project, you can download these dockerized modules from Amazon Elastic Container Registry(ECR) and create REST APIs on Amazon API Getaway. Once the project deployed, you can call the REST API to use these features easily.

For more information about the solution, please refer to our website.

Feature List

Optical Character Recognition(OCR)

Feature Name	Description
Lite OCR (Simplified Chinese)	Recognize and extract Simplified Chinese, numbers, alphabetical characters and symbols
Lite OCR (Traditional Chinese)	Recognize and extract Traditional Chinese, numbers, alphabetical characters and symbols
Advanced OCR (Multilingual)	Recognize and extract Simplified/Traditional Chinese, Vietnamese, Japanese, Korean, English, numbers, alphabetical characters and symbols. Return the information such as text or coordinates
Custom OCR	Recognize and extract structured text by predefined OCR templates
Car License Plate	Recognize text on Chinese car license plate

Facial & Body

Feature Name	Description
Face Comparison	Compare two faces of same person and return a confidence score of the similarity
Face Detection	Detect the face in a image and return coordinate information of the face
Human Attribute Recognition	Recognize the attributes of the human body in the image
Human Image Segmentation	Segment human bodies from background and return the alpha channel

Image Understanding

Feature Name	Description
Image Similarity	Compare two images and return similarity score
Object Recognition	Segment human bodies from background and return the alpha channel
Pornography Detection	Detect pornographic image in three dimensions (normal, sexy, porn) and return confidence scores

Computer Vision Production

Feature Name	Description
Image Super Resolution	Upscale the resolution and enhance details in the images

Natural Language Understanding(NLU)

Feature Name	Description
Text Similarity	Compare two Chinese words or sentences and return similarity score
General NLU	Support a variety of Chinese text understanding tasks, such as text classification, sentiment analysis, extraction, and customizable labeling systems

Quick deployment

This project is an AWS Cloud Development Kit(CDK) project written in Typescript, if you want to use the above deep learning features without building the entire project, you can use the Amazon CloudFormation template to deploy feature APIs quickly, the generated Amazon CloudFormation template is available at: https://aws-gcr-solutions.s3.amazonaws.com/Aws-gcr-ai-solution-kit/v1.4.0/AI-Solution-Kit.template

Below is the quick links to launch the AWS CloudFormation template into your AWS account

Region name	Region code	Launch
Global regions(switch to above region you want to deploy)	us-east-1(default)	Launch
AWS China(Beijing) Region	cn-north-1	Launch
AWS China(Ningxia) Region	cn-northwest-1	Launch

Once the AWS CloudFormation template is opened in your AWS account, you can choose the deep learning features in the Parameters section, the step-by-step instruction is available at: https://awslabs.github.io/aws-ai-solution-kit/en/deployment/

Build from source

You can also build this project from source.

Prerequisites

• An AWS account
• Configure credential of aws cli
• Install node.js LTS version, such as v14.x
• Install Docker Engine
• Install the dependencies of solution via executing command

yarn install && npx projen

• Initialize the CDK toolkit stack into AWS environment(only for deploying via AWS CDK first time)

yarn cdk-init

Deploy project

You can clone this repository to create a local copy on your computer and build this project in root directory by yarn.

• [Optional] Build docker images by yarn and push to an Amazon ECR repository

yarn build-containers

You can push images to an Amazon ECR repository, the step-by-step instructions is available at: https://docs.aws.amazon.com/AmazonECR/latest/userguide/docker-push-ecr-image.html

Once the docker images pushed to Amazon ECR repository, please replace the ecr registry in .projenrc.js file in project root directory.

context: {
    ecrRegistry: 'your-ECR-registry',
}

• Build CDK project

yarn build

• Deploy CDK project

Note If you do not build docker images, the ECR registry for pre-trained images will use default 'public.ecr.aws/aws-gcr-solutions/aws-gcr-ai-solution-kit'

Once the project built, you can deploy to your AWS account.

yarn deploy

Once the project deployed, please log in to your AWS console, choose to active deep learning features by updating the Amazon CloudFormation stack. The step-by-step instructions is available at: https://awslabs.github.io/aws-ai-solution-kit/en/deploy-add-delete-api/

Architecture

This project contains two types of implementation: Building on AWS Lambda and building on Amazon SageMaker.

Note The Amazon SageMaker type implementation is only available for image super-resolution feature.

• Building on AWS Lambda

1. Sending API requests to Amazon API Gateway. The request payload needs to contain the processed image or text.

2. Amazon API Gateway sends received user requests directly to AWS Lambda functions.

3. The AWS Lambda function returns the result(JSON format) to the caller.

• Building on Amazon SageMaker

1. Sending an API request to Amazon API Gateway. The request payload needs to contain the processed image or text.

2. Amazon API Gateway sends the request to the AWS Lambda function.

3. AWS Lambda invokes Amazon SageMaker Endpoint, executes the inference in Amazon SageMaker and returns results (JSON format).

API Reference

See API Reference

Authorization and Security

By default, the AI Gateway will enable the IAM authorization and Enable the CloudWatch Logs for accessing and debugging.

License

This project is licensed under the Apache-2.0 License.

aws-ai-solution-kit's Issues

Stack fails on PornographyDetection installation

Expected Behavior

Template should create stack

Current Behavior

Template fails

Possible Solution

How to reproduce (for bugs)

Import template and create stack with PornographyDetection | yes-lambda enabled

Stack creation fails with the following error, no other info provided.
Embedded stack arn:aws:cloudformation:us-east-1:948320724766:stack/AIImageDetector-PornographyDetectionNestedStackPornographyDetectionNestedStackResource-36PDX2AQZ7GR/b095a010-8c33-11ed-8a72-1232682a975b was not successfully created: The following resource(s) failed to create: [pornographydetectionpornographydetectionAppDCBBB4C1].

I can't create any service

Expected Behavior

I tested it many times, but it couldn't be used. I don't know what the problem is

Current Behavior

Unable to create any ai-solution-kit services, such as OCR, target recognition, etc.

The error:

Embedded stack arn:aws:cloudformation:us-east-1:975049896832:stack/AI-Solution-Kit-ObjectRecognitionSageMakerNestedStackObjectRecognitionSageMakerNestedS-1XG6MAPB45FLV/3590e2b0-f6e7-11ee-b785-12db9a32952b was not successfully created: The following resource(s) failed to create: [objectrecognitionconstructsagemakerEndpointEA8278DB].

Possible Solution

How to reproduce (for bugs)

from this url:

https://us-east-1.console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/create/template?stackName=AI-Solution-Kit&templateURL=https://aws-gcr-solutions.s3.amazonaws.com/Aws-gcr-ai-solution-kit/latest/AI-Solution-Kit.template

No operations defined in spec!

Expected Behavior

I expected the cloudformation template to launch the API for use

Current Behavior

With APIExplorer seeing No operations defined in spec!

Possible Solution

Look into the deployed api?

How to reproduce (for bugs)

Run the cloudformation template in any us-east-1

image super-resolution timeout / internal error

Use super-resolution with inf1 instance, I tried inf1.xlarge and inf1.6xlarge.

When super-resolution a large pixel image(4000x3000), got below errors,

• internal error when using inf1.xlarge instance
• timeout when using inf1.6xlarge instance

If the resolution of image can not be supported, pls give the friendly error message, like http status 4xx with proper information.

go 1.x runtime is no longer supported

Expected Behavior

Being able to deploy the template.

Current Behavior

CloudFormation errors out because of deprecated go1.x runtime.

Possible Solution

Move to provided.al2 or provided.al2023 runtimes.

Simply changing the value in the template does not seem to work though as the bootstrap handler is not found then.

Error: Couldn't find valid bootstrap(s): [/var/task/bootstrap /opt/bootstrap] Runtime.InvalidEntrypoint

How to reproduce (for bugs)

No method found matching route super-resolution/ for http method POST.

I've tried deploying this with and without authentication. I can get other endpoints to work (like pornography detection), however this endpoint (although it shows up in the "output" section) always returns this error:

{
    "message": "No method found matching route super-resolution/ for http method POST."
}

Reporting a vulnerability

Hello!

I hope you are doing well!

We are a security research team. Our tool automatically detected a vulnerability in this repository. We want to disclose it responsibly. GitHub has a feature called Private vulnerability reporting, which enables security research to privately disclose a vulnerability. Unfortunately, it is not enabled for this repository.

Can you enable it, so that we can report it?

Thanks in advance!

PS: you can read about how to enable private vulnerability reporting here: https://docs.github.com/en/code-security/security-advisories/repository-security-advisories/configuring-private-vulnerability-reporting-for-a-repository

Fail to deploy Lite OCR - Traditional Chinese

Dear Yan,

I encountered an issue while attempting to deploy the "Yes: Lite OCR - Traditional Chinese" solution.

Regrettably, the deployment process encountered a failure. Could you kindly assist me in identifying any oversights on my part?

The CloudFormation event status indicates the following:

Embedded stack arn:aws:cloudformation:us-east-1:xxxxxxxxxxxxxx:stack/AI-Solution-Kit-GeneralOCRTraditionalChineseNestedStackGeneralOCRTraditionalChineseNes-1VM9U651N5ADC/345f22c0-b5a7-11ee-9f27-12bf2aef6c25 was not successfully created. The following resource(s) failed to create: [generalocrtraditionalchinesegeneralocrtraditionalchineseApp89CAEB4E].

I appreciate your prompt attention to this matter and any assistance you can provide in resolving the deployment issue.

Best regards,
TC

api returns status 200 even the internal error happens

it’s related to use case in #2 ,

When super-resolution an image with internal error, however the API still responses http status 200. The error msg also expose the some internal information, such as account id.

Image Super Resolution 502

Execution log for request 410b4f79-3c8a-480e-9e29-25625f3d9784
Tue Nov 16 02:15:30 UTC 2021 : Starting execution for request: 410b4f79-3c8a-480e-9e29-25625f3d9784
Tue Nov 16 02:15:30 UTC 2021 : HTTP Method: POST, Resource Path: /resolution
Tue Nov 16 02:15:30 UTC 2021 : Method request path: {}
Tue Nov 16 02:15:30 UTC 2021 : Method request query string: {}
Tue Nov 16 02:15:30 UTC 2021 : Method request headers: {Content-Type= application/json}
Tue Nov 16 02:15:30 UTC 2021 : Method request body before transformations: {
  "url":"https://aikits.demo.solutions.aws.a2z.org.cn/img/sr-5.jpg"
}
Tue Nov 16 02:15:30 UTC 2021 : Endpoint request URI: https://lambda.us-east-1.amazonaws.com/2015-03-31/functions/arn:aws:lambda:us-east-1:958422602275:function:AIKitsSuperResolutionStac-humanseglambdaapp53B7B52-GJ7ESt0qZRFk/invocations
Tue Nov 16 02:15:30 UTC 2021 : Endpoint request headers: {X-Amz-Date=20211116T021530Z, x-amzn-apigateway-api-id=0pnzq370l8, Accept=application/json, User-Agent=AmazonAPIGateway_0pnzq370l8, Host=lambda.us-east-1.amazonaws.com, X-Amz-Content-Sha256=1bdcc6bde9e33fd77f3d296ea43d47fa25b680e764ca93656c0a069cce3dad0b, X-Amzn-Trace-Id=Root=1-61931442-e702479ee55d643149b8511a, x-amzn-lambda-integration-tag=410b4f79-3c8a-480e-9e29-25625f3d9784, Authorization=*********************************************************************************************************************************************************************************************************************************************************************************************************************************************635988, X-Amz-Source-Arn=arn:aws:execute-api:us-east-1:958422602275:0pnzq370l8/test-invoke-stage/POST/resolution, X-Amz-Security-Token=IQoJb3JpZ2luX2VjEMv//////////wEaCXVzLWVhc3QtMSJGMEQCIAcmMk7QzZrMp6fIOpqhIQSluMQRyi3sO5CHQwApdtvrAiAX+u55Xg9QMxMd3LERM0Aa1v/+gIWWvrdaCK [TRUNCATED]
Tue Nov 16 02:15:30 UTC 2021 : Endpoint request body after transformations: {"resource":"/resolution","path":"/resolution","httpMethod":"POST","headers":{"Content-Type":" application/json"},"multiValueHeaders":{"Content-Type":[" application/json"]},"queryStringParameters":null,"multiValueQueryStringParameters":null,"pathParameters":null,"stageVariables":null,"requestContext":{"resourceId":"pk31mf","resourcePath":"/resolution","httpMethod":"POST","extendedRequestId":"I4AabEFooAMFTEg=","requestTime":"16/Nov/2021:02:15:30 +0000","path":"/resolution","accountId":"958422602275","protocol":"HTTP/1.1","stage":"test-invoke-stage","domainPrefix":"testPrefix","requestTimeEpoch":1637028930564,"requestId":"410b4f79-3c8a-480e-9e29-25625f3d9784","identity":{"cognitoIdentityPoolId":null,"cognitoIdentityId":null,"apiKey":"test-invoke-api-key","principalOrgId":null,"cognitoAuthenticationType":null,"userArn":"arn:aws:iam::958422602275:user/fayland","apiKeyId":"test-invoke-api-key-id","userAgent":"aws-internal/3 aws-sdk-java/1.12.76 Linux/5.4.147-83.259.amzn [TRUNCATED]
Tue Nov 16 02:15:30 UTC 2021 : Sending request to https://lambda.us-east-1.amazonaws.com/2015-03-31/functions/arn:aws:lambda:us-east-1:958422602275:function:AIKitsSuperResolutionStac-humanseglambdaapp53B7B52-GJ7ESt0qZRFk/invocations
Tue Nov 16 02:15:30 UTC 2021 : Received response. Status: 200, Integration latency: 218 ms
Tue Nov 16 02:15:30 UTC 2021 : Endpoint response headers: {Date=Tue, 16 Nov 2021 02:15:30 GMT, Content-Type=application/json, Content-Length=1278, Connection=keep-alive, x-amzn-RequestId=9f578431-af11-4687-9cfb-dc93c457120b, X-Amz-Function-Error=Unhandled, x-amzn-Remapped-Content-Length=0, X-Amz-Executed-Version=$LATEST, X-Amzn-Trace-Id=root=1-61931442-e702479ee55d643149b8511a;sampled=0}
Tue Nov 16 02:15:30 UTC 2021 : Endpoint response body before transformations: {"errorMessage": "[ONNXRuntimeError] : 7 : INVALID_PROTOBUF : Load model from /mnt/lambda/Real_ESRGAN_x2.onnx failed:Protobuf parsing failed.", "errorType": "InvalidProtobuf", "stackTrace": ["  File \"/var/lang/lib/python3.8/imp.py\", line 234, in load_module\n    return load_source(name, filename, file)\n", "  File \"/var/lang/lib/python3.8/imp.py\", line 171, in load_source\n    module = _load(spec)\n", "  File \"<frozen importlib._bootstrap>\", line 702, in _load\n", "  File \"<frozen importlib._bootstrap>\", line 671, in _load_unlocked\n", "  File \"<frozen importlib._bootstrap_external>\", line 843, in exec_module\n", "  File \"<frozen importlib._bootstrap>\", line 219, in _call_with_frames_removed\n", "  File \"/var/task/super_resolution_app.py\", line 21, in <module>\n    ort_session_x2 = onnxruntime.InferenceSession(access_point + '/Real_ESRGAN_x2.onnx')\n", "  File \"/opt/onnxruntime/capi/onnxruntime_inference_collection.py\", line 283, in __init__\n     [TRUNCATED]
Tue Nov 16 02:15:30 UTC 2021 : Lambda execution failed with status 200 due to customer function error: [ONNXRuntimeError] : 7 : INVALID_PROTOBUF : Load model from /mnt/lambda/Real_ESRGAN_x2.onnx failed:Protobuf parsing failed.. Lambda request id: 9f578431-af11-4687-9cfb-dc93c457120b
Tue Nov 16 02:15:30 UTC 2021 : Method completed with status: 502

just build from the template and tested in apigateway. not working. please advise

the api is created without auth even deploying with auth type IAM_AUTH

Deployed super-resolution API with inf1 instance and IAM_AUTH auth type. But the deployed API does not have authentication at all!