azure-samples/papiro

Build 2022 AI Breakout Session Demo App: https://aka.ms/MSBuild2022

Papiro Document Intelligence

• Papiro Document Intelligence
  • Overview
  • Features
  • Tools & Services Used
  • Getting Started
    • Prerequisites
    • Quickstart
    • Text Generation

Overview

As part of Microsoft /BUILD 2022 the AI Platform Group showed a conference intelligence application that used a number of intelligence APIs to showcase what might be done to support conferences with AI. This is the project that was shown!

Microsoft Build free registration is now open and the sessions are listed.

Sign up for the AI session on showcasing this demo: https://aka.ms/MSBuild2022

Features

This sample conference app provides the following features:

• Quick user check-in using ID/Vaccination Card
• FAQ Multilingual Document Translation
• TLDR Policy Document Summarization
• Audio Session Feedback to Sentiment

Tools & Services Used

• Next.js CLI
• Azure Form Recognizer
  • New ID & Vaccination card pretrained models
  • Beta JavaScript SDK
• Azure Speech
  • Speech to Text
  • JavaScript SDK
• Azure Translator
  • Document Translation
  • REST API SDK
• Azure Language
  • Sentiment Analysis
  • Text Analytics JavaScript Client Library
• Text Summarization Preview
• Azure ML CLI
• Bicep is a language for declaratively deploying Azure resources. You can use Bicep instead of JSON for developing your Azure Resource Manager templates (ARM templates).

Getting Started

Prerequisites

A working operating system with Node, the Azure Command-Line Interface(az CLI), and PowerShell or Visual Studio Code and it's Powershell extention

Quickstart

There are a few steps for setting this up:

1. Clone this repo

2. Install yarn globally

   npm i -g yarn

3. Azure Infrastructure setup:

   ./provision.ps1 -name <YOUR_APP_NAME> -location westus
   

4. Azure Web App publish:

   Run the following command to build the web app package and publish it to Azure. It will prompt for login to Azure.

   ./publish.ps1 -name <YOUR_APP_NAME>
   

5. For Web App setup, please refer to /src/README

Text Generation

The text generation API is created using another sample shown at Microsoft /BUILD 2022. If you're interested in setting this up, head on over to the Managed AzureML Inference instructions.

FAQ

Getting Responsible AI error when setup the Azure infrastructure

Users who haven't created Face, Computer Vision, and Language service Azure resources will get the following error when running provision.ps1. Please go to the Azure Portal and create a Face, Computer Vision, or Language service to review and acknowledge the terms and conditions. See more details here.

This subscription cannot create TextAnalytics until you agree to Responsible AI terms for this resource. You can agree to Responsible AI terms by creating a resource through the Azure Portal then trying again. For more detail go to https://go.microsoft.com/fwlink/?linkid=2164190

papiro's Issues

Cannot trigger local run with ss

Do we miss any steps in ReadMe?

This issue is for a: (mark with an x)

- [x] bug report -> please search issues before submitting
- [ ] feature request
- [x] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

1. Clone the repo
2. In PowerShell (v5), cd to the repo root, run 'ss'

Any log messages given by the failure

ss: The term 'ss' is not recognized as a name of a cmdlet, function, script file, or executable program.
Check the spelling of the name, or if a path was included, verify that the path is correct and try again.

Expected/desired behavior

Run the app locally

Env

Windows
Node v16.13.0
azure-cli 2.34.1
PowerShell 5.

Cannot find text generation instances after running provision script

Don't find any Azure text generation service in the resource group created by provision.ps1. But it is required in .env. Do we miss creating the instance? or what could I have missed?

.env.example

This issue is for a: (mark with an x)

- [x] bug report -> please search issues before submitting
- [ ] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

1. ./provision.ps1 -name <YOUR_APP_NAME> -location westus
2. Go to Azure portal and looking for resource group (RG) created by the script <YOUR_APP_NAME>rg

Expected/desired behavior

Finding instances with "text generation" type.

These are what I have in the RG

Translation of Scanned PDF returns the original document

Please provide us with the following information:

I'm recording submitting the issue to record the issue. The issue is solved in the latest release of Translation service and it doesn't exist with PDFs that has selectable text, like the PDF provided here.

This issue is for a: (mark with an x)

- [ x] bug report -> please search issues before submitting
- [ ] feature request
- [x ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

-Run the command on src folder

yarn dev

• Navigate to the localhost:3000
• Drag a scanned PDF to translate

Any log messages given by the failure

No

Expected/desired behavior

When the translation is complete, the downloaded translation would be in expected language.

OS and Version?

Windows 7, 8 or 10. Linux (which distribution). macOS (Yosemite? El Capitan? Sierra?)
Windows 11

Versions

Mention any other details that might be useful

---

Thanks! We'll be in touch soon.

provision.ps1 failed due to storage name exceeds 24 chars

• Suggest adding description in ReadMe that <YOUR_APP_NAME> cannot exceed 13 char.

This issue is for a: (mark with an x)

- [ ] bug report -> please search issues before submitting
- [ ] feature request
- [x] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

1. Clone the repo
2. In PowerShell (v5) run, .\provision.ps1 -name build22AMLPapiro -location westus2

Any log messages given by the failure

Seeing warning and deployment failed due to storage name exceeeds 24 chars.

D:\GitHub\papiro\infra\cosmosdb.bicep(5,7) : Warning no-unused-params: Parameter "location" is declared but never used. [https://aka.ms/bicep/linter/no-unused-params]
D:\GitHub\papiro\infra\cosmosdb.bicep(9,13) : Warning no-hardcoded-location: A resource location should not use a hard-coded string or variable value. Please use a parameter value, an expression, or the string 'global'. Found: 'westus' [https://aka.ms/bicep/linter/no-hardcoded-location]

{'code': 'InvalidTemplateDeployment', 'message': "The template deployment 'main' is not valid according to the validation procedure. The tracking id is 'f9fd7422-ea25-4ed0-8388-47c5fd3317a0'. See inner errors for details."}

Inner Errors: 
{'code': 'PreflightValidationCheckFailed', 'message': 'Preflight validation failed. Please refer to the details for the specific errors.'}

Inner Errors: 
{'code': 'AccountNameInvalid', 'target': 'build22amlpapirostorage3upi', 'message': 'build22amlpapirostorage3upi is not a valid storage account name. Storage account name must be between 3 and 24 characters in length and use numbers and lower-case letters only.'}

Expected/desired behavior

No warning and deployment succeed.

OS and Version?

Windows 11

Cannot publish app

Cannot publish app with the insturctued command
./publish.ps1 -name MY_APP_NAME

This issue is for a: (mark with an x)

- [x] bug report -> please search issues before submitting
- [ ] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

In PowerShell, run the following commands

1. ./provision.ps1 -name <YOUR_APP_NAME> -location westus2
2. ./publish.ps1 -name MY_APP_NAME

Any log messages given by the failure

Expected/desired behavior

The script run without issues

OS and Version?

Windows 11

Web app seems not working

It seems to me the web app doesn't work (the CV feature throws error; the doc translation keep spinning). I suspect the ReadMe instructions are in wrong order. .env should be updated before publish the code to web app. The order of step 4 and 5 should be reversed.

This issue is for a: (mark with an x)

- [x] bug report -> please search issues before submitting
- [ ] feature request
- [x] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

1. ./provision.ps1 -name <YOUR_APP_NAME> -location westus
2. ./publish.ps1 -name <YOUR_APP_NAME>
3. Go to Azure, find the web app URL. Open the URL in browser. (Allow browser to access camera)

Screenshopts of the app

Expected/desired behavior

OS and Version?

Windows 10, Edge browser

[ vercel ] deployment failure - Type 'Component<{}, any, any>' is not assignable to type 'Element | ElementClass | null' in _app.tsx

Please provide us with the following information:

This issue is for a: (mark with an x)

- [ ] bug report -> please search issues before submitting
- [ ] feature request
- [ ] documentation issue or request
- [ x] regression (a behavior that used to work and stopped in a new release)

Related issue

Minimal steps to reproduce

Any log messages given by the failure

[ vercel ] deployment failure - Type 'Component<{}, any, any>' is not assignable to type 'Element | ElementClass | null' in _app.tsx

Expected/desired behavior

OS and Version?

Windows 7, 8 or 10. Linux (which distribution). macOS (Yosemite? El Capitan? Sierra?)

Versions

Mention any other details that might be useful

---

Thanks! We'll be in touch soon.

provision.ps1 failed on appservicedeploy (BadRequest)

Hit two errors

1. RAI error. The user is asked to sign RAI term by creating language service in Azure portal. This will happen to users who never create CV, Language service, or Face resource in Azure.
2. Quota error. Not sure what type of quota is required.

This issue is for a: (mark with an x)

- [x] bug report -> please search issues before submitting
- [ ] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

1. Clone the repo
2. In PowerShell (v5), run the command .\provision.ps1 -name build22Papiro -location westus2

Any log messages given by the failure

RAI error

D:\GitHub\papiro\infra\cosmosdb.bicep(5,7) : Warning no-unused-params: Parameter "location" is declared but never used. [https://aka.ms/bicep/linter/no-unused-params]
D:\GitHub\papiro\infra\cosmosdb.bicep(9,13) : Warning no-hardcoded-location: A resource location should not use a hard-coded string or variable value. Please use a parameter value, an expression, or the string 'global'. Found: 'westus' [https://aka.ms/bicep/linter/no-hardcoded-location]

{"status":"Failed","error":{"code":"DeploymentFailed","message":"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.","details":[{"code":"Conflict","message":"{\r\n  \"status\": \"Failed\",\r\n  \"error\": {\r\n    \"code\": \"ResourceDeploymentFailure\",\r\n    \"message\": \"The resource operation completed with terminal provisioning state 'Failed'.\",\r\n    \"details\": [\r\n      {\r\n        \"code\": \"DeploymentFailed\",\r\n        \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.\",\r\n        \"details\": [\r\n          {\r\n            \"code\": \"BadRequest\",\r\n            \"message\": \"{\\r\\n  \\\"error\\\": {\\r\\n    \\\"code\\\": \\\"ResourceKindRequireAcceptTerms\\\",\\r\\n    \\\"message\\\": \\\"This subscription cannot create TextAnalytics until you agree to Responsible AI terms for this resource. You can agree to Responsible AI terms by creating a resource through the Azure Portal then trying again. For more detail go to https://go.microsoft.com/fwlink/?linkid=2164190\\\"\\r\\n  }\\r\\n}\"\r\n          }\r\n        ]\r\n      }\r\n    ]\r\n  }\r\n}"}]}}

Quota error

{"status":"Failed","error":{"code":"DeploymentFailed","message":"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.","details":[{"code":"Conflict","message":"{\r\n  \"status\": \"Failed\",\r\n  \"error\": {\r\n    \"code\": \"ResourceDeploymentFailure\",\r\n    \"message\": \"The resource operation completed with terminal provisioning state 'Failed'.\",\r\n    \"details\": [\r\n      {\r\n        \"code\": \"DeploymentFailed\",\r\n        \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.\",\r\n        \"details\": [\r\n          {\r\n            \"code\": \"Unauthorized\",\r\n            \"message\": \"{\\r\\n  \\\"Code\\\": \\\"Unauthorized\\\",\\r\\n  \\\"Message\\\": \\\"This region has quota of 0 instances for your subscription. Try selecting different region or SKU.\\\",\\r\\n  \\\"Target\\\": null,\\r\\n  \\\"Details\\\": [\\r\\n    {\\r\\n      \\\"Message\\\": \\\"This region has quota of 0 instances for your subscription. Try selecting different region or SKU.\\\"\\r\\n    },\\r\\n    {\\r\\n      \\\"Code\\\": \\\"Unauthorized\\\"\\r\\n    },\\r\\n    {\\r\\n      \\\"ErrorEntity\\\": {\\r\\n        \\\"ExtendedCode\\\": \\\"52039\\\",\\r\\n        \\\"MessageTemplate\\\": \\\"{0}. Try selecting different region or SKU.\\\",\\r\\n        \\\"Parameters\\\": [\\r\\n          \\\"This region has quota of 0 instances for your subscription\\\"\\r\\n        ],\\r\\n        \\\"Code\\\": \\\"Unauthorized\\\",\\r\\n        \\\"Message\\\": \\\"This region has quota of 0 instances for your subscription. Try selecting different region or SKU.\\\"\\r\\n      }\\r\\n    }\\r\\n  ],\\r\\n  \\\"Innererror\\\": null\\r\\n}\"\r\n          }\r\n        ]\r\n      }\r\n    ]\r\n  }\r\n}"}]}}

Expected/desired behavior

Deployment succeed without errors.

Document translation requires S1 tier | bicep provisions F0

Please provide us with the following information:

This issue is for a: (mark with an x)

- [x] bug report -> please search issues before submitting
- [ ] feature request
- [x] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

Using standard PowerShell script

Any log messages given by the failure

n/a; however, the app hangs on document translation

Expected/desired behavior

For document translation to function when a document is dragged into the app.

OS and Version?

Windows 11

Versions

Mention any other details that might be useful

---

Thanks! We'll be in touch soon.