This file walks you through how to set up the infrastructure and applications, and run the code to create a multi-modal agents. The blog post provides a detailed discussion of this solution. This Solution uses Full-Access policies for certain services involved in the architecture and may not meet the security standards of certain business units, however, this solution should be used to show the art-of-possible and access policy should be scoped down for any other use.
This solution uses five Lambda functions, which are serverless, event-driven compute services that runs applications. The Python code for the applications are packaged as zip files, stored in lambda_zip_files in this repo. You need to add them to an S3 bucket in your account in order to set up the Lambda functions.
First, make an S3 bucket. Go to S3 page in AWS, click "Create bucket". Then enter a bucket name, which should be universally unique. Take a note of the name, because we will need it in another section. Leave the rest as default, and click "Create bucket" at the bottom of the page.
Once the bucket is created, click the bucket name, and create a folder called code. To create a folder, click "Create folder", and then enter the folder name "code", then click "Submit".
Upload the four zip files in folder lambda_zip_files to the S3 bucket.
This Soluition requires Anthropic Claude2 model in Bedrock. Make sure you have access to the model in Bedrock.
Navigate to the Bedrock console and click on Model Access to the left tab. Select the Manage model access button to the top-right.
Submit the Use case details and select the checkboxes beside the Anthropic models. Then select save changes button.
AWS CloudFormation allows you to create infrasturecture as code.
First, download the CloudFormation template Consolidated-cfn.yaml.
Then upload it in CloudFormation to create a stack. This stack sets up the necessary infrastructure, such as IAM roles and Lambda functions. Go to CloudFormation console in AWS, click Stacks -> Create stack -> With new resources (standard).
Upload cloudformation.yaml, and click "Next".
On the "Specify stack details" page,
Give the Stack a name. Take a note of this name as we will need it when running the app. Change the "SourceCodeBucket" to the name of the S3 bucket you created above.
Leave the rest as default. Check the acknowledgement box on the last page and submit it.
It takes a few minutes to create. Once it is created, you can check the generated resources by clicking on Stacks -> Stack Name. Then click "Resources". The example below shows that the AudioTranscriptsSourceBucketResource is an S3 bucket with the bucket name "test-cf-stack-audiotranscriptssourcebucketresourc-1kg41ts9dy7hk".
The CloudFormation stack creates a SageMaker Notebook instance that we can use to run the .ipynb file.
Go to SageMaker page, and click Notebook -> Notebook instances. You will see a Notebook instance named "MultiModalSagemakerInstance". Click "Open jupyter" next to it.
On Jupyter Notebook, click New -> Terminal. This opens a command-line interface.
Copy and paste the following command lines to pull the code from Github.
git init
git clone [github_link]
multimodal-demo.ipynb notebook to setup the stock database and all other tools used by this agent.
The streamlit app for this prioject is located in app_complete.py. It uses dependencies located in the utility folder.
To run this Streamlit App on Sagemaker Studio follow the steps in the link below:
- Follow the steps outlined above for the Prerequisite and Create infrastructure using CloudFormation section to deploy/create the relevant services.
- Set Up SageMaker Studio.
- Use the sagemaker execution role
SageMakerRoledeployed by the cloudformation template above for your SageMaker Studio Domain default execution role.
- Use the sagemaker execution role
- Launch SageMaker Studio
- Clone this git repo into studio
⚠️ Change the values for the variables REGION and STACK_NAME to the region you are working in and name of the deployed cloudformation stack respectively inapp_complete.py- Open a system terminal by clicking on Amazon SageMaker Studio and then System Terminal as shown in the diagram below
-
- Navigate into the cloned repository directory using the
cdcommand and run the commandpip install -r requirements.txtto install the needed python libraries - Run command
python3 -m streamlit run app_complete.pyto start the Streamlit server. Do not use the links generated by the command as they won't work in studio. - To enter the Streamlit app, open and run the StreamlitLink.ipynb notebook. This will generate the appropiate link to enter your Streamlit app from SageMaker studio. Click on the link to enter your Streamlit app. Happy querying :)
- ⚠ Note: If you rerun the Streamlit server it may use a different port. Take not of the port used (port number is the last 4 digit number after the last :) and modify the
portvariable in theStreamlitLink.ipynbnotebook to get the correct link.
To run this Streamlit App on AWS EC2 (I tested this on the Ubuntu Image)
- Follow the steps outlined above for the Prerequisiteand Create infrastructure using CloudFormation section to deploy/create the relevant services.
- Create a new ec2 instance
- Use the sagemaker execution role
SageMakerRoledeployed by the cloudformation template above for your ec2 instance profile IAM role.
- Use the sagemaker execution role
- Expose TCP port range 8500-9000 on Inbound connections of the attached Security group to the ec2 instance. TCP port 8501 is needed for Streamlit to work. See image below
-
- Connect to your ec2 instance
- Run the appropiate commands to update the ec2 instance (
sudo apt updateandsudo apt upgrade-for Ubuntu) - Clone this git repo
- Change the values for the variables
REGIONandSTACK_NAMEto the region you are working in and name of the deployed cloudformation stack respectively inapp_complete.py. Make use of thenanocommand to make this changes. e.g.nano app_complete.py - Install python3 and pip if not already installed
- Install the dependencies in the requirements.txt file by running the command
sudo pip install -r requirements.txt - Run command
python3 -m streamlit run app_complete.py - Copy the external link and paste in a new browser tab
See CONTRIBUTING for more information.
This library is licensed under the MIT-0 License. See the LICENSE file.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent