Build an end-to-end RAG solution with knowledge bases for Amazon Bedrock and AWS CDK

Retrieval Augmented Generation (RAG) is a cutting-edge approach to building question-answering systems that combines the strengths of retrieval and generative language models. RAG models retrieve relevant information from a large corpus of text and then use a generative language model to synthesize an answer based on the retrieved information.

Developing and deploying an end-to-end RAG solution is complex and involves multiple components, including a knowledge base, a query system, and a generative language model. Building and deploying these components can be complex and error-prone, especially when processing large amounts of data and models.

This post shows how the deployment of an end-to-end RAG solution can be seamlessly automated using Knowledge Bases for Amazon Bedrock and the AWS Cloud Development Kit (AWS CDK), enabling organizations to quickly set up a powerful question and answer system.

Solution overview

The solution provides an automated end-to-end deployment of a RAG workflow using Knowledge Bases for Amazon Bedrock. Using the AWS CDK, the solution sets up the required resources, including an AWS Identity and Access Management (IAM) role, an Amazon OpenSearch Serverless collection and index, and a knowledge base with its associated data source.

The RAG workflow lets you take your document data stored in an Amazon Simple Storage Service (Amazon S3) bucket and integrate it with the powerful natural language processing (NLP) capabilities of the Foundation Models (FMs) provided by Amazon Bedrock. The solution simplifies the setup process by allowing you to programmatically modify the infrastructure, deploy the model, and start querying your data using the selected FM.

Requirements

To implement the solution provided in this post, you should have:

• An active AWS account and familiarity with FMs, Amazon Bedrock, and Amazon OpenSearch Service.
• Model access enabled for the required models you want to experiment with.
• The AWS CDK is already set up. For installation instructions, see the AWS CDK Workshop.
• An S3 bucket set up with your documents in a supported format (.txt, .md, .html, .doc/docx, .csv, .xls/.xlsx, .pdf).
• The Amazon Titan Embeddings V2 model is enabled in Amazon Bedrock. You can activate it on the Model access Amazon Bedrock console page. If the Amazon Titan Embeddings V2 model is enabled, the access status is shown as Access grantedas shown in the following screenshot.

Setting up the solution

Once the required steps are completed, you can set up the solution:

1. Clone the GitHub repository containing the solution files:

   git clone https://github.com/aws-samples/amazon-bedrock-samples.git
   

2. Navigate to the solution directory:

   cd knowledge-bases/ features-examples/04-infrastructure/e2e_rag_using_bedrock_kb_cdk
   

3. Create and activate the virtual environment:

   $ python3 -m venv .venv
   $ source .venv/bin/activate

Activation of the virtual environment varies depending on the operating system. For information on activation in other environments, see the AWS CDK workshop.

4. After the virtual environment is activated, you can install the required dependencies:

   $ pip install -r requirements.txt

You can now prepare the code zip file and synthesize the AWS CloudFormation template for this code.

5. In your terminal, export your AWS credentials for a role or user to ACCOUNT_IDThe role must have all the required permissions for CDK deployment:
   export AWS_REGION=”” # Same region as ACCOUNT_REGION above
   export AWS_ACCESS_KEY_ID=”” # Set to the access key of your role/user
   export AWS_SECRET_ACCESS_KEY=”” # Set to your role/user’s secret key
6. Create the dependency:
7. If you are deploying the AWS CDK for the first time, run the following command:
8. To synthesize the CloudFormation template, run the following command:
9. Because this deployment contains multiple stacks, you must deploy them in a specific order. Deploy the stacks in the following order:

   $ cdk deploy KbRoleStack
   $ cdk deploy OpenSearchServerlessInfraStack
   $ cdk deploy KbInfraStack

10. Once the deployment is complete, you can see these deployed stacks by visiting the AWS CloudFormation console as shown below. You can also note the knowledge base details (i.e. Name, ID) under the Resources tab.

Testing the solution

Now that you have deployed the solution with AWS CDK, you can test it using the following steps:

1. In the Amazon Bedrock console, select Knowledge bases on the navigation page.
2. Select the knowledge base you created.
3. Choose Synchronize to start the data collection job.
   
4. After the data ingestion job completes, select the FM you want to retrieve and generate. (This FM must be granted model access in Amazon Bedrock before use.)
   
5. Start querying your data using natural language queries.

That’s it! You can now interact with your documents using the Amazon Bedrock RAG workflow.

Clean up

To avoid future AWS account charges, follow these steps:

1. Delete all files in the mounted S3 bucket.
2. To delete the CloudFormation stack deployed with AWS CDK, run the following command in the terminal:

Diploma

In this post, we showed how to quickly deploy an end-to-end RAG solution using knowledge bases for Amazon Bedrock and AWS CDK.

This solution simplifies the process of setting up the required infrastructure, including an IAM role, an OpenSearch Serverless collection and index, and a knowledge base with an associated data source. The automated deployment process enabled by the AWS CDK minimizes the complexity and potential errors associated with manually configuring and deploying the various components required for a RAG solution. By leveraging the power of FMs provided by Amazon Bedrock, you can seamlessly integrate your document data with advanced NLP capabilities, enabling you to efficiently retrieve relevant information and generate high-quality responses to natural language queries.

This solution not only simplifies the deployment process, but also provides a scalable and efficient way to leverage RAG’s Q&A capabilities. The ability to programmatically modify the infrastructure allows you to quickly adapt the solution to your organization’s specific needs, making it a valuable tool for a wide range of applications that require accurate and contextual information retrieval and generation.

About the authors

Sandeep Singh is a Senior Generative AI Data Scientist at Amazon Web Services, helping companies innovate with generative AI. He specializes in generative AI, machine learning, and systems design. He has successfully delivered cutting-edge AI/ML-powered solutions to solve complex business problems for various industries, optimizing efficiency and scalability.

Manoj Krishna Mohan is Machine Learning Engineering at Amazon. He specializes in developing AI/ML solutions using Amazon SageMaker. His passion is developing turnkey solutions for customers. Manoj holds a Master’s degree in Computer Science with a concentration in Data Science from the University of North Carolina, Charlotte.

Mani Khanuja is a Tech Lead – Generative AI Specialists, author of the book Applied Machine Learning and High-Performance Computing on AWS, and a member of the Board of Directors of the Women in Manufacturing Education Foundation. She leads machine learning projects in various areas such as computer vision, natural language processing, and generative AI. She speaks at internal and external conferences such as AWS re:Invent, Women in Manufacturing West, YouTube webinars, and GHC 23. In her free time, she enjoys going for long runs along the beach.