pixegami
pixegami
  • 68
  • 1 963 373
Deploy your RAG/AI App to AWS Cloud • Step-by-Step Tutorial
Learn how to deploy your Python RAG/AI project to the cloud in this advanced tutorial. We will turn the app into a public API, and host it on AWS Lambda (which makes it scalable and high-performance).
🔗 Code: github.com/pixegami/deploy-rag-to-aws
👉 Related Videos (in case you need it)
RAG Project Basic Tutorial: ua-cam.com/video/tcqEUSNCn8I/v-deo.html
RAG Project Advanced Tutorial: ua-cam.com/video/2TJxpyO3ei4/v-deo.html
FastAPI Tutorial: ua-cam.com/video/iWS9ogMPOI0/v-deo.html
Get Started with AWS: ua-cam.com/video/LFCTFzcMQuA/v-deo.html
AWS Bedrock (AI): ua-cam.com/video/kwkaBrK_-Bs/v-deo.html
Using Docker on Lambda: ua-cam.com/video/wbsbXfkv47A/v-deo.html
📚 Chapters
00:00 - Introduction
04:12 - RAG Recap
11:25 - Project Architecture
13:57 - Adding FastAPI
18:44 - Building a Docker Image
20:54 - Deployment Hacks
25:53 - Local Testing With Docker
29:17 - Build AWS Infrastructure with CDK
42:12 - Creating an Async API
54:11 - Wrapping Up
Переглядів: 1 356

Відео

Python RAG Tutorial (with Local LLMs): AI For Your PDFs
Переглядів 107 тис.Місяць тому
Learn how to build a RAG (Retrieval Augmented Generation) app in Python that can let you query/chat with your PDFs using generative AI. This project contains some more advanced topics, like how to run RAG apps locally (with Ollama), how to update a vector DB with new items, how to use RAG with PDFs (or any other files), and how to test the quality of AI generated responses. 👉 Links 🔗 GitHub: gi...
Ollama: Run LLMs Locally On Your Computer (Fast and Easy)
Переглядів 9 тис.2 місяці тому
With Ollama, you can run local, open-source LLMs on your own computer easily and for free. This tutorial walks through how to install and use Ollama, how to access it via a local REST API, and how to use it in a Python app (using a client library like Langchain). 👉 Links 🔗 Ollama GitHub: github.com/ollama 🔗 LLM Library: ollama.com/library 🔗 RAG Langchain Python Project: ua-cam.com/video/tcqEUSN...
Using Python and AI to Automate My Video Editing Process
Переглядів 2,7 тис.2 місяці тому
This is a behind-the-scenes breakdown of my current process for making videos (as of 2024). In this video I'll show all the steps I take as I make these videos, the equipment I use, and some of the custom AI and automation tooling I've created to help make the process go faster (originally it was 1-2 months, now it's down to about 7 days). I use a lot of automation (with Python and AI tools) to...
Python Langchain Tutorial: Use 3 Different LLMs in 10 Mins
Переглядів 4,9 тис.5 місяців тому
Learn how to easily switch between LLMs in Langchain for your Python applications: OpenAI's GPT-4, Amazon Bedrock (Claude V2), and Google Gemini Pro. Or if none of these suit your needs, you can also implement your own interface. 📚 Chapters 00:00 Introduction 01:43 Getting Started 03:18 Creating the Base App 05:33 How To Use OpenAI GPT-4 07:42 How To Use Claude/Llama2 (via AWS) 09:33 How To Use...
Top 5 High Paying Tech Skills to Learn in 2024
Переглядів 12 тис.5 місяців тому
What are the top 5 tech skills to learn in 2024 to get ahead? In this video, I'll share what I think are the top 5 skills to invest time into this year if you want to get a high-paying role in tech, or to advance your career. This is based on various data and trends I've observed in 2023, and some of my own gut feeling. Salary Data: www.levels.fyi/ State of AI: www.mckinsey.com/capabilities/qua...
Create a Custom AI Assistant + API in 10 Mins
Переглядів 67 тис.5 місяців тому
Learn how to build your own AI assistant using OpenAI's Assistants API and how to access it via Python. 👉 Links 🔗 Code: github.com/pixegami/openai-assistants-api-demo 🔗 Documentation: platform.openai.com/docs/assistants/overview 🔗 OpenAI Assistants: platform.openai.com/assistants 🔗 Data (PDF): www.eiu.com/n/campaigns/global-liveability-index-2023/ 📚 Chapters 00:00 OpenAI Assistants API 00:58 Cr...
Build a Custom AI RPG Game with OpenAI GPTs
Переглядів 3,4 тис.5 місяців тому
Learn how to build custom OpenAI GPTs by making an interactive text-based adventure game (in the style of Dungeons and Dragons). 🔗 ChatGPT Custom GPT: chat.openai.com/gpts/discovery 🔗 OpenAI Assistants API: platform.openai.com/assistants 🔗 DnD "Wealthy Merchant" Campaign: www.dmsguild.com/product/194450/An-Introduction-to-DD The-Wealthy-Merchant 📚 Chapters 00:00 Introduction 01:06 Project Demo ...
Python Dataclasses: Here's 7 Ways It Will Improve Your Code
Переглядів 7 тис.6 місяців тому
Dataclasses in Python simplify the creation of data structures with minimal code, and gives you a lot of useful utility right out of the box. 🔗 docs.python.org/3/library/dataclasses.html 📚 Chapters 00:00 Introduction 00:25 What is a Dataclass? 01:06 Create Data Structures with Less Code 02:34 Descriptive REPR 03:30 Built-in Equality Check 04:11 Fields and Default Values 06:19 Frozen Objects 07:...
Streamlit: The Fastest Way To Build Python Apps?
Переглядів 67 тис.6 місяців тому
Learn how to build a Python Streamlit app in just 12 minutes! Streamlit turns data scripts into shareable web apps in minutes. All in pure Python. No front‑end experience required. 👉 Links 🔗 Code: github.com/pixegami/streamlit-demo-app 🔗 Streamlit: docs.streamlit.io/ 📚 Chapters 00:00 Introduction 00:41 What is Streamlit? 01:49 Using Input Elements 03:22 Working with Data 04:47 Multipage Apps 06...
RAG + Langchain Python Project: Easy AI/Chat For Your Docs
Переглядів 120 тис.6 місяців тому
Learn how to build a "retrieval augmented generation" (RAG) app with Langchain and OpenAI in Python. You can use this to create chat-bots for your documents, books or files. You can also use it to build rich, interactive AI applications that use your data as a source. 👉 Links 🔗 Code: github.com/pixegami/langchain-rag-tutorial 📄 (Sample Data) AWS Docs: github.com/awsdocs/aws-lambda-developer-gui...
Langchain: The BEST Library For Building AI Apps In Python?
Переглядів 3,8 тис.7 місяців тому
Learn to build LLM applications using Langchain (an AI toolkit for Python and JS). 👉 Links 🔗 Code: github.com/pixegami/basic-langchain-examples 🔗 Langchain: www.langchain.com/ 🔗 OpenAI Platform: platform.openai.com 📚 Chapters 00:00 Introduction to Langchain 00:52 Setting Up Langchain and OpenAI 02:36 Generate Predictions 03:40 Getting Structured Output 05:52 Sequential Chains 07:03 Building an ...
How to Publish a Python Package to PyPI (pip)
Переглядів 12 тис.7 місяців тому
Learn how to publish your own custom Python package to PyPI. This tutorial covers setting up the project, configuring the setup.py file, building and testing the package, adding CLI functionality, publishing to PyPI, and installing using pip. Watch this video if you want to share your code and make it easily installable for others. 📚 Chapters 00:00 Introduction 00:34 Project Setup 02:06 Configu...
Amazon Bedrock Tutorial: Generative AI on AWS
Переглядів 13 тис.8 місяців тому
Learn how to use Amazon Bedrock for generative AI (text and images) on AWS. This tutorial will guide you through the setup process, showing you how to generate text and images via code. You'll learn how to generate text with the Anthropic Claude model, and images with the Stable Diffusion model. All you need is an AWS account and the AWS CLI installed and configured. 🔗 Code: github.com/pixegami...
Python Requests Tutorial: HTTP Requests and Web Scraping
Переглядів 2,9 тис.8 місяців тому
Learn how to install and use "requests", one of Python's most popular module by downloads and adoption. This step-by-step guide covers installation, making HTTP GET/POST requests, handling errors and timeouts, and scraping data from websites. It is important for Python developers who want to integrate their applications with external services and retrieve data from websites. 👉 Links 🔗 Requests:...
How to Get Started with AWS • Crash Course
Переглядів 2,9 тис.8 місяців тому
How to Get Started with AWS • Crash Course
Pydantic Tutorial • Solving Python's Biggest Problem
Переглядів 241 тис.8 місяців тому
Pydantic Tutorial • Solving Python's Biggest Problem
Python FastAPI Tutorial: Build a REST API in 15 Minutes
Переглядів 41 тис.9 місяців тому
Python FastAPI Tutorial: Build a REST API in 15 Minutes
Host a Python Discord Bot on AWS Lambda (Free and Easy)
Переглядів 12 тис.9 місяців тому
Host a Python Discord Bot on AWS Lambda (Free and Easy)
My Custom MacOS Terminal Setup and Theme ✨
Переглядів 6 тис.9 місяців тому
My Custom MacOS Terminal Setup and Theme ✨
How to Run a Python Docker Image on AWS Lambda
Переглядів 25 тис.9 місяців тому
How to Run a Python Docker Image on AWS Lambda
How To Use GitHub Actions • Automate Your AWS Deployments
Переглядів 4,4 тис.10 місяців тому
How To Use GitHub Actions • Automate Your AWS Deployments
Stripe & Firebase Tutorial • Add Payments To Your NextJS App
Переглядів 17 тис.10 місяців тому
Stripe & Firebase Tutorial • Add Payments To Your NextJS App
Learn Python • #12 Final Project • Build an Expense Tracking App!
Переглядів 35 тис.Рік тому
Learn Python • #12 Final Project • Build an Expense Tracking App!
Learn Python • #11 Classes • Create and Use Classes in Python
Переглядів 1,9 тис.Рік тому
Learn Python • #11 Classes • Create and Use Classes in Python
Learn Python • #10 User Input • 4 Ways To Get Input From Your User
Переглядів 3,6 тис.Рік тому
Learn Python • #10 User Input • 4 Ways To Get Input From Your User
Learn Python • #9 Functions • Python's Most Important Concept?
Переглядів 1,1 тис.Рік тому
Learn Python • #9 Functions • Python's Most Important Concept?
Learn Python • #8 Tuples and Sets • More Ways To Store Data!
Переглядів 1 тис.Рік тому
Learn Python • #8 Tuples and Sets • More Ways To Store Data!
Learn Python • #7 Dictionaries • The Most Useful Data Structure?
Переглядів 1,3 тис.Рік тому
Learn Python • #7 Dictionaries • The Most Useful Data Structure?
Learn Python • #6 Loops • How to Repeat Code Execution
Переглядів 7 тис.Рік тому
Learn Python • #6 Loops • How to Repeat Code Execution

КОМЕНТАРІ

  • @aalamansari8643
    @aalamansari8643 День тому

    can we use this to convert the data from pdfs to proper JSON format, if yes can u please tell the process and requirements for it. Would really appreciate it.

  • @anirbansom6682
    @anirbansom6682 День тому

    How to create a CLI Command which will accept arguments ? Like, pixegami-hello --file_name /file/path/

  • @GregBreak
    @GregBreak День тому

    How can I add multi language support? So I can type in spanish and get response in the same language

  • @alphasauroxviix4235
    @alphasauroxviix4235 День тому

    Thank you for putting so much effort into this video, the quality is just magnificent. Your channel is super underrated, I hope you get the recognition that you deserve soon.

  • @ManuelJimenez1
    @ManuelJimenez1 День тому

    Thanks for the whole tutorial, I would suggest adding speed at the queries of the vector database with PostgreSQL with pgvector, pg_embeddings plugin.

  • @pablomunsey
    @pablomunsey День тому

    Very well explained and done; detail information is provided, and the pace is perfect (considering I can pause and rewind). High quality. I've seen some of your other videos, and your talented at teaching!

  • @tomasemilio
    @tomasemilio День тому

    Your videos are awesome. I have also deployed my fastapi rag app, but i used AppRunner. it simplifes a lot of the stuff. I can use backgroundtasks from fastapi.

  • @subhodeepkayal4974
    @subhodeepkayal4974 День тому

    How to change the color??

  • @maxflokinho
    @maxflokinho День тому

    I would like it not only to be able to read PDFs but also if the final information was 'weak' or missing information, it would do an internet search on the topic provided in the 'query' and complete the final answer with the collected data. Do you think this is feasible to do? I thought about using agents for this, like crewai. I looked but couldn't find any tutorial that used both methods.

  • @PavlosKoproutsidis
    @PavlosKoproutsidis 2 дні тому

    Hi there, What theme are you using?

  • @NIHITHRAJ
    @NIHITHRAJ 2 дні тому

    Awesome.. can you explain if the python code asks for excel input from the user, what modifications should be done?

  • @nagireddygajjela5430
    @nagireddygajjela5430 2 дні тому

    Wonderful presentation. Keep doing the same. Great job

  • @wacioc
    @wacioc 2 дні тому

    Considering the following example: Somewhere in chunk #3 there's the text: "Mark buys a tomato" Somewhere in chunk #261 there's the text: "Mark buys a grapefruit" Prompt: "What fruits did Mark buy?" I expect the answer to be something like: "Mark has bought a tomato and a grapefruit" Which chunk will it return as source?

  • @BellyLaughQuotes
    @BellyLaughQuotes 2 дні тому

    Is this approach work only in English ? What about other language ?

  • @roshan9176610779
    @roshan9176610779 2 дні тому

    Love your content!

  • @roshan9176610779
    @roshan9176610779 2 дні тому

    Love your content!

  • @roshan9176610779
    @roshan9176610779 2 дні тому

    Love your content!

  • @MatsAlexanderNissen-Lie
    @MatsAlexanderNissen-Lie 2 дні тому

    Hey, what is the best free model for embedding out there? Cool video, great content!

  • @techieturnedequipmenttrader
    @techieturnedequipmenttrader 3 дні тому

    Why cant i see model 4 and beyond just as yours in the video? I Just see 3.5 and its variations!

  • @immanuelpeter1579
    @immanuelpeter1579 3 дні тому

    What if we want to use env variables? Where would we place them and how can we retrieve them in the py file?

  • @bryansailer5927
    @bryansailer5927 3 дні тому

    I have run into a problem with the summarize_expenses function. When I run the program I keep getting the following error. Summarize expense Traceback (most recent call last): File "/run/media/bryan/589cd83a-0ff7-4a0b-9175-49a22d7b6ff3/Python/bin/expense_tracker_1.py", line 68, in <module> main() File "/run/media/bryan/589cd83a-0ff7-4a0b-9175-49a22d7b6ff3/Python/bin/expense_tracker_1.py", line 13, in main summarize_expenses(expense_file_path) File "/run/media/bryan/589cd83a-0ff7-4a0b-9175-49a22d7b6ff3/Python/bin/expense_tracker_1.py", line 56, in summarize_expenses expense_name, expense_category, expense_amount = line.strip().split(",") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ValueError: not enough values to unpack (expected 3, got 1) I even downloaded your file from github and I get a similar error. I am not sure how to correct this problem.

  • @RodoGalan
    @RodoGalan 3 дні тому

    Awesome video bro! Why not use AWS Kendra for the RAG block? You woulda save at least half of the video and Infra issues you faced. Still good for learning, nice job!

  • @shahnaz9026
    @shahnaz9026 3 дні тому

    you didn't use azure open ai llm because most of the companies using closed openai like azure so.. We can learn azure open ai services so create video with azure open ai llm with multimodal support rag application ( input multiple pdfs with images and tables and text) and integrate with streamlit.. post the vdo by using azure open api key Azure open ai embedding and azurechatopenai for multiple pdfs rag application.. Using azure open api end point there are no videos on UA-cam so it would be helpful.. 😊

  • @nachoeigu
    @nachoeigu 3 дні тому

    Great video. It is very complete!!! I only have one question: Why didnt you use a cloud hosting for the vector database? (Like Pinecone, for example). I mean, in Production enviroment is more efficient, isnt it? Thank you for your content! Keep it going :)

  • @coreybarron907
    @coreybarron907 3 дні тому

    Did anyone find out how to get the langchain packages to work with pydantic?? I can't seem to do that

  • @akuratti
    @akuratti 3 дні тому

    This was great! Thank you.

  • @shahnaz9026
    @shahnaz9026 3 дні тому

    Great video why you didn't use azure open ai llm because most of the companies using closed openai like azure so.. We can learn azure open ai services so create video with azure open ai llm with multimodal support rag application ( input multiple pdfs with images and tables and text) and integrate with streamlit.. post the vdo by using azure open api key Azure open ai embedding and azurechatopenai for multiple pdfs rag application.. Using azure open api end point there are no videos on UA-cam so it would be helpful.. 😊

  • @mauriciojcYT
    @mauriciojcYT 3 дні тому

    Great explanation of the capabilities of this library, thanks for the info. If I develop in Github, how can I make private or hidden the data files (working with local txts), or password protected?

  • @samho9831
    @samho9831 3 дні тому

    It is really useful. Thanks a lot.

  • @baktybekbaiserkeev7611
    @baktybekbaiserkeev7611 3 дні тому

    awesome, more videos about AI, please🙂

  • @Mykyta-Korniienko-CS
    @Mykyta-Korniienko-CS 3 дні тому

    Deploying the model on the cloud would definitely be interesting! thank you for the video :D

  • @BFlago
    @BFlago 4 дні тому

    Hi, thank you very much for the video, very helpful. I am getting this error ” 2024-06-12 11:53:56,194 - openai - INFO - error_code=None error_message=‘Unrecognized request argument supplied: assistant_id’ error_param=None error_type=invalid_request_error message=‘OpenAI API error received’ stream_error=False 2024-06-12 11:53:56,194 - __main__ - ERROR - Error calling OpenAI API: Unrecognized request argument supplied: assistant_id" What should I do, GPT doesn't give a correct answer to this

  • @kam3580
    @kam3580 4 дні тому

    Thanks, great video! To avoid duplicate documents and to look for updated documents I use a document hash to check.

    • @pixegami
      @pixegami 3 дні тому

      Yup! Great idea :)

  • @개발동생
    @개발동생 4 дні тому

    Awsome

  • @python-for-everyone
    @python-for-everyone 4 дні тому

    What a great overview and comparison. Thank you for the video. By the way, the fact that variables in Python do not need to be annotated is not what makes it dynamically typed (0:11). It is what makes Python implicitly typed. As you say later (1:34) dynamic type checking more has to do with when types are checked.

    • @pixegami
      @pixegami 3 дні тому

      Thank you - that is a more useful and technical explanation of what I was trying to express :)

  • @techaisolution
    @techaisolution 4 дні тому

    Hi, this setup spike my billing very high, The setup was to build lambda function to read the latest file from the s3 dir and make transformation then finally to s3 target dir, So this all setup with the python script has to run once the s3 notification to lambda function that an file just came to s3. But it went into a loop and made the s3 and lambda billing spike Let me knew what is the issue in my setup that i didn't noticed at first while running this python script in lambda

    • @pixegami
      @pixegami 3 дні тому

      Uh oh. It sounds like you have a recursive Lambda function. I guess after you update the S3 file, it probably triggers another event (which triggers the function) in a loop. Normally, if you want to trigger Lambda from an S3 event, I wouldn't recommend to update/write back into the same S3 - maybe just use a different bucket for the output. For the billing spike, you can create a ticket to AWS support to explain the mistake, and see if they'll be able to give you a refund on the cost (it usually works if it's the first time you are making an honest mistake).

  • @j8ge
    @j8ge 5 днів тому

    do we have a learning path , this video is good but to advanced for me.

    • @pixegami
      @pixegami 3 дні тому

      Here's my first RAG video (easier to learn): ua-cam.com/video/tcqEUSNCn8I/v-deo.html Here's a Langchain tutorial: ua-cam.com/video/R3KgD86VRzc/v-deo.html And if you are starting all the way from the beginning, here's my Python programming playlist: ua-cam.com/video/NfTrFhfqmFY/v-deo.html

  • @dmna3722
    @dmna3722 5 днів тому

    Did you ever post or share how to add authentication? Would appreciate if you gave some guidance or pointed in the direction of adding it. Is it still possible to deploy without it? @pixegami

    • @pixegami
      @pixegami 3 дні тому

      I did make two separate videos on using Firebase for Authentication: ua-cam.com/video/BQrE98bP6m4/v-deo.html And with Stripe for payments: ua-cam.com/video/xi3F2Zv91UE/v-deo.html But this might be a bit outdated now so I'll have to review and see if they are still practical options for SaaS today.

    • @dmna3722
      @dmna3722 3 дні тому

      Thank you for your response. I still keep hitting errors for missing pydantic modules in the lambda function. Is there any way around this?

  • @lesptitsoiseaux
    @lesptitsoiseaux 5 днів тому

    Hash the value of a chunk and store the hash in the metadata?

  • @DataPythonist-pc4pk
    @DataPythonist-pc4pk 6 днів тому

    hiii how can we get open ai api key?

    • @pixegami
      @pixegami 3 дні тому

      Once you create an account at openai.com/ and set up billing, you should be able to get an API key.

  • @yahiachammemi8267
    @yahiachammemi8267 6 днів тому

    Looking for more amazing content !

    • @pixegami
      @pixegami 3 дні тому

      Check out this more advanced RAG tutorial: ua-cam.com/video/2TJxpyO3ei4/v-deo.html And how to deploy it to the cloud (AWS): ua-cam.com/video/ldFONBo2CR0/v-deo.html

  • @seanwayland
    @seanwayland 6 днів тому

    I thought it was well explained . The "probabilistic" unit testing scared me. Does it work ? "maybe"

    • @pixegami
      @pixegami 3 дні тому

      I guess they aren't "true" unit tests - your actual unit tests will probably be more functional and deterministic. True I'm using a unit test framework, but this is more like a qualitative evaluation. E.g. "what % of answers does this implementation answer correctly?"

  • @TheShocobo
    @TheShocobo 6 днів тому

    which shortcut key is he using at 4:20 ?

  • @ryanperkins1525
    @ryanperkins1525 6 днів тому

    you cut and paste a little bit too fast...

    • @pixegami
      @pixegami 3 дні тому

      I've received that feedback a few times, I'll try to slow it down a bit :)

  • @AlonAvramson
    @AlonAvramson 7 днів тому

    Thank you! your videos are of a great help. I am interested in customizing and training options for the off-the-shelf open-source models such as llama3 locally, preferably using LangChain, and also, when generating a RAG model, can llama3 be used also for embedding? The idea is to develop a RAG model with Streamlit, and deploy ot on Streamlit cloud with the need to expose any keys for OpenAI. Also, will the streamlit cloud could use the llama3 locally?

  • @samho9831
    @samho9831 7 днів тому

    Thanks for sharing and I can have a grasp of the concept. If possible, it will be deeply appreciated if you can show how to do it in the cloud. Thanks.

    • @pixegami
      @pixegami 3 дні тому

      Stay tuned for my next video! ua-cam.com/video/ldFONBo2CR0/v-deo.html

  • @user-yb2tp4qf1r
    @user-yb2tp4qf1r 7 днів тому

    That was very helpful, Thanks a lot.

    • @pixegami
      @pixegami 3 дні тому

      Glad it was helpful!

  • @liqwis9598
    @liqwis9598 7 днів тому

    any idea how this can be implemented in Java / Spring boot

  • @IdPreferNot1
    @IdPreferNot1 7 днів тому

    Thank you for updating the repo and the details on the c++ install as well!!

    • @pixegami
      @pixegami 3 дні тому

      If you are using a local LLM, then it might depend on your computer specs. If you are using a cloud AI (e.g. AWS or OpenAI), it might be a connection bottleneck. Normally it doesn't take more than a few seconds to me for 10 pages, so it does sound like a performance issue.

  • @bryansailer5927
    @bryansailer5927 7 днів тому

    How are you importing emojis without importing the emoji module?

    • @pixegami
      @pixegami 3 дні тому

      Emojis are just normal UTF-8 unicode characters (like every other letter), and most terminal/editors should be able to understand and display them. Go ahead and try them! Just copy paste an emoji into the text and it should work.

    • @bryansailer5927
      @bryansailer5927 3 дні тому

      @@pixegami Thank you, after searching online I found what I was looking for. Your instruction is great.