Project Overview
This project is an Autonomous Data Scientist Agent that replicates the workflow of a real-world data scientist using AI. It allows users to upload datasets (CSV) or documents (PDF) and interact through natural language queries. The system intelligently understands user intent and performs tasks such as data analysis, document retrieval, and fetching real-time information from the web, all within a unified interface.
Unlike traditional chatbots, this system leverages Agentic RAG (Retrieval-Augmented Generation) to dynamically combine multiple sources of knowledge, including user data, stored documents, and live internet data. It can generate Python code for data analysis, execute it, handle errors automatically, and return meaningful insights along with visualizations. This makes it a powerful tool for both technical and non-technical users.
Built with a modular and scalable architecture, the platform supports multi-user access with authentication, ensuring that each user has a personalized and isolated experience. With integrated memory, the agent maintains conversation history and context, enabling intelligent multi-turn interactions. This project demonstrates the convergence of AI agents, data science automation, and real-time intelligence in a production-ready system.
Key Features
- Autonomous AI Agent with decision-making capability
- Agentic RAG for dynamic knowledge retrieval
- CSV data analysis using Python (pandas, visualization)
- PDF document querying using vector search (FAISS)
- Real-time web search for latest data science insights
- Automatic Python code generation
- Multi-user authentication with Supabase (private chat memory)
Technical Approach
The system is built using a modular architecture with a FastAPI backend acting as the central controller. The frontend provides a user-friendly interface for file uploads and chat interaction, while the backend manages API requests, authentication, and communication with the AI model. Supabase is used for user authentication and storing chat history, enabling secure and personalized user sessions.
At the core lies the Agent Controller, which interprets user queries and dynamically decides which tool or pipeline to activate. Depending on the query, it can trigger the Python execution engine for data analysis, the RAG pipeline for document retrieval, or the web search module for real-time information. The FAISS vector database is used to store and retrieve document embeddings efficiently, ensuring fast and relevant context retrieval.
The system integrates a Gemini LLM API for natural language understanding, reasoning, and code generation. When required, the model generates Python code that is executed in a controlled environment to perform data analysis and visualization. An iterative error-handling loop ensures robustness by detecting execution failures, refining the code, and retrying automatically. This combination of LLM reasoning, tool usage, and feedback loops enables the system to function as a truly autonomous data science agent.