Overview
Automated website scraping and AI-powered data structuring for a chatbot knowledge base.
Project Details
Engineered a multi-stage Python data pipeline (JSON → CSV → Markdown) to extract, clean, and structure unstructured web content into AI-ready formats.
Developed a content analysis workflow using Google Gemini API to generate structured metadata, including content classification and relevance scoring.
Deployed an automated scraping architecture with Crawl4AI to systematically build and update knowledge repositories.