mirror of
https://github.com/thecyberlearn/chat-backend.git
synced 2026-08-18 17:32:53 +00:00
a0fce0b484
Features: - Django REST API backend - Multi-strategy web scraping system (Beautiful Soup, Playwright, Firecrawl) - Anti-detection features (proxy rotation, user-agent rotation) - Data export functionality (JSON, CSV, TXT) - Business and CrawledPage models - Enhanced crawling service with fallback mechanisms 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
68 lines
1.9 KiB
Markdown
68 lines
1.9 KiB
Markdown
# Quick Start Guide
|
|
|
|
## Getting Back Up and Running
|
|
|
|
### 1. Navigate to Project
|
|
```bash
|
|
cd /home/amit/projects/chat-backend
|
|
```
|
|
|
|
### 2. Activate Virtual Environment
|
|
```bash
|
|
source venv/bin/activate
|
|
```
|
|
|
|
### 3. Start Server
|
|
```bash
|
|
python manage.py runserver
|
|
```
|
|
|
|
### 4. Access Application
|
|
- Open: http://localhost:8000
|
|
- Admin: http://localhost:8000/admin/
|
|
|
|
## What We Built
|
|
|
|
### Fresh Start Results
|
|
- ✅ Removed old 718-line messy codebase
|
|
- ✅ Created clean Django project from scratch
|
|
- ✅ Built modern Firecrawl integration (~80 lines)
|
|
- ✅ Working website scraper with clean UI
|
|
- ✅ Proper environment configuration
|
|
|
|
### File Structure
|
|
```
|
|
chat-backend/
|
|
├── .env # Environment variables
|
|
├── requirements.txt # Dependencies
|
|
├── manage.py # Django management
|
|
├── chat_backend/ # Project settings
|
|
│ ├── settings.py # Clean settings with env vars
|
|
│ └── urls.py # URL routing
|
|
└── scraping/ # Main app
|
|
├── models.py # Site + ScrapedContent models
|
|
├── services.py # Clean Firecrawl service
|
|
├── views.py # Simple view functions
|
|
├── urls.py # App URLs
|
|
├── admin.py # Admin interface
|
|
└── templates/ # Clean HTML templates
|
|
```
|
|
|
|
### Key Features
|
|
1. **Add Sites**: Enter website name and URL
|
|
2. **Scrape Content**: Uses Firecrawl extract API
|
|
3. **View Results**: Company name, description, full content
|
|
4. **Status Tracking**: Real-time scraping progress
|
|
|
|
### Configuration
|
|
- Set FIRECRAWL_API_KEY in .env file
|
|
- Uses demo mode if API key not configured
|
|
- Clean environment variable handling
|
|
|
|
## Next Steps for Your Vision
|
|
Based on your original plan, ready to add:
|
|
1. Content Management (PDFs, videos, forms)
|
|
2. AI Q&A Generation
|
|
3. Export to business.json
|
|
|
|
**This is now a professional foundation you can actually build on!** |