🎯 Implement dynamic agent configuration system for ultimate scalability

BREAKTHROUGH: Configuration-driven agent management that scales to 100+ agents

## New System Architecture:
- **JSON Configuration Files**: All agents defined in version-controlled JSON
- **Dynamic Loading**: populate_agents.py reads from config files automatically
- **Zero Code Changes**: Add new agents by creating JSON files only
- **Automatic Railway Sync**: Single command ensures database consistency

## File Structure:
- agents/configs/categories/categories.json - All categories
- agents/configs/agents/*.json - Individual agent configurations
- agents/configs/README.md - Complete documentation

## Key Benefits:
 **Ultimate Scalability**: Add 1000+ agents without touching code
 **Version Control**: All agent definitions tracked in git
 **Railway Consistency**: Single command syncs local and production
 **Developer Experience**: JSON files are easier than Python commands
 **Validation**: Built-in error handling and field validation
 **Documentation**: Self-documenting with clear examples

## Migration Path:
- Extracted all 6 existing agents to JSON configurations
- Updated populate_agents.py to load dynamically from configs
- Maintained backward compatibility
- Added comprehensive documentation

## Usage:
```bash
# Add new agent: Create JSON file in agents/configs/agents/
# Deploy to Railway: python manage.py populate_agents
# Result: Agent automatically appears in marketplace
```

This solves the original issue: "if we create another will it show on railway also???"
Answer: YES - just create JSON file and run populate_agents\!

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Claude 2025-08-05 08:38:59 +05:30
parent be0eecbafd
commit bc662e6af0
9 changed files with 421 additions and 227 deletions

113
agents/configs/README.md Normal file
View File

@ -0,0 +1,113 @@
# Agent Configuration System
This directory contains JSON configuration files for dynamically creating agents and categories in the Quantum Tasks AI platform.
## Directory Structure
```
agents/configs/
├── categories/
│ └── categories.json # All agent categories
├── agents/
│ ├── ai-brand-strategist.json # Direct access agent
│ ├── cybersec-career-navigator.json # Direct access agent
│ ├── five-whys-analysis.json # Chat webhook agent
│ ├── job-posting-generator.json # Form webhook agent
│ ├── pdf-summarizer.json # File upload webhook agent
│ └── social-ads-generator.json # Form webhook agent
└── README.md # This file
```
## How It Works
1. **Categories** are defined in `categories/categories.json`
2. **Agents** are defined in individual JSON files in `agents/`
3. Run `python manage.py populate_agents` to create all agents from configs
4. **Adding new agents** is as simple as creating a new JSON file
## Adding New Agents
### Step 1: Create JSON Configuration File
Create a new file in `agents/` directory, e.g., `email-writer.json`:
```json
{
"slug": "email-writer",
"name": "Email Writer",
"short_description": "AI-powered professional email writing assistant",
"description": "Generate professional emails for any purpose with AI assistance.",
"category": "marketing",
"price": 3.0,
"agent_type": "form",
"system_type": "webhook",
"form_schema": {
"fields": [
{
"name": "email_type",
"type": "select",
"label": "Email Type",
"required": true,
"options": [
{"value": "business", "label": "Business Email"},
{"value": "marketing", "label": "Marketing Email"}
]
}
]
},
"webhook_url": "http://localhost:5678/webhook/email-writer",
"access_url_name": "",
"display_url_name": ""
}
```
### Step 2: Run Population Command
```bash
python manage.py populate_agents
```
### Step 3: Agent Appears Automatically
The agent will now appear in the marketplace with the configured settings.
## Agent Types
### Webhook Agents (N8N Integration)
- Set `system_type`: `"webhook"`
- Include detailed `form_schema` with fields
- Set `webhook_url` to N8N endpoint
- Leave `access_url_name` and `display_url_name` empty
### Direct Access Agents (External Forms)
- Set `system_type`: `"direct_access"`
- Set `form_schema`: `{"fields": []}`
- Set `webhook_url` to external form URL (JotForm, etc.)
- Set `access_url_name`: `"agents:direct_access_handler"`
- Set `display_url_name`: `"agents:direct_access_display"`
## Field Types for Webhook Agents
- `text`: Single-line text input
- `textarea`: Multi-line text input
- `select`: Dropdown with options array
- `file`: File upload with drag-and-drop
- `url`: URL input with validation
- `checkbox`: Boolean checkbox
## Benefits
**Scalable**: Add 100+ agents without code changes
**Version Controlled**: All agent definitions in git
**Consistent**: Ensures local and Railway databases match
**Simple**: Just create JSON file and run command
**Validated**: Built-in validation and error handling
## Railway Deployment
On Railway, just run:
```bash
python manage.py populate_agents
```
All agents defined in JSON files will be created automatically, ensuring Railway marketplace shows all agents consistently.

View File

@ -0,0 +1,16 @@
{
"slug": "ai-brand-strategist",
"name": "AI Brand Strategist",
"short_description": "Get AI-powered brand strategy insights and recommendations for your business",
"description": "Transform your brand with AI-driven strategic insights. Get expert guidance on brand positioning, messaging, visual identity, and competitive differentiation.",
"category": "marketing",
"price": 0.0,
"agent_type": "form",
"system_type": "direct_access",
"form_schema": {
"fields": []
},
"webhook_url": "https://agent.jotform.com/01986502acd276b48e3d5f39337046c8d9b6",
"access_url_name": "agents:direct_access_handler",
"display_url_name": "agents:direct_access_display"
}

View File

@ -0,0 +1,16 @@
{
"slug": "cybersec-career-navigator",
"name": "CyberSec Career Navigator",
"short_description": "Get personalized cybersecurity career guidance from AI expert Jessica",
"description": "Navigate your cybersecurity career path with expert AI guidance. Get personalized advice on certifications, job roles, skills development, and career progression.",
"category": "career-education",
"price": 0.0,
"agent_type": "form",
"system_type": "direct_access",
"form_schema": {
"fields": []
},
"webhook_url": "https://agent.jotform.com/019865a942ab7fa5b5b743a5fd2abe09e345",
"access_url_name": "agents:direct_access_handler",
"display_url_name": "agents:direct_access_display"
}

View File

@ -0,0 +1,14 @@
{
"slug": "five-whys-analysis",
"name": "5 Whys Analysis",
"short_description": "Interactive problem-solving using the proven 5 Whys methodology",
"description": "Systematically find root causes through guided 5 Whys methodology. Perfect for troubleshooting operational problems, understanding failures, and identifying systemic issues.",
"category": "analysis",
"price": 15.0,
"agent_type": "chat",
"system_type": "webhook",
"form_schema": null,
"webhook_url": "http://localhost:5678/webhook/5-whys-web",
"access_url_name": "",
"display_url_name": ""
}

View File

@ -0,0 +1,55 @@
{
"slug": "job-posting-generator",
"name": "Job Posting Generator",
"short_description": "Create professional job postings that attract top talent",
"description": "Generate comprehensive and attractive job postings with AI-powered content creation. Perfect for HR teams and recruiters.",
"category": "human-resources",
"price": 10.0,
"agent_type": "form",
"system_type": "webhook",
"form_schema": {
"fields": [
{
"name": "job_title",
"type": "text",
"label": "Job Title",
"required": true
},
{
"name": "company_name",
"type": "text",
"label": "Company Name",
"required": true
},
{
"name": "job_description",
"type": "textarea",
"label": "Job Description",
"required": true,
"rows": 5
},
{
"name": "seniority_level",
"type": "select",
"label": "Seniority Level",
"required": true,
"options": [
{"value": "", "label": "Select level..."},
{"value": "entry", "label": "Entry Level"},
{"value": "junior", "label": "Junior"},
{"value": "mid", "label": "Mid Level"},
{"value": "senior", "label": "Senior"}
]
},
{
"name": "location",
"type": "text",
"label": "Location",
"required": true
}
]
},
"webhook_url": "http://localhost:5678/webhook/43f84411-eaaa-488c-9b1f-856e90d0aaf6",
"access_url_name": "",
"display_url_name": ""
}

View File

@ -0,0 +1,38 @@
{
"slug": "pdf-summarizer",
"name": "PDF Summarizer",
"short_description": "Extract and summarize content from PDF documents with AI analysis",
"description": "Upload PDF documents and get comprehensive AI-powered summaries, key insights, and analysis. Perfect for processing reports and research papers.",
"category": "document-processing",
"price": 8.0,
"agent_type": "form",
"system_type": "webhook",
"form_schema": {
"fields": [
{
"name": "pdf_file",
"type": "file",
"label": "Upload PDF Document",
"required": true,
"accept": ".pdf",
"max_size": "10MB"
},
{
"name": "analysis_type",
"type": "select",
"label": "Analysis Type",
"required": true,
"default": "summary",
"options": [
{"value": "", "label": "Select analysis type..."},
{"value": "summary", "label": "Document Summary"},
{"value": "key_points", "label": "Key Points Extraction"},
{"value": "detailed_analysis", "label": "Detailed Analysis"}
]
}
]
},
"webhook_url": "http://localhost:5678/webhook/simple-pdf-processor",
"access_url_name": "",
"display_url_name": ""
}

View File

@ -0,0 +1,49 @@
{
"slug": "social-ads-generator",
"name": "Social Ads Generator",
"short_description": "Create compelling social media advertisements optimized for different platforms",
"description": "Generate engaging social media advertisements with AI-powered content generation. Optimized for Facebook, Instagram, LinkedIn, Twitter, TikTok, and YouTube.",
"category": "marketing",
"price": 6.0,
"agent_type": "form",
"system_type": "webhook",
"form_schema": {
"fields": [
{
"name": "description",
"type": "textarea",
"label": "Describe what you'd like to generate",
"placeholder": "Describe the product, service, or campaign",
"required": true,
"rows": 4
},
{
"name": "social_platform",
"type": "select",
"label": "For Social Media Platform",
"required": true,
"options": [
{"value": "", "label": "Select a platform..."},
{"value": "facebook", "label": "Facebook"},
{"value": "instagram", "label": "Instagram"},
{"value": "linkedin", "label": "LinkedIn"},
{"value": "twitter", "label": "X (Twitter)"}
]
},
{
"name": "include_emoji",
"type": "select",
"label": "Include Emoji",
"required": true,
"options": [
{"value": "", "label": "Select an option..."},
{"value": "yes", "label": "Yes"},
{"value": "no", "label": "No"}
]
}
]
},
"webhook_url": "http://localhost:5678/webhook/2dc234d8-7217-454a-83e9-81afe5b4fe2d",
"access_url_name": "",
"display_url_name": ""
}

View File

@ -0,0 +1,32 @@
[
{
"slug": "analysis",
"name": "Analysis & Problem Solving",
"description": "AI-powered analysis tools for problem-solving and decision making",
"icon": "🧠"
},
{
"slug": "career-education",
"name": "Career & Education",
"description": "Professional career guidance and educational resources",
"icon": "🎓"
},
{
"slug": "document-processing",
"name": "Document Processing",
"description": "AI-powered document analysis and processing tools",
"icon": "📄"
},
{
"slug": "human-resources",
"name": "Human Resources",
"description": "HR automation and talent management solutions",
"icon": "💼"
},
{
"slug": "marketing",
"name": "Marketing & Advertising",
"description": "AI-powered marketing tools and advertising solutions",
"icon": "📢"
}
]

View File

@ -1,52 +1,30 @@
import json
import os
from pathlib import Path
from django.core.management.base import BaseCommand from django.core.management.base import BaseCommand
from agents.models import AgentCategory, Agent from agents.models import AgentCategory, Agent
class Command(BaseCommand): class Command(BaseCommand):
help = 'Populate all agents and categories - ensures database consistency between local and production' help = 'Dynamically populate all agents and categories from JSON configuration files'
def handle(self, *args, **options): def handle(self, *args, **options):
self.stdout.write(self.style.SUCCESS('🚀 Populating all agents and categories...')) self.stdout.write(self.style.SUCCESS('🚀 Dynamically populating agents from configuration files...'))
self.stdout.write('') self.stdout.write('')
# Track creation statistics # Track creation statistics
categories_created = 0 categories_created = 0
agents_created = 0 agents_created = 0
config_base_path = Path(__file__).parent.parent.parent / 'configs'
# Define all categories # Load and create categories
categories_data = [ categories_file = config_base_path / 'categories' / 'categories.json'
{ if not categories_file.exists():
'slug': 'analysis', self.stdout.write(self.style.ERROR(f'❌ Categories file not found: {categories_file}'))
'name': 'Analysis & Problem Solving', return
'description': 'AI-powered analysis tools for problem-solving and decision making',
'icon': '🧠' with open(categories_file, 'r', encoding='utf-8') as f:
}, categories_data = json.load(f)
{
'slug': 'career-education',
'name': 'Career & Education',
'description': 'Professional career guidance and educational resources',
'icon': '🎓'
},
{
'slug': 'document-processing',
'name': 'Document Processing',
'description': 'AI-powered document analysis and processing tools',
'icon': '📄'
},
{
'slug': 'human-resources',
'name': 'Human Resources',
'description': 'HR automation and talent management solutions',
'icon': '💼'
},
{
'slug': 'marketing',
'name': 'Marketing & Advertising',
'description': 'AI-powered marketing tools and advertising solutions',
'icon': '📢'
}
]
# Create categories
categories = {} categories = {}
for category_data in categories_data: for category_data in categories_data:
category, created = AgentCategory.objects.get_or_create( category, created = AgentCategory.objects.get_or_create(
@ -66,204 +44,82 @@ class Command(BaseCommand):
self.stdout.write('') self.stdout.write('')
# Define all agents # Load and create agents from JSON files
agents_data = [ agents_dir = config_base_path / 'agents'
# Webhook Agents if not agents_dir.exists():
{ self.stdout.write(self.style.ERROR(f'❌ Agents directory not found: {agents_dir}'))
'slug': 'five-whys-analysis', return
'name': '5 Whys Analysis',
'short_description': 'Interactive problem-solving using the proven 5 Whys methodology',
'description': 'Systematically find root causes through guided 5 Whys methodology. Perfect for troubleshooting operational problems, understanding failures, and identifying systemic issues.',
'category': 'analysis',
'price': 15.0,
'agent_type': 'chat',
'form_schema': None,
'webhook_url': 'http://localhost:5678/webhook/5-whys-web',
'access_url_name': '',
'display_url_name': ''
},
{
'slug': 'social-ads-generator',
'name': 'Social Ads Generator',
'short_description': 'Create compelling social media advertisements optimized for different platforms',
'description': 'Generate engaging social media advertisements with AI-powered content generation. Optimized for Facebook, Instagram, LinkedIn, Twitter, TikTok, and YouTube.',
'category': 'marketing',
'price': 6.0,
'agent_type': 'form',
'form_schema': {
'fields': [
{
'name': 'description',
'type': 'textarea',
'label': 'Describe what you\'d like to generate',
'placeholder': 'Describe the product, service, or campaign',
'required': True,
'rows': 4
},
{
'name': 'social_platform',
'type': 'select',
'label': 'For Social Media Platform',
'required': True,
'options': [
{'value': '', 'label': 'Select a platform...'},
{'value': 'facebook', 'label': 'Facebook'},
{'value': 'instagram', 'label': 'Instagram'},
{'value': 'linkedin', 'label': 'LinkedIn'},
{'value': 'twitter', 'label': 'X (Twitter)'}
]
},
{
'name': 'include_emoji',
'type': 'select',
'label': 'Include Emoji',
'required': True,
'options': [
{'value': '', 'label': 'Select an option...'},
{'value': 'yes', 'label': 'Yes'},
{'value': 'no', 'label': 'No'}
]
}
]
},
'webhook_url': 'http://localhost:5678/webhook/2dc234d8-7217-454a-83e9-81afe5b4fe2d',
'access_url_name': '',
'display_url_name': ''
},
{
'slug': 'job-posting-generator',
'name': 'Job Posting Generator',
'short_description': 'Create professional job postings that attract top talent',
'description': 'Generate comprehensive and attractive job postings with AI-powered content creation. Perfect for HR teams and recruiters.',
'category': 'human-resources',
'price': 10.0,
'agent_type': 'form',
'form_schema': {
'fields': [
{'name': 'job_title', 'type': 'text', 'label': 'Job Title', 'required': True},
{'name': 'company_name', 'type': 'text', 'label': 'Company Name', 'required': True},
{'name': 'job_description', 'type': 'textarea', 'label': 'Job Description', 'required': True, 'rows': 5},
{
'name': 'seniority_level',
'type': 'select',
'label': 'Seniority Level',
'required': True,
'options': [
{'value': '', 'label': 'Select level...'},
{'value': 'entry', 'label': 'Entry Level'},
{'value': 'junior', 'label': 'Junior'},
{'value': 'mid', 'label': 'Mid Level'},
{'value': 'senior', 'label': 'Senior'}
]
},
{'name': 'location', 'type': 'text', 'label': 'Location', 'required': True}
]
},
'webhook_url': 'http://localhost:5678/webhook/43f84411-eaaa-488c-9b1f-856e90d0aaf6',
'access_url_name': '',
'display_url_name': ''
},
{
'slug': 'pdf-summarizer',
'name': 'PDF Summarizer',
'short_description': 'Extract and summarize content from PDF documents with AI analysis',
'description': 'Upload PDF documents and get comprehensive AI-powered summaries, key insights, and analysis. Perfect for processing reports and research papers.',
'category': 'document-processing',
'price': 8.0,
'agent_type': 'form',
'form_schema': {
'fields': [
{
'name': 'pdf_file',
'type': 'file',
'label': 'Upload PDF Document',
'required': True,
'accept': '.pdf',
'max_size': '10MB'
},
{
'name': 'analysis_type',
'type': 'select',
'label': 'Analysis Type',
'required': True,
'default': 'summary',
'options': [
{'value': '', 'label': 'Select analysis type...'},
{'value': 'summary', 'label': 'Document Summary'},
{'value': 'key_points', 'label': 'Key Points Extraction'},
{'value': 'detailed_analysis', 'label': 'Detailed Analysis'}
]
}
]
},
'webhook_url': 'http://localhost:5678/webhook/simple-pdf-processor',
'access_url_name': '',
'display_url_name': ''
},
# Direct Access Agents
{
'slug': 'cybersec-career-navigator',
'name': 'CyberSec Career Navigator',
'short_description': 'Get personalized cybersecurity career guidance from AI expert Jessica',
'description': 'Navigate your cybersecurity career path with expert AI guidance. Get personalized advice on certifications, job roles, skills development, and career progression.',
'category': 'career-education',
'price': 0.0,
'agent_type': 'form',
'form_schema': {'fields': []},
'webhook_url': 'https://agent.jotform.com/019865a942ab7fa5b5b743a5fd2abe09e345',
'access_url_name': 'agents:direct_access_handler',
'display_url_name': 'agents:direct_access_display'
},
{
'slug': 'ai-brand-strategist',
'name': 'AI Brand Strategist',
'short_description': 'Get AI-powered brand strategy insights and recommendations for your business',
'description': 'Transform your brand with AI-driven strategic insights. Get expert guidance on brand positioning, messaging, visual identity, and competitive differentiation.',
'category': 'marketing',
'price': 0.0,
'agent_type': 'form',
'form_schema': {'fields': []},
'webhook_url': 'https://agent.jotform.com/01986502acd276b48e3d5f39337046c8d9b6',
'access_url_name': 'agents:direct_access_handler',
'display_url_name': 'agents:direct_access_display'
}
]
# Create agents # Get all JSON files in agents directory
for agent_data in agents_data: agent_files = list(agents_dir.glob('*.json'))
category = categories[agent_data['category']] if not agent_files:
self.stdout.write(self.style.WARNING('⚠️ No agent configuration files found'))
return
self.stdout.write(f'📁 Found {len(agent_files)} agent configuration files')
self.stdout.write('')
# Process each agent configuration file
for agent_file in sorted(agent_files):
try:
with open(agent_file, 'r', encoding='utf-8') as f:
agent_data = json.load(f)
# Validate required fields
required_fields = ['slug', 'name', 'category', 'price', 'agent_type']
missing_fields = [field for field in required_fields if field not in agent_data]
if missing_fields:
self.stdout.write(self.style.ERROR(f'❌ Missing fields in {agent_file.name}: {missing_fields}'))
continue
# Get category
category_slug = agent_data['category']
if category_slug not in categories:
self.stdout.write(self.style.ERROR(f'❌ Unknown category "{category_slug}" in {agent_file.name}'))
continue
category = categories[category_slug]
# Create agent
agent, created = Agent.objects.get_or_create(
slug=agent_data['slug'],
defaults={
'name': agent_data['name'],
'short_description': agent_data.get('short_description', ''),
'description': agent_data.get('description', ''),
'category': category,
'price': agent_data['price'],
'agent_type': agent_data['agent_type'],
'form_schema': agent_data.get('form_schema'),
'webhook_url': agent_data.get('webhook_url', ''),
'access_url_name': agent_data.get('access_url_name', ''),
'display_url_name': agent_data.get('display_url_name', '')
}
)
if created:
agents_created += 1
system_type = agent_data.get('system_type', 'webhook')
self.stdout.write(f'✅ Created agent: {agent.name} ({system_type.title()})')
self.stdout.write(f' 💰 Price: {agent.price} AED')
self.stdout.write(f' 📁 Config: {agent_file.name}')
else:
self.stdout.write(f' Agent exists: {agent.name} (from {agent_file.name})')
agent, created = Agent.objects.get_or_create( except json.JSONDecodeError as e:
slug=agent_data['slug'], self.stdout.write(self.style.ERROR(f'❌ Invalid JSON in {agent_file.name}: {e}'))
defaults={ continue
'name': agent_data['name'], except Exception as e:
'short_description': agent_data['short_description'], self.stdout.write(self.style.ERROR(f'❌ Error processing {agent_file.name}: {e}'))
'description': agent_data['description'], continue
'category': category,
'price': agent_data['price'],
'agent_type': agent_data['agent_type'],
'form_schema': agent_data['form_schema'],
'webhook_url': agent_data['webhook_url'],
'access_url_name': agent_data['access_url_name'],
'display_url_name': agent_data['display_url_name']
}
)
if created:
agents_created += 1
system_type = 'Direct Access' if agent.access_url_name else 'Webhook'
self.stdout.write(f'✅ Created agent: {agent.name} ({system_type})')
self.stdout.write(f' 💰 Price: {agent.price} AED')
else:
self.stdout.write(f' Agent exists: {agent.name}')
self.stdout.write('') self.stdout.write('')
self.stdout.write(self.style.SUCCESS('🎉 Population completed successfully!')) self.stdout.write(self.style.SUCCESS('🎉 Dynamic population completed successfully!'))
self.stdout.write('') self.stdout.write('')
self.stdout.write(f'📊 Summary:') self.stdout.write(f'📊 Summary:')
self.stdout.write(f' Categories created: {categories_created}') self.stdout.write(f' Categories created: {categories_created}')
self.stdout.write(f' Agents created: {agents_created}') self.stdout.write(f' Agents created: {agents_created}')
self.stdout.write(f' Configuration files processed: {len(agent_files)}')
self.stdout.write('') self.stdout.write('')
# Final verification # Final verification
@ -278,4 +134,9 @@ class Command(BaseCommand):
self.stdout.write(f' Webhook agents: {webhook_agents}') self.stdout.write(f' Webhook agents: {webhook_agents}')
self.stdout.write(f' Direct access agents: {direct_agents}') self.stdout.write(f' Direct access agents: {direct_agents}')
self.stdout.write('') self.stdout.write('')
self.stdout.write(self.style.SUCCESS('✅ Database is now consistent and ready!')) self.stdout.write(self.style.SUCCESS('✅ Database is now consistent and ready!'))
self.stdout.write('')
self.stdout.write('🚀 To add new agents:')
self.stdout.write(' 1. Create new JSON file in agents/configs/agents/')
self.stdout.write(' 2. Run this command again')
self.stdout.write(' 3. Agent will automatically appear in marketplace!')