About Extract Chat - chat with firecrawl
From the template README, written by its publisher.- Extract.chat: AI Chatbot with Firecrawl Integration
- Extract.chat is an AI-powered chatbot application that enables real-time data extraction and search capabilities through Firecrawl integration. This project is a fork of Vercel's AI Chatbot, enhanced with specialized tools for retrieving and analyzing information from websites.
- Project Overview
- Extract.chat combines the power of Next.js, AI SDK, and Firecrawl to create a versatile chatbot that can not only engage in conversations but also fetch and extract structured data from the web in real-time.
Short description (75 characters):
- Web-enabled AI assistant that searches and extracts structured data from websites.
- Key Features
- Core Functionality
- Firecrawl Integration: Search the web for information and extract structured data from multiple websites
- Next.js App Router: Advanced routing with React Server Components (RSCs) and Server Actions
- AI SDK Integration: Unified API for generating text, structured objects, and tool calls with LLMs
- User Authentication: Secure login/registration system with anonymous access option
- Document Creation: Generate and manipulate text, code, and spreadsheet documents
Technical Highlights
- Data Persistence: PostgreSQL database integration for chat history and user data
- File Storage: Vercel Blob support for efficient file handling
- Responsive UI: Modern interface built with shadcn/ui and Tailwind CSS
- Real-time Streaming: Server-sent events for responsive chat interactions
Architecture
The application follows a modern web architecture:
- Frontend: React-based UI with Next.js App Router
- Backend: Server-side functions for authentication, chat processing, and database operations
- AI Integration: Connection to AI models via the AI SDK
- Web Integration: Firecrawl API for searching and extracting data from websites
- Database: Vercel Postgres for data persistence
Deployment Requirements
To deploy Extract.chat, you'll need:
Environment Variables
- OPENAI_API_KEY: API key for OpenAI
- AUTH_SECRET: A secret for authentication
- BLOB_READ_WRITE_TOKEN: Vercel Blob storage token
- POSTGRES_URL: Connection string for Postgres database
- FIRECRAWL_API_KEY: API key for Firecrawl service
Vercel Account: For hosting and database integration
Node.js environment: For local development
Node.js environment: For local development
Getting Started
Clone the repository
- Install dependencies: pnpm install
- Set up environment variables in .env.local
- Start the development server: pnpm dev
Customization Options
- AI Model: Configured to use OpenAI gpt-4o by default, but can be switched to other providers
- UI Appearance: Customizable with Tailwind CSS
- Authentication Flow: Modifiable login/register processes
Use Cases
- Research Assistant: Quickly gather information from multiple websites
- Data Extraction: Pull structured data from web pages
- Content Creation: Generate documents and code with AI assistance
- Knowledge Base: Create and maintain a personal knowledge repository
