An AI-powered search engine with a generative UI
TypeScript generative-aigenerative-uinextjsollamareactsearxngshadcn-uitailwindcsstavilytypescriptupstashvercel-ai-sdk
Fix SearXNG from SearchXNG
Yoshiki Miurae54a818142fa
diff --git a/.env.local.example b/.env.local.exampleindex 8ae604ba..064b5394 100644--- a/.env.local.example+++ b/.env.local.example@@ -15,8 +15,9 @@ LOCAL_REDIS_URL=redis://localhost:6379 # or redis://redis:6379 if you're using dUPSTASH_REDIS_REST_URL=[YOUR_UPSTASH_REDIS_REST_URL]UPSTASH_REDIS_REST_TOKEN=[YOUR_UPSTASH_REDIS_REST_TOKEN]-SEARCHXNG_API_URL=http://localhost:8080 # Replace with your local SearchXNG API URL or docker http://searchxng:8080-SEARCH_API=tavily # use searchxng, tavily or exa+# SearXNG Configuration+SEARXNG_API_URL=http://localhost:8080 # Replace with your local SearXNG API URL or docker http://searxng:8080+SEARCH_API=tavily # use searxng, tavily or exaSEARXNG_SECRET="" # generate a secret key e.g. openssl rand -base64 32SEARXNG_PORT=8080 # default portSEARXNG_BIND_ADDRESS=0.0.0.0 # default addressdiff --git a/README.md b/README.mdindex e91ec343..52ccecb9 100644--- a/README.md+++ b/README.md@@ -39,7 +39,7 @@ An AI-powered search engine with a generative UI.- App framework: [Next.js](https://nextjs.org/)- Text streaming / Generative UI: [Vercel AI SDK](https://sdk.vercel.ai/docs)- Generative Model: [OpenAI](https://openai.com/)-- Search API: [Tavily AI](https://tavily.com/) / [Serper](https://serper.dev) / [SearchXNG](https://docs.searxng.org/)+- Search API: [Tavily AI](https://tavily.com/) / [Serper](https://serper.dev) / [SearXNG](https://docs.searxng.org/)- Reader API: [Jina AI](https://jina.ai/)- Database (Serverless/Local): [Upstash](https://upstash.com/) / [Redis](https://redis.io/)- Component library: [shadcn/ui](https://ui.shadcn.com/)@@ -158,40 +158,39 @@ If you want to use Morphic as a search engine in your browser, follow these stepThis will allow you to use Morphic as your default search engine in the browser.-### Using SearchXNG as an Alternative Search Backend+### Using SearXNG as an Alternative Search Backend-Morphic now supports SearchXNG as an alternative search backend. To use SearchXNG:+Morphic now supports SearXNG as an alternative search backend. To use SearXNG:1. Ensure you have Docker and Docker Compose installed on your system.2. In your `.env.local` file, set the following variables:- - SEARCHXNG_API_URL=http://localhost:8080 # Replace with your local SearchXNG API URL or docker http://searchxng:8080+ - SEARXNG_API_URL=http://localhost:8080 # Replace with your local SearXNG API URL or docker http://searxng:8080- SEARXNG_SECRET=your_secret_key_here- SEARXNG_PORT=8080- SEARXNG_IMAGE_PROXY=true- - SEARCH_API=searchxng+ - SEARCH_API=searxng- SEARXNG_LIMITER=false # can be enabled to limit the number of requests per IP- - SEARCH_API=searchxng3. Two configuration files are provided in the root directory:-- `searxng-settings.yml`: This file contains the main configuration for SearchXNG, including engine settings and server options.-- `searxng-limiter.toml`: This file configures the rate limiting and bot detection features of SearchXNG.+- `searxng-settings.yml`: This file contains the main configuration for SearXNG, including engine settings and server options.+- `searxng-limiter.toml`: This file configures the rate limiting and bot detection features of SearXNG.-4. Run `docker-compose up` to start the Morphic stack with SearchXNG included.-5. SearchXNG will be available at `http://localhost:8080` and Morphic will use it as the search backend.+4. Run `docker-compose up` to start the Morphic stack with SearXNG included.+5. SearXNG will be available at `http://localhost:8080` and Morphic will use it as the search backend.-#### Customizing SearchXNG+#### Customizing SearXNG- You can modify `searxng-settings.yml` to enable/disable specific search engines, change UI settings, or adjust server options.-- The `searxng-limiter.toml` file allows you to configure rate limiting and bot detection. This is useful if you're exposing SearchXNG directly to the internet.-- If you prefer not to use external configuration files, you can set these options using environment variables in the `docker-compose.yml` file or directly in the SearchXNG container.+- The `searxng-limiter.toml` file allows you to configure rate limiting and bot detection. This is useful if you're exposing SearXNG directly to the internet.+- If you prefer not to use external configuration files, you can set these options using environment variables in the `docker-compose.yml` file or directly in the SearXNG container.#### Advanced Configuration-- To disable the limiter entirely, set `LIMITER=false` in the SearchXNG service environment variables.+- To disable the limiter entirely, set `LIMITER=false` in the SearXNG service environment variables.- For production use, consider adjusting the `SEARXNG_SECRET_KEY` to a secure, randomly generated value.-- The `SEARXNG_IMAGE_PROXY` option allows SearchXNG to proxy image results, enhancing privacy. Set to `true` to enable this feature.+- The `SEARXNG_IMAGE_PROXY` option allows SearXNG to proxy image results, enhancing privacy. Set to `true` to enable this feature.#### Troubleshootingdiff --git a/docker-compose.yaml b/docker-compose.yamlindex fd0d2715..a36c45a6 100644--- a/docker-compose.yaml+++ b/docker-compose.yaml@@ -9,29 +9,29 @@ services:command: bun dev # Use `bun dev -H 0.0.0.0` to listen on all interfacesenv_file: .env.local # Load environment variablesports:- - "3000:3000" # Maps port 3000 on the host to port 3000 in the container.+ - '3000:3000' # Maps port 3000 on the host to port 3000 in the container.depends_on:- redis- - searchxng+ - searxngredis:image: redis:alpineports:- - "6379:6379"+ - '6379:6379'volumes:- redis_data:/datacommand: redis-server --appendonly yes- searchxng:+ searxng:image: searxng/searxngports:- - "${SEARXNG_PORT:-8080}:8080"+ - '${SEARXNG_PORT:-8080}:8080'env_file: .env.local # can remove if you want to use env variables or in settings.ymlvolumes:- ./searxng-limiter.toml:/etc/searxng/limiter.toml- ./searxng-settings.yml:/etc/searxng/settings.yml- - searchxng_data:/data+ - searxng_data:/datavolumes:redis_data:- searchxng_data:\ No newline at end of file+ searxng_data:diff --git a/lib/agents/tools/search.tsx b/lib/agents/tools/search.tsxindex c8743e65..79129549 100644--- a/lib/agents/tools/search.tsx+++ b/lib/agents/tools/search.tsx@@ -9,8 +9,8 @@ import {SearchResultImage,SearchResults,SearchResultItem,- SearchXNGResponse,- SearchXNGResult+ SearXNGResponse,+ SearXNGResult} from '@/lib/types'export const searchTool = ({ uiStream, fullResponse }: ToolProps) =>@@ -39,7 +39,7 @@ export const searchTool = ({ uiStream, fullResponse }: ToolProps) =>query.length < 5 ? query + ' '.repeat(5 - query.length) : querylet searchResult: SearchResultsconst searchAPI =- (process.env.SEARCH_API as 'tavily' | 'exa' | 'searchxng') || 'tavily'+ (process.env.SEARCH_API as 'tavily' | 'exa' | 'searxng') || 'tavily'console.log(`Using search API: ${searchAPI}`)try {@@ -47,7 +47,7 @@ export const searchTool = ({ uiStream, fullResponse }: ToolProps) =>? tavilySearch: searchAPI === 'exa'? exaSearch- : searchXNGSearch)(+ : searxngSearch)(filledQuery,max_results,search_depth,@@ -168,16 +168,16 @@ async function exaSearch(}}-async function searchXNGSearch(+async function searxngSearch(query: string,maxResults: number = 10,_searchDepth: string,includeDomains: string[] = [], //keep for future useexcludeDomains: string[] = [] //keep for future use): Promise<SearchResults> {- const apiUrl = process.env.SEARCHXNG_API_URL+ const apiUrl = process.env.SEARXNG_API_URLif (!apiUrl) {- throw new Error('SEARCHXNG_API_URL is not set in the environment variables')+ throw new Error('SEARXNG_API_URL is not set in the environment variables')}try {@@ -188,7 +188,7 @@ async function searchXNGSearch(// Enable both general and image resultsurl.searchParams.append('categories', 'general,images')- // Fetch results from SearchXNG+ // Fetch results from SearXNGconst response = await fetch(url.toString(), {method: 'GET',headers: {@@ -198,13 +198,13 @@ async function searchXNGSearch(if (!response.ok) {const errorText = await response.text()- console.error(`SearchXNG API error (${response.status}):`, errorText)+ console.error(`SearXNG API error (${response.status}):`, errorText)throw new Error(- `SearchXNG API error: ${response.status} ${response.statusText} - ${errorText}`+ `SearXNG API error: ${response.status} ${response.statusText} - ${errorText}`)}- const data: SearchXNGResponse = await response.json()+ const data: SearXNGResponse = await response.json()// Separate general results and image results, and limit to maxResultsconst generalResults = data.results@@ -217,7 +217,7 @@ async function searchXNGSearch(// Format the results to match the expected SearchResults structurereturn {results: generalResults.map(- (result: SearchXNGResult): SearchResultItem => ({+ (result: SearXNGResult): SearchResultItem => ({title: result.title,url: result.url,content: result.content@@ -233,7 +233,7 @@ async function searchXNGSearch(number_of_results: data.number_of_results}} catch (error) {- console.error('SearchXNG API error:', error)+ console.error('SearXNG API error:', error)throw error}}diff --git a/lib/types/index.ts b/lib/types/index.tsindex 8a694e3d..ff7c5da7 100644--- a/lib/types/index.ts+++ b/lib/types/index.ts@@ -13,8 +13,7 @@ export type SearchResultImage =url: stringdescription: stringnumber_of_results?: number-}-+ }export type ExaSearchResults = {results: ExaSearchResultItem[]@@ -83,15 +82,15 @@ export type AIMessage = {| 'end'}-export interface SearchXNGResult {- title: string;- url: string;- content: string;- img_src?: string;+export interface SearXNGResult {+ title: string+ url: string+ content: string+ img_src?: string}-export interface SearchXNGResponse {- query: string;- number_of_results: number;- results: SearchXNGResult[];+export interface SearXNGResponse {+ query: string+ number_of_results: number+ results: SearXNGResult[]}