Skip to content

An AI-powered search engine with a generative UI

Fix SearXNG from SearchXNG

Yoshiki Miurae54a818142fa
diff --git a/.env.local.example b/.env.local.example
index 8ae604ba..064b5394 100644
--- a/.env.local.example
+++ b/.env.local.example
@@ -15,8 +15,9 @@ LOCAL_REDIS_URL=redis://localhost:6379 # or redis://redis:6379 if you're using d
UPSTASH_REDIS_REST_URL=[YOUR_UPSTASH_REDIS_REST_URL]
UPSTASH_REDIS_REST_TOKEN=[YOUR_UPSTASH_REDIS_REST_TOKEN]
-SEARCHXNG_API_URL=http://localhost:8080 # Replace with your local SearchXNG API URL or docker http://searchxng:8080
-SEARCH_API=tavily # use searchxng, tavily or exa
+# SearXNG Configuration
+SEARXNG_API_URL=http://localhost:8080 # Replace with your local SearXNG API URL or docker http://searxng:8080
+SEARCH_API=tavily # use searxng, tavily or exa
SEARXNG_SECRET="" # generate a secret key e.g. openssl rand -base64 32
SEARXNG_PORT=8080 # default port
SEARXNG_BIND_ADDRESS=0.0.0.0 # default address
diff --git a/README.md b/README.md
index e91ec343..52ccecb9 100644
--- a/README.md
+++ b/README.md
@@ -39,7 +39,7 @@ An AI-powered search engine with a generative UI.
- App framework: [Next.js](https://nextjs.org/)
- Text streaming / Generative UI: [Vercel AI SDK](https://sdk.vercel.ai/docs)
- Generative Model: [OpenAI](https://openai.com/)
-- Search API: [Tavily AI](https://tavily.com/) / [Serper](https://serper.dev) / [SearchXNG](https://docs.searxng.org/)
+- Search API: [Tavily AI](https://tavily.com/) / [Serper](https://serper.dev) / [SearXNG](https://docs.searxng.org/)
- Reader API: [Jina AI](https://jina.ai/)
- Database (Serverless/Local): [Upstash](https://upstash.com/) / [Redis](https://redis.io/)
- Component library: [shadcn/ui](https://ui.shadcn.com/)
@@ -158,40 +158,39 @@ If you want to use Morphic as a search engine in your browser, follow these step
This will allow you to use Morphic as your default search engine in the browser.
-### Using SearchXNG as an Alternative Search Backend
+### Using SearXNG as an Alternative Search Backend
-Morphic now supports SearchXNG as an alternative search backend. To use SearchXNG:
+Morphic now supports SearXNG as an alternative search backend. To use SearXNG:
1. Ensure you have Docker and Docker Compose installed on your system.
2. In your `.env.local` file, set the following variables:
- - SEARCHXNG_API_URL=http://localhost:8080 # Replace with your local SearchXNG API URL or docker http://searchxng:8080
+ - SEARXNG_API_URL=http://localhost:8080 # Replace with your local SearXNG API URL or docker http://searxng:8080
- SEARXNG_SECRET=your_secret_key_here
- SEARXNG_PORT=8080
- SEARXNG_IMAGE_PROXY=true
- - SEARCH_API=searchxng
+ - SEARCH_API=searxng
- SEARXNG_LIMITER=false # can be enabled to limit the number of requests per IP
- - SEARCH_API=searchxng
3. Two configuration files are provided in the root directory:
-- `searxng-settings.yml`: This file contains the main configuration for SearchXNG, including engine settings and server options.
-- `searxng-limiter.toml`: This file configures the rate limiting and bot detection features of SearchXNG.
+- `searxng-settings.yml`: This file contains the main configuration for SearXNG, including engine settings and server options.
+- `searxng-limiter.toml`: This file configures the rate limiting and bot detection features of SearXNG.
-4. Run `docker-compose up` to start the Morphic stack with SearchXNG included.
-5. SearchXNG will be available at `http://localhost:8080` and Morphic will use it as the search backend.
+4. Run `docker-compose up` to start the Morphic stack with SearXNG included.
+5. SearXNG will be available at `http://localhost:8080` and Morphic will use it as the search backend.
-#### Customizing SearchXNG
+#### Customizing SearXNG
- You can modify `searxng-settings.yml` to enable/disable specific search engines, change UI settings, or adjust server options.
-- The `searxng-limiter.toml` file allows you to configure rate limiting and bot detection. This is useful if you're exposing SearchXNG directly to the internet.
-- If you prefer not to use external configuration files, you can set these options using environment variables in the `docker-compose.yml` file or directly in the SearchXNG container.
+- The `searxng-limiter.toml` file allows you to configure rate limiting and bot detection. This is useful if you're exposing SearXNG directly to the internet.
+- If you prefer not to use external configuration files, you can set these options using environment variables in the `docker-compose.yml` file or directly in the SearXNG container.
#### Advanced Configuration
-- To disable the limiter entirely, set `LIMITER=false` in the SearchXNG service environment variables.
+- To disable the limiter entirely, set `LIMITER=false` in the SearXNG service environment variables.
- For production use, consider adjusting the `SEARXNG_SECRET_KEY` to a secure, randomly generated value.
-- The `SEARXNG_IMAGE_PROXY` option allows SearchXNG to proxy image results, enhancing privacy. Set to `true` to enable this feature.
+- The `SEARXNG_IMAGE_PROXY` option allows SearXNG to proxy image results, enhancing privacy. Set to `true` to enable this feature.
#### Troubleshooting
diff --git a/docker-compose.yaml b/docker-compose.yaml
index fd0d2715..a36c45a6 100644
--- a/docker-compose.yaml
+++ b/docker-compose.yaml
@@ -9,29 +9,29 @@ services:
command: bun dev # Use `bun dev -H 0.0.0.0` to listen on all interfaces
env_file: .env.local # Load environment variables
ports:
- - "3000:3000" # Maps port 3000 on the host to port 3000 in the container.
+ - '3000:3000' # Maps port 3000 on the host to port 3000 in the container.
depends_on:
- redis
- - searchxng
+ - searxng
redis:
image: redis:alpine
ports:
- - "6379:6379"
+ - '6379:6379'
volumes:
- redis_data:/data
command: redis-server --appendonly yes
- searchxng:
+ searxng:
image: searxng/searxng
ports:
- - "${SEARXNG_PORT:-8080}:8080"
+ - '${SEARXNG_PORT:-8080}:8080'
env_file: .env.local # can remove if you want to use env variables or in settings.yml
volumes:
- ./searxng-limiter.toml:/etc/searxng/limiter.toml
- ./searxng-settings.yml:/etc/searxng/settings.yml
- - searchxng_data:/data
+ - searxng_data:/data
volumes:
redis_data:
- searchxng_data:
\ No newline at end of file
+ searxng_data:
diff --git a/lib/agents/tools/search.tsx b/lib/agents/tools/search.tsx
index c8743e65..79129549 100644
--- a/lib/agents/tools/search.tsx
+++ b/lib/agents/tools/search.tsx
@@ -9,8 +9,8 @@ import {
SearchResultImage,
SearchResults,
SearchResultItem,
- SearchXNGResponse,
- SearchXNGResult
+ SearXNGResponse,
+ SearXNGResult
} from '@/lib/types'
export const searchTool = ({ uiStream, fullResponse }: ToolProps) =>
@@ -39,7 +39,7 @@ export const searchTool = ({ uiStream, fullResponse }: ToolProps) =>
query.length < 5 ? query + ' '.repeat(5 - query.length) : query
let searchResult: SearchResults
const searchAPI =
- (process.env.SEARCH_API as 'tavily' | 'exa' | 'searchxng') || 'tavily'
+ (process.env.SEARCH_API as 'tavily' | 'exa' | 'searxng') || 'tavily'
console.log(`Using search API: ${searchAPI}`)
try {
@@ -47,7 +47,7 @@ export const searchTool = ({ uiStream, fullResponse }: ToolProps) =>
? tavilySearch
: searchAPI === 'exa'
? exaSearch
- : searchXNGSearch)(
+ : searxngSearch)(
filledQuery,
max_results,
search_depth,
@@ -168,16 +168,16 @@ async function exaSearch(
}
}
-async function searchXNGSearch(
+async function searxngSearch(
query: string,
maxResults: number = 10,
_searchDepth: string,
includeDomains: string[] = [], //keep for future use
excludeDomains: string[] = [] //keep for future use
): Promise<SearchResults> {
- const apiUrl = process.env.SEARCHXNG_API_URL
+ const apiUrl = process.env.SEARXNG_API_URL
if (!apiUrl) {
- throw new Error('SEARCHXNG_API_URL is not set in the environment variables')
+ throw new Error('SEARXNG_API_URL is not set in the environment variables')
}
try {
@@ -188,7 +188,7 @@ async function searchXNGSearch(
// Enable both general and image results
url.searchParams.append('categories', 'general,images')
- // Fetch results from SearchXNG
+ // Fetch results from SearXNG
const response = await fetch(url.toString(), {
method: 'GET',
headers: {
@@ -198,13 +198,13 @@ async function searchXNGSearch(
if (!response.ok) {
const errorText = await response.text()
- console.error(`SearchXNG API error (${response.status}):`, errorText)
+ console.error(`SearXNG API error (${response.status}):`, errorText)
throw new Error(
- `SearchXNG API error: ${response.status} ${response.statusText} - ${errorText}`
+ `SearXNG API error: ${response.status} ${response.statusText} - ${errorText}`
)
}
- const data: SearchXNGResponse = await response.json()
+ const data: SearXNGResponse = await response.json()
// Separate general results and image results, and limit to maxResults
const generalResults = data.results
@@ -217,7 +217,7 @@ async function searchXNGSearch(
// Format the results to match the expected SearchResults structure
return {
results: generalResults.map(
- (result: SearchXNGResult): SearchResultItem => ({
+ (result: SearXNGResult): SearchResultItem => ({
title: result.title,
url: result.url,
content: result.content
@@ -233,7 +233,7 @@ async function searchXNGSearch(
number_of_results: data.number_of_results
}
} catch (error) {
- console.error('SearchXNG API error:', error)
+ console.error('SearXNG API error:', error)
throw error
}
}
diff --git a/lib/types/index.ts b/lib/types/index.ts
index 8a694e3d..ff7c5da7 100644
--- a/lib/types/index.ts
+++ b/lib/types/index.ts
@@ -13,8 +13,7 @@ export type SearchResultImage =
url: string
description: string
number_of_results?: number
-}
-
+ }
export type ExaSearchResults = {
results: ExaSearchResultItem[]
@@ -83,15 +82,15 @@ export type AIMessage = {
| 'end'
}
-export interface SearchXNGResult {
- title: string;
- url: string;
- content: string;
- img_src?: string;
+export interface SearXNGResult {
+ title: string
+ url: string
+ content: string
+ img_src?: string
}
-export interface SearchXNGResponse {
- query: string;
- number_of_results: number;
- results: SearchXNGResult[];
+export interface SearXNGResponse {
+ query: string
+ number_of_results: number
+ results: SearXNGResult[]
}