feat(search): harvest custom DuckDuckGo endpoint#2767
Conversation
Add optional [search].base_url support for DuckDuckGo-compatible private search endpoints, including a preferred CODEWHALE_SEARCH_BASE_URL env override and the legacy DEEPSEEK_SEARCH_BASE_URL alias. Network policy now gates the configured endpoint host, custom endpoints do not fall back to public Bing, non-DuckDuckGo provider/base_url combinations and challenge pages return explicit errors, and custom endpoint results report the configured host as their source. Fixes #2436 Reported by @Artenx Harvested from PR #2510 by @cyq1017 Co-authored-by: cyq1017 <61975706+cyq1017@users.noreply.github.com>
There was a problem hiding this comment.
Hmbown has reached the 50-review limit for trial accounts. To continue receiving code reviews, upgrade your plan.
There was a problem hiding this comment.
Code Review
This pull request adds support for custom DuckDuckGo-compatible private search endpoints via the [search].base_url configuration or the CODEWHALE_SEARCH_BASE_URL environment variable. Custom endpoints bypass the public Bing fallback, enforce network policies on their configured host, and report the custom host as the result source. The review feedback suggests validating that the custom base_url uses an http or https scheme to prevent potential runtime errors during request execution.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| let mut url = reqwest::Url::parse(raw).map_err(|err| { | ||
| ToolError::invalid_input(format!( | ||
| "Invalid DuckDuckGo-compatible search base_url: {err}" | ||
| )) | ||
| })?; |
There was a problem hiding this comment.
The custom base_url is parsed using reqwest::Url::parse, but there is no validation to ensure that the scheme is either http or https. If an unsupported scheme (like ftp or file) is configured, it could lead to unexpected runtime errors or failures when reqwest attempts to execute the request. Adding an explicit check for http or https schemes improves robustness.
let mut url = reqwest::Url::parse(raw).map_err(|err| {
ToolError::invalid_input(format!(
"Invalid DuckDuckGo-compatible search base_url: {err}"
))
})?;
if url.scheme() != "http" && url.scheme() != "https" {
return Err(ToolError::invalid_input(format!(
"DuckDuckGo-compatible search base_url must use http or https scheme, got: {}",
url.scheme()
)));
}
Summary
[search].base_url, preferredCODEWHALE_SEARCH_BASE_URL, and legacyDEEPSEEK_SEARCH_BASE_URLsupport.sourceinstead of alwaysduckduckgo.Credit
Harvested from PR #2510 by @cyq1017.
Reported by @Artenx in #2436.
Verification
cargo test -p codewhale-tui --all-features --locked custom_duckduckgo -- --nocapturecargo test -p codewhale-tui --all-features --locked search_base_url -- --nocapturecargo test -p codewhale-tui --all-features --locked search_provider -- --nocapturecargo check -p codewhale-tui --all-features --lockedcargo clippy -p codewhale-tui --all-features --locked -- -D warningscargo fmt --all -- --checkgit diff --check./scripts/release/check-versions.shpython3 scripts/check-coauthor-trailers.py --author-map .github/AUTHOR_MAP --range origin/codex/v0.9.0-stewardship..HEAD --check-authorsFixes #2436.