Scout — 10 candidates

[github-actions]

Candidates (10)

• Designing Efficient Verifiers for Legal Agents — eval-philosophy · article
  Source: langchain-blog | Proposed slug: designing-efficient-verifiers-legal-agents
  Blurb: Harvey × LangChain Labs study on making LLM-as-judge verifiers cheaper and more reliable for legal-agent evaluation and post-training. Covers verifier design trade-offs, cost/accuracy curves, and how verifier quality propagates into RLVR training signal.
  Rationale: Vendor blog but contains a concrete, reproducible study (Harvey + LangChain Labs) on verifier design for a real production domain. Covers practical trade-offs between verifier cost and reliability and how those choices affect both evaluation and post-training — directly relevant to the eval-philosophy section's focus on building rigorous evaluation loops beyond vibes-based testing. The legal-agent domain provides a high-stakes, structured use case that generalises well to other agent verticals.

• How we build evals for Deep Agents — eval-philosophy · article
  Source: langchain-blog | Proposed slug: langchain-evals-deep-agents
  Blurb: Practical eval construction for long-horizon agents: data curation, behavior-targeted metrics, and testing strategies grounded in real agent runs.
  Rationale: LangChain has direct production experience operating "Deep Agents" (long-horizon, multi-step agents) and this post covers concrete eval construction decisions — data curation, metric design, and testing strategies — rather than generic advice. Vendor-origin is acceptable here because the techniques are reproducible and grounded in real agent behaviour. Fits eval-philosophy (foundational thinking on eval loops) though could also sit in eval-frameworks; eval-philosophy is the better fit given the strategic framing around what to measure and why. Summary is thin but the topic and source credibility justify inclusion pending content review.

• Building a 100x Cheaper Trace Judge with Fireworks — eval-philosophy · article
  Source: langchain-blog | Proposed slug: langchain-fireworks-trace-judge
  Blurb: Fine-tuning a small open model on production trace signals to match frontier judge performance at ~1/100th the cost — covers data mining strategy, labelling pipeline, and distillation approach.
  Rationale: Substantive technical content from a vendor blog that contains reproducible techniques: using production traces as a signal source, distilling a frontier judge into a cheaper fine-tuned model, and the labelling/training pipeline decisions. Directly relevant to eval infrastructure and the "production trace sampling back into eval" theme of production-testing. Not pure marketing — the methodology is detailed and actionable. Could also sit in eval-philosophy but the emphasis on cost and production trace mining makes it a strong fit there.

• Iterating Towards LLM Reliability with Evaluation Driven Development — eval-philosophy · article
  Source: langchain-blog | Proposed slug: eval-driven-development-dosu
  Blurb: Dosu's production loop: define evaluators first, gate deploys on eval regressions, and feed production traces back into the test suite — evaluation-driven development as an engineering discipline.
  Rationale: This is a vendor blog post (LangChain/LangSmith) but it contains reproducible architectural decisions and a concrete methodology — evaluation-driven development as an engineering practice, with production trace feedback loops and deploy gates. That's substantive content directly relevant to eval philosophy and production testing patterns. The Dosu case study grounds the abstract methodology in real implementation choices. Not pure marketing; the technique is the content.

• Aligning LLM-as-a-Judge with Human Preferences — eval-philosophy · article
  Source: langchain-blog | Proposed slug: langsmith-aligning-llm-judge-human-preferences
  Blurb: Self-improving LLM-as-a-Judge evaluators via few-shot learning and human preference alignment — how to close the gap between automated scores and human judgment in LangSmith.
  Rationale: Vendor blog, but addresses a substantive engineering problem — calibrating LLM judges to human preferences — with reproducible techniques (few-shot example selection, preference alignment loops). LLM-as-a-Judge calibration is a core eval-philosophy concern and this goes beyond marketing into concrete methodology. Acceptable under the vendor-blog rule.

• Building LangGraph: Designing an Agent Runtime from First Principles — multi-agent-frameworks · article
  Source: langchain-blog | Proposed slug: building-langgraph-runtime
  Blurb: Design rationale behind LangGraph: why graph-based orchestration, how control flow, durability, and state management were architected for production agents.
  Rationale: Vendor blog, but clearly contains substantive architectural content — design decisions, trade-offs, and first-principles reasoning behind a widely-used agent runtime. This is the kind of "why we built it this way" content that has direct implications for practitioners choosing or extending orchestration frameworks. LangGraph is a notable production artefact with broad adoption, so the design rationale is genuinely useful. Not a press release or marketing piece. Acceptable under the vendor blog carve-out.

• How to turn Claude Code into a domain specific coding agent — coding-agent-infra · article
  Source: langchain-blog | Proposed slug: claude-code-domain-specific-agent
  Blurb: Context engineering techniques for adapting Claude Code to domain-specific libraries — CLAUDE.md, tool allowlists, and custom docs injection — with eval results comparing approaches.
  Rationale: Directly relevant to coding agent infrastructure: covers harness design (what context to inject, how), tool allowlists, and domain-specific customization with evaluation-backed results. The eval component lifts it above a simple tutorial — concrete takeaways on which techniques move the needle. Vendor blog (LangChain) but contains reproducible techniques rather than marketing. Good fit for coding-agent-infra; also touches eval philosophy but the primary focus is harness/customization design.

• Evaluating Deep Agents: Our Learnings — eval-philosophy · article
  Source: langchain-blog | Proposed slug: evaluating-deep-agents-langchain
  Blurb: Five evaluation patterns for long-horizon agents: bespoke assertions, single-step tool validation, full-turn scoring, multi-turn simulation, and environment setup hygiene. Grounded in production experience rather than theory.
  Rationale: Vendor blog, but contains reproducible evaluation patterns with clear architectural distinctions between granularities (single-step → full turn → multi-turn). The five-pattern taxonomy is practical and opinionated, fitting the eval-philosophy section's mandate. Not purely marketing — the content is method-driven and applicable beyond LangChain tooling.

• How to Choose the Right Sandbox for AI Agents — sandboxing · article
  Source: langchain-blog | Proposed slug: langchain-choose-right-sandbox-ai-agents
  Blurb: Decision framework for sandbox selection: filesystem isolation, network access controls, resource limits, and microVM trade-offs — mapped to agent execution patterns.
  Rationale: Directly on-topic for the sandboxing section — covers the core axes of sandbox selection (filesystem isolation, network, resource limits, microVMs) with practical guidance applicable to agent builders. Vendor blog (LangChain) but the content appears architectural/decision-making in nature rather than a product pitch. Meets the bar for reproducible techniques and design decisions rather than pure marketing.

• Fault Tolerance in LangGraph: Retries, Timeouts and Error Handlers — durable-execution · article
  Source: langchain-blog | Proposed slug: langgraph-fault-tolerance
  Blurb: Three fault-tolerance primitives inside LangGraph: RetryPolicy (backoff retries), TimeoutPolicy (wall-clock and idle caps), and error_handler (post-exhaustion cleanup). Covers how they compose and applies the SAGA pattern to multi-step workflows with real-world side effects.
  Rationale: Substantive vendor content with concrete, reproducible techniques: three named primitives with clear semantics, composition rules, and a SAGA pattern applied to agentic workflows. Directly relevant to durable-execution concerns (fault tolerance, compensating transactions) rather than marketing. No substantially similar resource covering LangGraph-specific fault-tolerance mechanics exists in the guide.