A carefully curated collection of SEO guides, tools, and references for people who build and operate websites. Started as personal SEO notes in 2019; now maintained as a public knowledge resource by i365.
New to SEO? Begin with Google Search Essentials and the official SEO Starter Guide. Use Search Console to measure search visibility before choosing tools.
Curation policy: This is a reference list, not a product directory, advertising channel, or backlink placement service. Unsolicited vendor and self-promotional listings are not accepted. See CONTRIBUTING.md.
flowchart TD
intent["Search intent and user needs"] --> content["Useful, distinctive content"]
content --> access["Crawlable and indexable pages"]
access --> visibility["Potential search visibility and visits"]
visibility --> evidence["Measure with Search Console and site analytics"]
evidence --> improve["Fix issues and improve content"]
improve --> content
Indexable does not mean ranked: visibility depends on relevance, competition, and search-engine decisions. The loop is about publishing helpful pages, observing results, and improving them—not producing links or content for their own sake.
The following older mind map is a broad reference; check current Google Search documentation for technical guidance.
Google's baseline technical requirements, spam policies, and search best practices.
Google's introduction to making content accessible, useful, and discoverable through Search.
SEO beginners tutorial on the Moz web, helps you learn SEO systematically.
A Technical SEO’s Guide to Crawl Budget Optimization
A comprehensive guide explaining the importance of crawl budgets in managing indexing for search engines, how it impacts website traffic, and provides optimization tips to improve online visibility and SEO results.
In this book, four of the most noted experts in the field of search engine optimization (SEO) provide you with proven guidelines and cutting-edge techniques. The third edition was written in 2015, so if you only want to read one book about SEO, this book is the best option.
这就是搜索引擎(Chinese Book)
This book is about the internal technology of popular science search engines. Although Using SEO does not need to understand the internal implementation of search engines, for some technical optimization, understanding the characteristics of some search engines will help you optimize your website from a macro perspective.
The videos about popular science search engine that Google published.
12 SEO Best Practices That Everyone Should Follow
SEO best practices that Ahrefs write.
17 Actionable Content Marketing Tips For More Traffic
The content marketing is one of important directions in SEO. The article content mainly involves content marketing suggestions, and it written by Brian Dean who research backlink.
Here’s What We Learned About SEO
The article studies the factors influencing search engine rankings through large amounts of data analysis.
What Is SEO Writing? How to Be a Better SEO Writer
The article mainly introduce the writing framework of SEO, and suggestions about how to be a better SEO write.
The Authority Site Business Model
The article introduce the whole process that an authority site from construction to profitability.
零成本搭建SEO友好型博客指南(Chinese Blogs)
This is a series of blogs that introduces how to build a SEO from scratch. These blogs content covered blog building, SEO optimization, subcribe and push, website loading speed optimization.
This article detail explains how to analyse website access logs to identify problems in SEO. Google Analytics can help us understand user access, and system logs can help us better understand crawler requests, website techical SEO issues such as 301 redirects and so on.
Google Ads Benchmarks for YOUR Industry
Last updated: October 5, 2020. It can help to learn about the money-sucking power of each industry.
如何准确查询关键词热度和搜索量(Chinese Website)
For knowing what's hot on the web and avoiding Red Sea competition.
The resources below prioritize official search-engine tools, mature open-source projects, and established industry tools—not new vendor listings.
Google's first-party tool for search queries, page performance, indexing, and crawl diagnostics.
Query Google Search data, manage sitemaps, and inspect URL indexing status programmatically; requires property access.
Microsoft's first-party search performance, indexing, crawling, and keyword diagnostics.
Compare relative search interest and trends across topics, regions, and time; not a source of absolute search volume.
Google's official test for structured data eligible for rich-result features in Google Search.
Validate Schema.org JSON-LD, RDFa, and Microdata independently of Google-specific rich-result eligibility.
Google's open-source audits for page performance, accessibility, SEO, and best practices.
Run Lighthouse audits and regression assertions in continuous integration.
Established desktop crawler for technical SEO, including broken links, redirects, and metadata; the free version crawls up to 500 URLs per run.
Python library providing a comprehensive set of SEO and digital marketing tools.
SEO conprehensive analysis tool.
SEO conprehensive analysis tool.
SEO platform for technical audits, rank tracking, Core Web Vitals, uptime monitoring, and reports.
Website speed measurement tool, essential for technical SEO optimization.
Website speed measurement tool, essential for technical SEO optimization.
Website Backlink Analysis Plugins.
Open Site Explorer/MozBar Plugins.
Free SEO Tools.
A comprehensive toolbox for link building.
Track search engine rankings / Solve the problem of GA that keywords not provided.
Website Traffic View Tools.
SEO Chrome Plugin that published by semrush.
Prerender serves 100% cached versions of your web pages to search engine bots that have problems executing JavaScript.
Generative Engine Optimization (GEO) is best approached as a discoverability and measurement problem—not a replacement for SEO, nor a promise of getting cited by a model. Start with useful original content, stable URLs, accessible HTML text, descriptive titles, internal links, and structured data consistent with visible page content.
Google: AI features and your website
Google says AI Overviews and AI Mode have no additional technical requirements beyond eligibility for Google Search. Crawlability, textual content, and accurate structured data still matter.
Google: Generative AI performance reports in Search Console
Official reports for impressions and pages surfaced in generative AI features on Search and Discover; Google says the reports were available worldwide by August 31, 2026. Impressions are not visits or conversions.
Distinguishes
OAI-SearchBot(ChatGPT search discovery),GPTBot(potential model-training crawl), andChatGPT-User(certain user-triggered requests). Do not assume one robots.txt policy controls all three identically.
Anthropic: Claude crawler controls
Documents distinct
Claude-SearchBot,ClaudeBot, andClaude-Userpurposes and controls.
Manage crawler access intentionally. Robots.txt is not authentication, an authorization mechanism, or a reliable way to keep private pages secret. Verify CDN/WAF challenges and real fetch responses separately.
Accessible, semantically labeled buttons, forms, and navigation improve usability for people and can help browser-based agents interpret interactions. ARIA alone does not guarantee agent compatibility.
A voluntary proposal for machine-readable Markdown navigation and linked documentation. It may be useful for agent-facing technical docs, but is not a universally adopted search standard or a Google ranking requirement. Google explicitly says llms.txt has no positive or negative impact on Google Search visibility. Maintain it only if a specific consumer or measurable use case justifies the extra content.
Practical distinction: search crawlers, model-training crawlers, and user-directed browser/agent sessions are different actors. A bot fetch is not a human referral; browser analytics generally does not measure crawler traffic. Never bypass login, privacy, abuse prevention, or payment controls to improve purported “AI friendliness.”
robots.txt and edge security are different controls. A crawler allowed by robots.txt can still receive a Cloudflare 403/429, challenge page, or other unusable response. Likewise, allowing a crawler through the WAF does not override its robots.txt obligations. Audit both layers rather than assuming a public page is machine-accessible.
Cloudflare distinguishes Search, Agent, and Training behavior. New-domain defaults introduced September 15, 2026 keep Search allowed but restrict Training and Agent on pages with ads; mixed-purpose bots may also be blocked by training policies. Check each zone's actual settings; existing domains and configurations may differ.
Cloudflare: managed robots.txt
Cloudflare's Bot Preference Sync can prepend managed rules to an existing robots.txt, based on the zone's policies. Read the final public
https://your-domain.example/robots.txt, not just the file in Git, and check for conflicts.
Review AI Crawl Control → Crawlers / Directives for request outcomes and robots policy. On Free plans, crawler recognition is primarily user-agent based; don't treat this dashboard as independently verified attribution.
Cloudflare: bot false positives
Bot Fight Mode can challenge legitimate clients and cannot be skipped with a WAF custom rule. Super Bot Fight Mode supports targeted Skip rules. Inspect Security Events / Analytics before changing protections.
Minimal audit for a public content page:
- Decide whether you want AI search discovery, user-directed agent access, and/or training independently. For example, OpenAI documents allowing
OAI-SearchBotfor search while disallowingGPTBotfor training. - Fetch the public
/robots.txtand sitemap over your real CDN hostname. Check effective policies and Cloudflare-generated directives. Ensure public article/asset-detail URLs return meaningful text and a canonical URL, without requiring login or Turnstile. - Check Cloudflare Security Events and AI Crawl Control for actual requests from identified bots: status codes, blocked/challenged decisions, and the precise host/path. A
curl -A "OAI-SearchBot"probe checks only the declared user agent and does not prove how the authentic verified crawler is treated. - Resolve unintended challenges only for known-good traffic on public read-only paths. Do not broadly bypass protections for every user agent claiming to be a search bot; keep downloads, paid content, APIs, and write actions behind their original authorization and anti-abuse controls.
- Recheck after changes to DNS proxy/CDN, robots.txt, WAF, or bot settings. Record actual fetch responses rather than assuming a configuration toggle guarantees indexing or referrals.
The plugin to check the PV traffic of website. But the results are sometimes accurate and sometimes biased.
Check the SEO-related information of the website, such as the number of external links, the number of indexes, and SEO Internal website issues, etc.
This is a keyword tool to view monthly search volume for a number of keywords. Of course, you also can use Keyword Planner of Google Ads to view monthly search volume for a number of keywords.
The view tool for common internal issues of website.
The view tool for meta data of website.
The list mainly show SEO masters Twitter.
Brian Dean’s Blog (He has some achievements in backlink research.)
Published by Google.
High-quality blogs about SEO that published by ahrefs.
High-quality blogs about SEO that published by moz.
英文SEO实战派(Chinese Blog)
Author: John. He has amount of research on Google Authority Site.
富裕者联盟(Chinese Blog)
The author is Affiliate master, It can help you learn some online earning techniques.
Harris(Chinese Blog)
About SEO Timeliness.
SEO前线(Chinese Blog)
Author: noname. He main research Baidu SEO.
Collected by ahrefs.
If you want to recommend other good communities, please commit PR.
Measure discovery → qualified visits → meaningful product use → revenue. Different tools answer different questions; page views, bot requests, AI citations, signups, and paid customers are not interchangeable.
| Question | Practical source |
|---|---|
| Which search queries and pages receive impressions and clicks? | Search Console; Bing Webmaster Tools |
| Is a page shown inside Google's generative AI search features? | Search Console generative AI performance reports (visibility/impressions) |
| Which users arrived through a browser referral or tagged link? | GA4, Umami, or Mixpanel (referrer and UTM) |
| Did visitors complete a valuable action and return? | Product-specific events, funnels, and retention |
| Was the growth worth its cost? | Verified sales/refunds and direct operating/production costs |
Google Analytics 4: traffic acquisition
Analyze traffic sources, sessions, and key events. It complements Search Console; it does not replace search impression or crawler-log data.
Established product analytics for conversion steps, usage, and retention after the visitor reaches the product.
Open-source web and product analytics with referrers, UTM parameters, custom events, funnels, and attribution reporting; can be self-hosted.
First-click and last-click attribution for chosen page views or events; these are reporting models, not proof of causal impact.
- For ChatGPT-originated visits, inspect observed referrers and UTM parameters (for example,
utm_source=chatgpt.comwhen present). Do not assume all ChatGPT or other AI referrals are tagged, or classify all direct traffic as AI-driven. OpenAI's current crawler documentation describes bot discovery—not how every browser referral is attributed. - Server/CDN logs can show crawler requests, but those are not user sessions or citations. GSC AI impressions likewise do not establish click-through or revenue.
- Define meaningful events per product (e.g., a verified artifact download, completed research workflow, or purchase). Deduplicate and validate events, document attribution limitations, and separate gross revenue from costs.
- Choose analytics according to the question and existing stack; do not install all three trackers just because they are listed.
Organic search traffic creates opportunities, not revenue by itself. Common models include advertising, affiliate commissions, and selling products or memberships.
flowchart TB
pv["Organic page views"] --> ads["Display advertising"]
pv --> affiliate["Affiliate commissions"]
pv --> products["Products and memberships"]
ads --> revenue["Total revenue"]
affiliate --> revenue
products --> revenue
revenue --> rpm["Blended RPM"]
pv --> rpm
Blended RPM = total revenue ÷ page views × 1,000. This measures revenue per 1,000 page views across the channels shown; it is different from an ad platform's ad-only RPM. Track conversions and operating costs separately to understand profitability. Results vary by niche, geography, audience, and product fit—historical CPM/CPC benchmarks are not universal assumptions.
Google Algorithm Update History
Currently, Google has some of more important Google algorithms including the Panda algorithm, the Penguin algorithm, and the Hummingbird algorithm. This is the page that Moz tracks updates to Google's search algorithm.
Some reports on important statistics related to HTTP on the Internet, published by httparchive.
Maintained by i365, an agent-native digital product studio. This list is an independent reference collection; inclusion does not imply endorsement, partnership, or payment.