Skip to content

Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.

Fix sitemap double-path bug caused by mike + site_url interaction

Louis Choquel62ed147b43d2
diff --git a/Makefile b/Makefile
index 81b8e5232..e90a018e5 100644
--- a/Makefile
+++ b/Makefile
@@ -809,7 +809,7 @@ define ROOT_ROBOTS_TXT
User-agent: *
Allow: /latest/
Disallow: /
-Sitemap: https://$(SITE_DOMAIN)/latest/sitemap.xml
+Sitemap: https://$(SITE_DOMAIN)/sitemap.xml
endef
export ROOT_ROBOTS_TXT
@@ -890,9 +890,10 @@ endif
cp docs/404.html "$$TMPDIR/404.html" && \
echo "$$ROOT_ROBOTS_TXT" > "$$TMPDIR/robots.txt" && \
echo "$$ROOT_INDEX_HTML" > "$$TMPDIR/index.html" && \
+ sed 's|/$(DOCS_VERSION)/|/latest/|g' "$$TMPDIR/$(DOCS_VERSION)/sitemap.xml" > "$$TMPDIR/sitemap.xml" && \
cd "$$TMPDIR" && \
- git add 404.html robots.txt index.html && \
- (git diff --cached --quiet || git commit -m "Update root assets (404.html, robots.txt, index.html)") && \
+ git add 404.html robots.txt index.html sitemap.xml && \
+ (git diff --cached --quiet || git commit -m "Update root assets (404.html, robots.txt, index.html, sitemap.xml)") && \
git push origin gh-pages
docs-delete: export PATH := $(VIRTUAL_ENV)/bin:$(PATH)
diff --git a/docs/CLAUDE.md b/docs/CLAUDE.md
index cff5522a7..a4b5cb453 100644
--- a/docs/CLAUDE.md
+++ b/docs/CLAUDE.md
@@ -5,7 +5,7 @@ This file documents non-obvious architectural decisions for the docs site (`docs
## Site Architecture Overview
- MkDocs Material with `mike` versioning, published at `docs.pipelex.com`
-- `site_url` is `https://docs.pipelex.com/latest/` — all canonical URLs include `/latest/`
+- `site_url` is `https://docs.pipelex.com/` (bare domain, NOT `/latest/`) — mike rewrites sitemap URLs by inserting the version prefix during deployment; if `site_url` already contains `/latest/`, mike creates double paths like `latest/0.20.9/page/`. Canonical URLs are forced to `/latest/` via a template override in `docs/overrides/main.html`.
- Custom domain via `docs/CNAME` (value: `docs.pipelex.com`)
- Theme overrides live in `docs/overrides/`
@@ -38,11 +38,11 @@ There must be **NO** `docs/404.md` — a Markdown 404 would be treated as conten
## robots.txt: Two Files, Different Purposes
- **`docs/robots.txt`** — Lands at `/latest/robots.txt` inside the versioned site. Crawlers ignore it (RFC 9309: only domain-root robots.txt is authoritative). Exists as a courtesy/fallback.
-- **`Makefile` `ROOT_ROBOTS_TXT`** — The authoritative robots.txt deployed to domain root by `docs-deploy-root`. Allows only `/latest/`, disallows everything else, points sitemap to `/latest/sitemap.xml`.
+- **`Makefile` `ROOT_ROBOTS_TXT`** — The authoritative robots.txt deployed to domain root by `docs-deploy-root`. Allows only `/latest/`, disallows everything else, points sitemap to root `/sitemap.xml` (which contains `/latest/` URLs, generated by `docs-deploy-root` from the versioned sitemap via sed rewrite).
## Deployment (`docs-deploy-root`)
-The `docs-deploy-root` Makefile target deploys root assets (`404.html`, `robots.txt`, `index.html`) directly to the `gh-pages` branch via a temporary git worktree.
+The `docs-deploy-root` Makefile target deploys root assets (`404.html`, `robots.txt`, `index.html`, `sitemap.xml`) directly to the `gh-pages` branch via a temporary git worktree. The root `sitemap.xml` is generated by sed-rewriting the versioned sitemap to use `/latest/` URLs.
- Called automatically after `docs-deploy-stable` and `docs-deploy-specific-version-pre-release`
- The root `index.html` is a meta-refresh redirect to `/latest/`
diff --git a/docs/overrides/main.html b/docs/overrides/main.html
index f68e4814e..619d4cc10 100644
--- a/docs/overrides/main.html
+++ b/docs/overrides/main.html
@@ -1,9 +1,43 @@
{# overrides/main.html #}
{% extends "base.html" %}
+{# Force canonical URL to /latest/ path regardless of deployed version.
+ site_url is https://docs.pipelex.com/ (bare domain) so that mike can
+ properly prefix version paths in sitemaps. We override canonical here
+ to always point crawlers at /latest/. #}
+
+{% block site_meta %}
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width,initial-scale=1">
+ {% if page.meta and page.meta.description %}
+ <meta name="description" content="{{ page.meta.description }}">
+ {% elif config.site_description %}
+ <meta name="description" content="{{ config.site_description }}">
+ {% endif %}
+ {% if page.meta and page.meta.author %}
+ <meta name="author" content="{{ page.meta.author }}">
+ {% elif config.site_author %}
+ <meta name="author" content="{{ config.site_author }}">
+ {% endif %}
+ {% set canonical_latest = "https://docs.pipelex.com/latest/" ~ page.url %}
+ <link rel="canonical" href="{{ canonical_latest }}">
+ {% if page.previous_page %}
+ <link rel="prev" href="{{ page.previous_page.url | url }}">
+ {% endif %}
+ {% if page.next_page %}
+ <link rel="next" href="{{ page.next_page.url | url }}">
+ {% endif %}
+ {% if config.extra.alternate is iterable %}
+ {% for alt in config.extra.alternate %}
+ <link rel="alternate" href="{{ alt.link | url }}" hreflang="{{ alt.lang }}">
+ {% endfor %}
+ {% endif %}
+{% endblock %}
+
{% block extrahead %}
{{ super() }}
{% if page %}
+{% set canonical_latest = "https://docs.pipelex.com/latest/" ~ page.url %}
{% set desc = page.meta.description if page.meta and page.meta.description else config.site_description %}
{% set title = page.meta.title if page.meta and page.meta.title else page.title %}
{% set img = page.meta.image if page.meta and page.meta.image else "https://d2cinlfp2qnig1.cloudfront.net/docs/pipelex_banner_1200x630.png?v=1" %}
@@ -16,7 +50,7 @@
<meta property="og:image" content="{{ img }}">
<meta property="og:image:width" content="1200">
<meta property="og:image:height" content="630">
-<meta property="og:url" content="{{ page.canonical_url }}">
+<meta property="og:url" content="{{ canonical_latest }}">
<meta name="twitter:card" content="summary_large_image">
<meta name="twitter:title" content="{{ title }}">
<meta name="twitter:description" content="{{ desc }}">
@@ -27,7 +61,7 @@
"@context": "https://schema.org",
"@type": "WebSite",
"name": "{{ config.site_name }}",
- "url": "{{ config.site_url }}",
+ "url": "{{ canonical_latest }}",
"description": "{{ config.site_description }}"
}
</script>
@@ -38,7 +72,7 @@
"@type": "TechArticle",
"headline": "{{ title }}",
"description": "{{ desc }}",
- "url": "{{ page.canonical_url }}",
+ "url": "{{ canonical_latest }}",
"publisher": {
"@type": "Organization",
"name": "Pipelex",
diff --git a/mkdocs.yml b/mkdocs.yml
index b7a184ea2..f6aa204b0 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -1,5 +1,5 @@
site_name: Pipelex Documentation
-site_url: https://docs.pipelex.com/latest/
+site_url: https://docs.pipelex.com/
site_description: "Official documentation for Pipelex, an open-source library/CLI for building and running executable AI methods."
docs_dir: docs
repo_url: "https://github.com/Pipelex/pipelex"