feat: sitemap / robots.txt #783
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| import { createFileRoute } from '@tanstack/react-router' | ||
| import { setResponseHeader } from '@tanstack/react-start/server' | ||
| import { generateRobotsTxt, getSiteOrigin } from '~/utils/sitemap' | ||
|
|
||
| export const Route = createFileRoute('/robots.txt')({ | ||
| server: { | ||
| handlers: { | ||
| GET: async ({ request }: { request: Request }) => { | ||
| const content = generateRobotsTxt(getSiteOrigin(request)) | ||
|
|
||
| setResponseHeader('Content-Type', 'text/plain; charset=utf-8') | ||
| setResponseHeader( | ||
| 'Cache-Control', | ||
| 'public, max-age=300, must-revalidate', | ||
| ) | ||
| setResponseHeader( | ||
| 'CDN-Cache-Control', | ||
| 'max-age=3600, stale-while-revalidate=3600', | ||
| ) | ||
|
|
||
| return new Response(content) | ||
| }, | ||
| }, | ||
| }, | ||
| }) | ||
|
Comment on lines
+5
to
+25
There was a problem hiding this comment. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
echo "== Candidate static files =="
fd -a '^(robots\.txt|sitemap\.xml)$'
echo
echo "== References that may indicate static serving expectations =="
rg -n -C2 'robots\.txt|sitemap\.xml|publicDir|static' -g '!src/routeTree.gen.ts'Repository: TanStack/tanstack.com Length of output: 31805 🏁 Script executed: find . -maxdepth 2 -type f \( -name "vite.config.*" -o -name "start.config.*" -o -name "tsconfig.json" -o -name "package.json" \) -exec head -50 {} +Repository: TanStack/tanstack.com Length of output: 4407 🏁 Script executed: rg -n "publicDir|assetDir|public.*robots" --type ts --type json -A 2 -B 2 | head -100Repository: TanStack/tanstack.com Length of output: 47 🏁 Script executed: git ls-files | grep -E "(config|vite|start)" | head -20Repository: TanStack/tanstack.com Length of output: 1323 🏁 Script executed: find . -maxdepth 2 -name "netlify.toml" -o -name ".netlify.toml" | xargs cat 2>/dev/nullRepository: TanStack/tanstack.com Length of output: 1118 🏁 Script executed: rg -n "publicDir|public.*assets|robots" vite.config.tsRepository: TanStack/tanstack.com Length of output: 47 🏁 Script executed: cat -n src/routes/robots[.]txt.tsRepository: TanStack/tanstack.com Length of output: 1006 🏁 Script executed: cat -n public/robots.txtRepository: TanStack/tanstack.com Length of output: 159 🏁 Script executed: cat vite.config.ts | head -150Repository: TanStack/tanstack.com Length of output: 3588 🏁 Script executed: rg -n "publicDir|public" vite.config.ts -A 3 -B 3Repository: TanStack/tanstack.com Length of output: 47 🏁 Script executed: rg -n "generateRobotsTxt\|getSiteOrigin" src/utils/sitemap.ts -A 5 -B 2Repository: TanStack/tanstack.com Length of output: 47 🏁 Script executed: cat -n src/utils/sitemap.ts | grep -A 20 "generateRobotsTxt"Repository: TanStack/tanstack.com Length of output: 345 🏁 Script executed: cat -n src/utils/sitemap.tsRepository: TanStack/tanstack.com Length of output: 5713 Remove the static The static file hardcodes 🤖 Prompt for AI Agents |
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| import { createFileRoute } from '@tanstack/react-router' | ||
| import { setResponseHeader } from '@tanstack/react-start/server' | ||
| import { generateSitemapXml, getSiteOrigin } from '~/utils/sitemap' | ||
|
|
||
| export const Route = createFileRoute('/sitemap.xml')({ | ||
| server: { | ||
| handlers: { | ||
| GET: async ({ request }: { request: Request }) => { | ||
| const content = await generateSitemapXml(getSiteOrigin(request)) | ||
|
|
||
| setResponseHeader('Content-Type', 'application/xml; charset=utf-8') | ||
| setResponseHeader( | ||
| 'Cache-Control', | ||
| 'public, max-age=300, must-revalidate', | ||
| ) | ||
| setResponseHeader( | ||
| 'CDN-Cache-Control', | ||
| 'max-age=3600, stale-while-revalidate=3600', | ||
| ) | ||
|
|
||
| return new Response(content) | ||
| }, | ||
| }, | ||
| }, | ||
| }) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Potential duplicate robots meta tags for filtered showcase pages.
The root layout injects
<meta name="robots" content="noindex, nofollow">whenshouldIndexPathreturns false (line 183-185). However, the showcase route also injects the same meta tag viaseo({ noindex: loaderData?.hasNonCanonicalSearch })in itsheadconfig. When both conditions are true, duplicate robots tags will render since TanStack Router's<HeadContent>doesn't deduplicate meta tags by name.While search engines generally handle duplicates gracefully, this could cause HTML validation warnings and indicates overlapping responsibility.
Consider either:
🤖 Prompt for AI Agents