Technical SEO: Crawlability, Core Web Vitals, and Index Control
Technical SEO is the foundation: if crawlers cannot reach, render, and understand your pages efficiently, all content work fizzles out.
What you will learn
- How robots.txt, XML sitemap, and canonicals work together
- The three Core Web Vitals and their most important levers
- When noindex, canonical, or 301 is the right tool
- How structured data (Schema.org) enables rich results
Crawl control: robots.txt, sitemap, canonicals
- robots.txt: controls which areas crawlers may access. Caution: it blocks crawling, not indexing — a blocked URL can still end up in the index.
- XML sitemap: your invitation list to Google — all indexable URLs, kept up to date, submitted in Search Console.
- Canonical tag: for similar or duplicate content, it names the main version and thereby bundles the ranking signals onto one URL.
Core Web Vitals: performance made measurable
Metric | Measures | Target value | Most common lever |
|---|---|---|---|
LCP | Loading time of the largest element | < 2.5 s | Optimize images, server response time |
INP | Response time to interactions | < 200 ms | Reduce JavaScript |
CLS | Visual stability during loading | < 0.1 | Reserve dimensions for images/embeds |
Measurement uses real user data (CrUX). PageSpeed Insights combines this field data with lab tests — always optimize the field data first, because only it feeds into the evaluation.
Index control: choosing the right tool
- noindex: the page should be reachable but not appear in the search results (e.g., internal search results, filter pages).
- Canonical: several similar URLs exist deliberately — the signals should be bundled on one main version.
- 301 redirect: a URL has moved permanently — users and ranking signals migrate to the new address.
Structured data
Schema.org markup (JSON-LD) makes content machine-readable: articles, FAQs, products, reviews, organizations. The reward is rich results — enhanced search results with more surface area and a higher click-through rate — and better usability of your content by AI systems.
Achtung, klassischer Fehler: Eine per robots.txt blockierte Seite kann trotzdem indexiert werden (ohne Inhalt, als „leeres" Ergebnis). Wer eine Seite aus dem Index halten will, braucht noindex — und dafür muss die Seite crawlbar sein.
Quiz
Test your knowledge
A short quiz on the chapter — the explanations reinforce what you have learned.
Question 1 of 4
You want a page to be guaranteed not to appear in the search results. What is the right way?