Technical SEO: Crawlability, Core Web Vitals, and Index Control
Technical SEO is the foundation: if crawlers cannot reach, render, and understand your pages efficiently, all content work fizzles out.
What you will learn
- How robots.txt, XML sitemap, and canonicals work together
- The three Core Web Vitals and their most important levers
- When noindex, canonical, or 301 is the right tool
- How structured data (Schema.org) enables rich results
Crawl control: robots.txt, sitemap, canonicals
- robots.txt: controls which areas crawlers may access. Caution: it blocks crawling, not indexing. A blocked URL can still end up in the index.
- XML sitemap: your invitation list to Google, namely all indexable URLs, kept up to date, submitted in Search Console.
- Canonical tag: for similar or duplicate content, it names the main version and thereby bundles the ranking signals onto one URL.
Core Web Vitals: performance made measurable
Metric | Measures | Target value | Most common lever |
|---|---|---|---|
LCP | Loading time of the largest element | < 2.5 s | Optimize images, server response time |
INP | Response time to interactions | < 200 ms | Reduce JavaScript |
CLS | Visual stability during loading | < 0.1 | Reserve dimensions for images/embeds |
Measurement uses real user data (CrUX). PageSpeed Insights combines this field data with lab tests. Always optimize the field data first, because only it feeds into the evaluation.
Index control: choosing the right tool
- noindex: the page should be reachable but not appear in the search results (e.g., internal search results, filter pages).
- Canonical: several similar URLs exist deliberately, since the signals should be bundled on one main version.
- 301 redirect: a URL has moved permanently. Users and ranking signals migrate to the new address.
Structured data
Schema.org markup (JSON-LD) makes content machine-readable: articles, FAQs, products, reviews, organizations. The reward is rich results (enhanced search results with more surface area and a higher click-through rate) and better usability of your content by AI systems.
Achtung, klassischer Fehler: Eine per robots.txt blockierte Seite kann trotzdem indexiert werden (ohne Inhalt, als „leeres" Ergebnis). Wer eine Seite aus dem Index halten will, braucht noindex, und dafür muss die Seite crawlbar sein.
Quiz
Test your knowledge
A short quiz on the chapter, the explanations reinforce what you have learned.
Question 1 of 4
You want a page to be guaranteed not to appear in the search results. What is the right way?