Topic 10: Technical SEO

📖 5 min read · 🎯 beginner · 🧭 Prerequisites: keyword-analysis, on-page-seo

Why this matters

Here's the thing — you can write brilliant content and nail your keywords, but if your website has broken links, slow load times, or pages Google can't even read, none of that work shows up in search results. Technical SEO is the behind-the-scenes foundation that makes everything else possible. It's like having a shop with great products but a locked front door and no address sign. In this lesson, we're going to look at what search engine crawlers actually need from your site — and how to make sure nothing is blocking them from finding and ranking your pages.

What You'll Learn

• What Technical SEO is and why it differs from On-Page SEO
• How crawling and indexing work, and how to control them with robots.txt and XML sitemaps
• How site structure, mobile-friendliness, and page speed affect search rankings
• Advanced techniques: HTTPS, structured data, canonical tags, and error handling
• Which tools to use for ongoing Technical SEO monitoring and auditing

The Analogy

Think of your website as a public art gallery. On-Page SEO is how you arrange the paintings, write the plaques, and train the staff to welcome visitors. Technical SEO, by contrast, is everything visitors never see: the lighting circuits that illuminate every room, the accessible ramps that let everyone enter, the catalogue system that tells the city archivist exactly which pieces you hold and where to find them. A gallery with breathtaking art but no working catalogue, broken elevators, and a locked back entrance will never show up on the city's cultural map — no matter how good the paintings are.

Chapter 1: The Hidden Gears — Understanding Technical SEO

Technical SEO refers to optimizing your website's technical aspects to improve its visibility and performance in search engines. Unlike On-Page SEO, which focuses on content and user experience, Technical SEO ensures that search engines can efficiently crawl, index, and understand your website.

Example: If your website is an online art gallery, Technical SEO ensures that search engines can easily discover and showcase your art pieces to potential buyers — even before a single visitor types a search query.

Chapter 2: Building the Framework — Key Components of Technical SEO

1. Crawling and Indexing: The Pathfinders

Crawling is the process by which search engines discover your web pages. Indexing is how they store and organize that information so it can be returned in search results.

Tools:

• robots.txt — A plain-text file that guides search engine crawlers on which pages to crawl and which to avoid.
• XML Sitemap — A roadmap of your website that helps search engines find and index all your pages.

A minimal robots.txt looks like this:

User-agent: *
Disallow: /admin/
Disallow: /private/
Allow: /

Sitemap: https://www.example.com/sitemap.xml

A minimal XML sitemap entry looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://www.example.com/paintings/impressionism/</loc>
    <lastmod>2024-11-01</lastmod>
    <changefreq>weekly</changefreq>
    <priority>0.8</priority>
  </url>
</urlset>

Example: Ensure your robots.txt allows search engines to crawl your main gallery pages while blocking irrelevant admin routes, and create an XML sitemap that includes all your art collections.

2. Site Structure: The Blueprint

A well-organized site structure helps search engines understand the hierarchy and relationship between your pages — and helps users navigate without getting lost.

Tip: Use a logical, hierarchical URL structure. Categories and subcategories should be clearly defined.

# Good hierarchical URL structure
https://gallery.example.com/paintings/impressionism/monet-water-lilies/
https://gallery.example.com/sculptures/modern/rodin-thinker/
https://gallery.example.com/photographs/documentary/

# Avoid flat, meaningless URLs
https://gallery.example.com/page?id=4821

Example: Organize your gallery into top-level sections like "Paintings," "Sculptures," and "Photographs," with subcategories for different styles or periods — creating a clear breadcrumb trail for both bots and humans.

3. Mobile-Friendliness: The Adaptability

With the majority of web traffic now coming from mobile devices, ensuring your site is mobile-friendly is critical for both user experience and search rankings. Google uses mobile-first indexing, meaning it primarily uses the mobile version of your site for ranking.

Tool: Use Google's Mobile-Friendly Test to check and improve your site's mobile compatibility.

Technique: Implement responsive design so your layout adapts fluidly to any screen size:

/* Responsive image example */
img {
  max-width: 100%;
  height: auto;
}

/* Responsive grid */
@media (max-width: 768px) {
  .gallery-grid {
    grid-template-columns: 1fr;
  }
}

Example: Apply responsive design techniques to ensure your art gallery website looks and functions well on smartphones and tablets — visitors browsing on a phone should never have to pinch-zoom to read a price or artist name.

4. Page Speed: The Efficiency

A fast-loading website provides a better user experience and is favored by search engines. Google's Core Web Vitals — including Largest Contentful Paint (LCP), First Input Delay (FID), and Cumulative Layout Shift (CLS) — are directly tied to page speed.

Techniques:

• Optimize images and videos (compress without sacrificing visual quality)
• Minimize CSS, JavaScript, and HTML (remove whitespace, comments, unused code)
• Enable browser caching (return visitors load pages faster)
• Use a Content Delivery Network (CDN) to serve assets from servers geographically close to the user

Tool: Google PageSpeed Insights analyzes your page and provides specific, actionable recommendations.

Example: Compress high-resolution images of your artwork to reduce load times without compromising quality. A painting that takes 8 seconds to load will cost you both visitors and rankings.

Chapter 3: Enhancing the Mechanisms — Advanced Technical SEO Techniques

1. HTTPS: The Security

Secure your website with HTTPS to ensure data integrity and protect user privacy. Search engines explicitly prioritize secure sites, and browsers actively warn users when a site is unencrypted.

Tip: Obtain an SSL/TLS certificate for your website (free options include Let's Encrypt) and ensure all pages are served over HTTPS. Redirect all HTTP traffic to HTTPS with a permanent redirect.

# Nginx redirect example — force HTTPS
server {
    listen 80;
    server_name gallery.example.com;
    return 301 https://$host$request_uri;
}

Example: A secure padlock symbol in the browser bar assures visitors that their transactions and personal data are safe — especially important if your gallery processes purchases online.

2. Structured Data: The Clarity

Structured data (also called schema markup) helps search engines understand the content and context of your pages, enhancing how your site appears in search results through rich snippets — star ratings, prices, event dates, and more.

Tool: Use Google's Structured Data Markup Helper to generate schema markup for your pages.

Example: For your art pieces, use JSON-LD schema to provide detailed information such as artist name, creation date, medium, and price:

<script type="application/ld+json">
{
  "@context": "https://schema.org/",
  "@type": "VisualArtwork",
  "name": "Water Lilies No. 12",
  "creator": {
    "@type": "Person",
    "name": "Claude Monet"
  },
  "dateCreated": "1906",
  "artMedium": "Oil on canvas",
  "offers": {
    "@type": "Offer",
    "price": "4500.00",
    "priceCurrency": "USD"
  }
}
</script>

This can result in rich snippets in search results that make your listing stand out from plain blue links.

3. Canonical Tags: The Direction

Canonical tags prevent duplicate content issues by telling search engines which version of a webpage is the authoritative one. Without them, search engines may split ranking signals across multiple near-identical URLs.

Tip: Use canonical tags to indicate the main version of a page when you have multiple pages with similar or identical content.

<!-- On a duplicate or similar page, point to the canonical original -->
<link rel="canonical" href="https://gallery.example.com/paintings/water-lilies/" />

Example: If you have multiple pages showcasing the same artwork in different contexts (exhibitions, sales, artist spotlights), use a canonical tag to point to the primary product page so all ranking authority consolidates there.

4. Error Handling: The Maintenance

Errors — broken links, missing pages, moved content — damage both user experience and crawl efficiency. Handling them gracefully keeps your site healthy in the eyes of search engines and visitors alike.

Tip: Set up custom 404 pages that guide users back to your main content, and use 301 redirects for permanently moved pages so link equity transfers to the new URL.

# 301 redirect for a permanently moved page
rewrite ^/old-gallery-page$ /paintings/impressionism/ permanent;

Example: Create a custom 404 page for your gallery that includes navigation links to popular sections like "Current Exhibitions" and "Top-Selling Artworks" — turning a dead end into a guided tour.

Chapter 4: Continuous Monitoring and Improvement — The Ongoing Task

Technical SEO is not a one-time setup. It is an ongoing discipline. Pages get moved, code gets updated, new errors creep in, and search engine algorithms evolve. Regularly monitoring your site's health and making necessary adjustments is how you stay ahead.

Tools:

• Google Search Console — Monitor your site's health, fix crawl issues, submit sitemaps, and analyze search performance. Free and essential.
• Ahrefs — Track backlinks, perform comprehensive site audits, identify broken links and redirect chains.
• Screaming Frog — Desktop crawler that mirrors how Googlebot sees your site; surfaces broken links, duplicate content, missing tags, and redirect loops.

flowchart TD
    A[Schedule SEO Audit] --> B[Crawl Site with Screaming Frog]
    B --> C{Issues Found?}
    C -- Yes --> D[Fix: Broken Links / Redirects / Missing Tags]
    C -- No --> E[Check Google Search Console]
    D --> E
    E --> F{Errors or Coverage Issues?}
    F -- Yes --> G[Resolve & Resubmit Sitemap]
    F -- No --> H[Review Core Web Vitals in PageSpeed Insights]
    G --> H
    H --> I[Document Changes & Set Next Audit Date]

Activity: Schedule regular Technical SEO audits — monthly for active sites, quarterly for stable ones — to identify and resolve issues before they compound into ranking drops.

🧪 Try It Yourself

Task: Run a Technical SEO audit on any website you own or manage (even a simple personal site works).

1. Submit your URL to Google PageSpeed Insights and note your mobile and desktop scores.
2. Open your browser's developer tools, go to the Network tab, reload the page, and identify the three largest files being loaded.
3. Check whether your site has a robots.txt file by visiting yourdomain.com/robots.txt in the browser.
4. Check whether your site has an XML sitemap at yourdomain.com/sitemap.xml.

Success criterion: You can name at least one concrete improvement suggested by PageSpeed Insights and confirm whether your robots.txt and sitemap exist. Bonus: add a canonical tag to one of your pages and verify it renders correctly using "View Page Source."

Starter snippet — add a canonical tag to any HTML page:

<head>
  <!-- Add this inside <head>, replacing the href with your page's canonical URL -->
  <link rel="canonical" href="https://www.yourdomain.com/your-page-slug/" />
</head>

🔍 Checkpoint Quiz

Q1. What is the primary difference between On-Page SEO and Technical SEO?

A) On-Page SEO targets mobile users; Technical SEO targets desktop users B) On-Page SEO focuses on content and user experience; Technical SEO focuses on infrastructure that helps search engines crawl and index the site C) Technical SEO is only relevant for e-commerce sites D) On-Page SEO uses code; Technical SEO uses only written content

Q2. A developer adds the following to robots.txt. What does it do?

User-agent: *
Disallow: /checkout/

A) Blocks all pages from being indexed B) Prevents search engine crawlers from crawling any URL that begins with /checkout/ C) Redirects users from /checkout/ to the homepage D) Marks /checkout/ as the canonical version of the site

Q3. Your analytics show that a popular product page has very high bounce rates on mobile. Which Technical SEO area should you investigate first, and what tool would you use?

A) Structured data — check with Google's Structured Data Markup Helper B) Mobile-friendliness and page speed — check with Google's Mobile-Friendly Test and PageSpeed Insights C) Canonical tags — check with Screaming Frog D) XML sitemap — check with Google Search Console

Q4. What problem do canonical tags solve?

A) They speed up page load times by caching assets B) They tell search engines which is the preferred version of a page when duplicate or near-duplicate content exists across multiple URLs C) They encrypt data transmitted between the browser and server D) They define the structured data schema for a page's content

A1. B — On-Page SEO is about content and user-facing elements; Technical SEO is the underlying infrastructure that makes a site discoverable and crawlable by search engine bots.

A2. B — The Disallow: /checkout/ directive tells all crawlers (*) not to crawl any URL path beginning with /checkout/. It does not affect users or set redirects.

A3. B — High mobile bounce rates often signal a poor mobile experience or slow load times. Google's Mobile-Friendly Test checks responsiveness; PageSpeed Insights surfaces load-time bottlenecks. Both are the correct first stops before investigating other Technical SEO factors.

A4. B — Canonical tags consolidate ranking signals when the same (or very similar) content lives at multiple URLs, preventing search engines from splitting authority across duplicates and ensuring the preferred URL is indexed.

🪞 Recap

• Technical SEO optimizes the infrastructure of your site so search engines can efficiently crawl, index, and rank it — independent of content quality.
• robots.txt controls which pages crawlers access; XML sitemaps guide them to every page you want indexed.
• Mobile-friendliness and page speed are ranking factors; Google PageSpeed Insights and the Mobile-Friendly Test are your primary diagnostic tools.
• HTTPS, structured data (schema markup), canonical tags, and proper error handling (404 pages, 301 redirects) are the advanced layer that separates good Technical SEO from great.
• Technical SEO is an ongoing discipline: use Google Search Console, Ahrefs, and Screaming Frog to audit regularly and catch issues before they cost you rankings.

📚 Further Reading

• Google Search Central — Technical SEO Basics — the authoritative starting point from Google itself
• Google PageSpeed Insights — free tool to measure and improve Core Web Vitals and page speed
• Schema.org Full Vocabulary — complete reference for structured data types including VisualArtwork, Product, Event, and more
• Screaming Frog SEO Spider — desktop crawler used by professional SEOs for site-wide technical audits
• Ahrefs Site Audit Guide — deep-dive walkthrough on using Ahrefs to find and fix technical issues at scale
• ⬅️ Previous: On-Page SEO
• ➡️ Next: Off-Page SEO