Technical SEO – what it is and how to optimize your site

Google may never see your site, even if you have great content. Technical SEO is the layer that decides whether a crawler reaches your content at all, understands it, and decides it’s worth showing to users. I’ll show you what technical optimization is, what it consists of, and how to check that your site isn’t blocking itself.

What is technical SEO

Technical SEO is a set of optimization work concerned with a website’s infrastructure – not the content, not external links, but the architecture itself and how the site communicates with search engine crawlers. The goal is for Google to be able to reach every page efficiently, render it correctly, understand its context, and decide whether it’s worth indexing.

Put simply, content SEO answers what the page says, link building builds its authority, and technical SEO makes sure Google can even see and assess it in the first place. Those are the three pillars of search engine optimization – and technical SEO is where it all starts.

You can have an article perfectly matched to user intent, covered in keywords, with sensible headings, and still not rank, because robots.txt is blocking indexing, the page takes 9 seconds to load, or the mobile version is falling apart. I’ve seen this in audits dozens of times.

What technical SEO consists of

Technical SEO covers several large areas. Each one could fill its own article – and I will write those – but here I want to give you an overview of the whole picture, so you understand how these elements connect.

Indexing and crawling

Google’s crawler (Googlebot) scans the web by following links. When it reaches your site, it moves through the URLs and decides which pages are worth indexing.

How do we know this?

That information is documented in Google patent US20120078874A1 – Search Engine Indexing.

To influence that, you have two basic tools:

The robots.txt file – a text file in the domain’s root directory that tells search engine crawlers what not to scan. It can block entire sections, specific files, or particular URL parameters. A mistake in robots.txt can block an entire site from being indexed – with no error message shown in the browser at all. The user sees the page normally, Google sees nothing.

The XML sitemap – a file listing the URLs you want indexed. Think of it as a table of contents for Googlebot. It doesn’t guarantee indexing, but it makes it easier for the crawler to find every page, especially ones with no internal links pointing to them.

You can check both these files using Google Search Console. That’s the basic tool for anyone doing technical SEO – free, straight from Google, and it shows indexing errors, index coverage, mobile usability problems and much more.

URL structure and site architecture

URLs aren’t just about aesthetics. A logical URL structure helps crawlers understand a site’s hierarchy and the relationships between pages.

A good URL looks like this: digispot.pl/blog/technical-seo/

Not like this: digispot.pl/index.php?id=1247&cat=3&lang=en

Beyond the format itself, information architecture matters too – how pages link to each other, how deeply they’re buried (how many clicks from the homepage), and whether there are any orphan pages with no internal links at all.

Orphan pages are pages Google may never reach, because no other page links to them. Internal linking also decides how ranking signals flow through the site. A page buried at the fourth level of the structure, reached by a single link from page 4 of pagination, gets treated by Google completely differently from one linked from the main navigation and several related articles.

Internal linking is one of the underrated elements of technical SEO with a real effect on search visibility.

Meta tags – title, description and robot directives

Meta tags are HTML code fragments the user doesn’t see directly on the page, but does see in search results.

Title and description are the two most important tags for CTR. The title appears as the blue heading in Google results, the description as the gray text underneath. Meta description isn’t a direct ranking factor, but it genuinely affects clicks, which could put it among the behavioral factors.

I’ve lost count of how many times I’ve seen duplicate titles and meta descriptions across hundreds of pages – especially in online stores, where templates generate identical titles for entire product categories.

Meta robots – tells Googlebot whether it can index the page (index) and follow its links (follow), or not – whether it always respects that is a topic for another post 🙂

The canonical tag (rel=”canonical”) – indicates which version of a page is the “original”. Useful when the same content is available at several URLs (a www and non-www version, HTTP and HTTPS, or filter-parameter pages in an online store, say). Without canonicals, Google may treat the content as duplicate and lower its visibility.

These are three distinct mechanisms that often get confused. Robots.txt says “don’t scan this”, meta robots says “don’t index this”, and canonical says “treat this as a copy of that”. Each does something different.

Page speed and Core Web Vitals

Google has used Core Web Vitals as a ranking signal since 2021. These are three metrics that measure the user experience while a page loads:

• LCP (Largest Contentful Paint) – how fast the largest visible element on screen loads (usually a hero image or the main heading). A good score is under 2.5 seconds.
• CLS (Cumulative Layout Shift) – whether page elements jump around while loading (buttons that shift before you can click them, say). A good score is under 0.1.
• INP (Interaction to Next Paint) – how fast the page responds to clicks and user interaction. It replaced FID in 2024.

In practice, the biggest LCP problems show up in online stores, and not only on the homepage. Core Web Vitals have to be met on every page. The homepage often performs fine, because it’s polished and light. The problem starts on category pages with dozens of product thumbnails, on product pages with a high-resolution gallery, or on older blog posts where nobody thought about image optimization.

Check one URL from each type:

• the homepage,
• a category,
• a product,
• a blog post
• an information page.

Only that gives you a real picture of the site’s health.

Personally, I lean heavily on Google Search Console, whose “Core Web Vitals” report shows the health of every page type in one place:

JavaScript SEO

A separate matter worth knowing about, especially if your site relies heavily on JavaScript. Googlebot renders JS, but not right away. There’s something called the second wave of indexing – Google first scans the HTML, and only in a second pass does it render the JavaScript and index the dynamically generated content. That can mean an indexing delay of anywhere from a few hours to a few days.

In practice this mostly affects SPAs (Single Page Applications) and sites built on frameworks like Next.js or Nuxt, where a large share of the content is generated client-side. If your page’s source looks like an empty HTML structure with no content, Google may struggle to index it correctly. In Google Search Console you can check how Google actually sees your page using the “Inspect URL” tool and the rendered-page preview. Worth noting: this tool doesn’t show 100% exactly how Googlebot “sees” your page, but it’s the closest thing to it.

Google Search Console → Inspect any URL → View tested page

Responsiveness and mobile-first indexing

Google indexes pages mainly using their mobile version. This is called mobile-first indexing, and it’s been in effect since 2019 (for new sites) and 2023 for everyone else.

Responsiveness isn’t just “the page displays on a phone”. The mobile version should contain exactly the same content and functionality as desktop – no hidden sections, no shortened descriptions, no missing structured data.

Security – SSL certificates and HTTPS

HTTPS has been a ranking signal in Google for years – a weak one, but it’s there. More important is that browsers flag HTTP pages as “not secure”, which discourages users and lowers your conversion rate.

Migrating from HTTP to HTTPS requires correctly configuring 301 redirects from every old URL to the new one, in the right order, so you don’t end up creating a redirect chain on top of it. If the redirects are set up wrong (HTTP → HTTPS but www → non-www with no redirect, say), you can end up with duplicate content and scattered link juice.

Structured data

Structured data (schema markup) is code added to a page’s HTML that helps Google understand the content’s context. Thanks to it, search results can show enhanced information – star ratings, product prices, event dates, recipes. These are called rich snippets.

They don’t directly affect ranking, but if you’re already ranking in the top 10, rich snippets can noticeably raise your click-through rate – star ratings, a product price or an article date make a result stand out from the competition. It’s also worth remembering that structured data isn’t used only by Google’s crawlers – AI Overview and other AI systems use schema to understand a page’s context when generating answers. Schema is also an invaluable addition for local SEO, which I’ll write more about in another article.

How to check your site’s technical health

You don’t need a full audit right away. Start with a few things you can check within an hour:

Google Search Console – the “Indexing” → “Pages” tab. Check how many pages are indexed and how many aren’t, and why.

PageSpeed Insights – paste in the URL of your homepage, a category page and a product page, choose the mobile view. Look at the Field Data (real user data), not just Lab Data.

The robots.txt file – open yoursite.com/robots.txt and read it. Check whether important paths are blocked, or in extreme cases the entire site — this happens after migrations.

Screaming Frog (free version, up to 500 URLs) – crawl the site and check the 404 and redirect tabs.

That isn’t a full technical SEO audit, but these four checks are often enough to find the most serious problems. If you already have a list of errors, fix them in this order:

• first, anything blocking indexing of valuable pages,
• then duplicate content and canonical errors,
• Core Web Vitals last.

A robots.txt error blocking half the site hurts more than an LCP of 3.2 seconds instead of 2.4.

The technical errors I see most often

A few things repeat across audits regardless of industry or site size:

Duplicate homepage versions – https://domain.com, https://www.domain.com, http://domain.com, http://www.domain.com – all four available with no redirect. Google sees four different pages with identical content.

robots.txt blocking CSS and JavaScript – An old habit from a time when nobody rendered pages. Today Google renders JavaScript – if you block access to JS and CSS files, the crawler won’t see the page the way a user does.

Canonical tags pointing to a different version than the one indexed – I’ve seen sites where the canonical pointed to a staging version or a broken URL. Google politely followed the directive and ignored the actual page.

A bad canonical on pagination pages – Every pagination page should have a self-canonical, pointing to itself, not the first page. Setting /page/2, /page/3 with a canonical to /page/1 makes Googlebot stop visiting the later pages. And those pages have links to products, which then drop out of crawling too.

No sitemap submitted in Search Console – The XML sitemap exists, but it hasn’t been submitted in Google Search Console. Google might find it on its own, but why leave it to chance when submitting a correctly generated sitemap is literally 2 clicks.

Unblocked filters in online stores – Every combination of filters (color, size, price, brand) generates a separate URL on many CMSs. With a few hundred products and a handful of filters, that turns into thousands of URLs with nearly identical content. If they aren’t blocked in robots.txt, Googlebot wastes its crawl budget browsing pages that were never going to rank anyway. The exception is filters deliberately designed for indexing, like separate pages for color variants or specific product categories. But that’s part of an SEO strategy you have to plan deliberately, rather than blindly blocking access to pages that might be important.

With the growing popularity of stores built on SaaS platforms, a lot of these problems get solved automatically – the provider ships a correct template and some of the technical details happen behind the scenes. In practice, though, custom templates can still surprise you. I’ve seen stores on popular platforms with errors that in theory shouldn’t be able to exist there.

Technical SEO is a foundation you have to keep coming back to

Technical SEO isn’t a project you check off once and forget. Sites are alive – new features get shipped, the CMS changes, developers fix things that weren’t broken. I know of situations where a project manager learned from a client conversation that the headings looked too small, and instead of changing the CSS styling a developer changed an H1 to an H2, or someone rolled out a new search based on the GET method and the store gained several thousand duplicates overnight. Something can break with every deployment, and often those things only surface months later, when for some unknown reason the store’s visibility keeps steadily dropping.

That’s why monitoring in Google Search Console and running periodic technical reviews isn’t paranoia, it’s insurance that works quietly until the day you find out how badly you need it.