SiteScan Analyzer Documentation

Identifies pages returning 4xx (client errors) and 5xx (server errors) HTTP status codes, and maps which internal pages link to them.

Detects pages that return HTTP 200 but behave like error pages (soft 404s) using multi-signal scoring: reference matching, phrase detection, and statistical anomaly.

Analyzes HTTP redirect chains, detecting excessive hops, external redirects, and structural link rot from internal pages pointing to redirected URLs.

Comprehensive character encoding analysis: missing charset declarations, mismatches between HTTP and HTML charset, mojibake patterns, BOM presence, and double-encoding.

Analyzes robots.txt format, directives, and effectiveness. Checks for missing or malformed files, overly restrictive rules, sitemap presence, and crawl budget impact.

Checks viewport meta tag presence and configuration for mobile-friendly design. Detects missing viewports, disabled zoom, fixed-width layouts, and inconsistent viewport coverage.

Detects meta robots directives (noindex, nofollow) and meta refresh redirects across all pages. Flags critical issues like noindex on homepage, sitemap conflicts, and HTML-level redirects.

Per-URL analysis of HTTP response headers captured by the crawler: cookie security flags, server fingerprint, X-Frame-Options / X-Content-Type-Options / HSTS coverage across all pages, ETag inode leak, TTFB statistics. Complements 014 (homepage-only score) with site-wide coverage.

Verifies that the cookie banner (CMP) actually gates Google Tag Manager, gtag and other marketing tags: known CMP vendor detection, script order and Consent Mode v2 defaults, Cookie-Script account configuration, double-loading through GTM, site-wide coverage — plus a runtime check with a headless browser proving whether Google tag hits carry consent state.

Detects pages that return HTTP 200 but where a CMS form component failed to render: instead of the form, visitors see the form plugin's own error message (e.g. Contact Form 7 "Error: Contact form not found." in any of its official translations) or the raw shortcode text. The page loads fine for every HTTP check, but its key function — contact or download — is dead. Deterministic string matching on the crawled page text, no extra requests.

Audits the site's public Google Tag Manager container with the same checks as the GTM Audit service, so the results appear in the report, in scheduled scans, in email routing and in scan comparisons. Static checks on the published container: measurement IDs shared by several tags, legacy Universal Analytics, paused tags, Custom HTML, community templates, tags with no trigger, consent defaults, dynamic IDs. Runtime checks: the homepage is loaded in a headless browser with consent denied, then again with consent granted, to see requests to Google and other trackers before consent, consent signals granted by default, undeclared measurement IDs, extra containers and cookies written before consent. Findings that need access to the container through the Google Tag Manager API (A codes) come from the latest saved GTM Audit with API access. Runs in every scan; a site without a GTM container is simply ok.

Validates H1-H6 heading hierarchy, checking for missing H1, multiple H1 tags, and heading level skips that harm SEO and accessibility.

Finds pages sharing identical H1 headings, title tags or meta descriptions, which dilutes SEO value and confuses search engines.

Checks heading text length for SEO optimization. Headings that are too short provide little SEO value, while overly long headings may be truncated.

Validates HTML lang attributes, canonical URLs, and hreflang implementations for multilingual SEO. Checks for missing, invalid, or conflicting declarations.

Validates Open Graph meta tags and Twitter Cards for social sharing. Checks required tags, content quality, consistency with page metadata, and duplicate detection.

Evaluates image ALT text coverage and quality. Detects missing ALT attributes, empty ALT text, untranslated ALT for multilingual sites, and overly long descriptions.

Validates JSON-LD structured data blocks against Schema.org specifications. Checks syntax, required fields per type, URL format, and type recognition.

Analyzes internal linking structure using BFS from homepage. Detects orphan pages, deep pages (>4 clicks), generic anchor text, and self-linking patterns.

Compares the just-generated sitemap.xml with the previous scan's snapshot. Detects URL count spikes/drops (vs 7-day baseline) and suspicious new URLs (webshell paths, spam keywords, exotic charsets). Replaces the daily sitemap watchdog: the customer's sitemap.xml is SiteScan output, so meaningful diffs only happen between consecutive scans, not day-by-day.

Verifies that the homepage's JSON-LD structured data actually describes the real business, by comparing it against the AI-derived site profile (context_llm.json). Parses the whole document (head+body), so it catches body-injected JSON-LD that 021 misses. Two layers: deterministic NAP match (name/phone/email/locality) + neutral AI semantic judgement (Gemini) that also flags moved/empty/parked sites, language drift, or a different business — without presuming the cause. Homepage-only.

Finds page titles that are nearly the same (for example the same title in two language versions with only a connector word translated, or the same title with a different separator or case), complementing 005, which only finds exact duplicates. Titles are compared word by word, never letter by letter. No extra requests, no AI. Experimental: optional until the thresholds are calibrated on real sites.

Validates all external links: detects broken links (4xx/5xx), connection failures, slow responses, HTTP-only links, and suspicious URL shorteners.

Evaluates 10 HTTP security headers using Mozilla Observatory methodology. Grades from A+ to F based on CSP, HSTS, cookie flags, CORS, X-Frame-Options, and more.

Checks the JavaScript libraries loaded by the homepage and by the error page (404) against the OSV vulnerability database. Libraries are identified by CDN URL, file name, the version banners anywhere in the file (also inside bundles that join several scripts), Retire.js content rules and fingerprints, and by the version actually in use read in the page. Reports severity (Critical/High/Medium/Low), affected versions and available fixes; libraries whose database lookup failed are listed as not verified, and files in which no library could be identified are listed as such. Scripts served and updated by their provider (advertising pixels, video players, map APIs) are listed as hosted services: they are not npm packages, so the database does not cover them. Scripts that appear only on other pages are not checked.

Detects mixed content on HTTPS sites: HTTP scripts and stylesheets (blocked by browsers), HTTP images in OG tags, HTTP canonical/internal links, protocol-relative resource URLs, and unsafe cross-origin links.

Active pentest-style server security testing: TRACE/XST, host header injection, dangerous HTTP methods, sensitive file exposure, directory listing, TLS weakness, CRLF injection, verbose errors, path traversal, backup file discovery, security.txt compliance.

Checks heading accessibility: empty headings, headings hidden via CSS, and image-only headings that screen readers cannot interpret.

Analyzes grammar, spelling, and linguistic quality across all site languages using AI (Claude/Gemini). Reports errors, warnings, and suggestions with corrected text.

Pre-launch editorial QA for staging sites, run on demand: re-reads the already-crawled corpus (no live fetch), runs six deterministic gap-checks not covered by the other analyzers, then an AI editorial synthesis produces content findings and a go-live verdict.

Browser-based homepage analysis using headless Chrome. Reports JavaScript console errors, failed network requests, third-party domains, and page load performance metrics.

Checks the downloadable files your pages link to — images, PDFs, Office documents, audio, video, fonts and archives: detects broken resources (4xx/5xx), oversized files, redirects and malformed URLs. Stylesheets and scripts are not covered by this check.