While basics of SEO like the most efficient ways to build links to drive search engine rankings have changed in recent years (and content marketing has become increasingly important) what many people would think of as more “traditional SEO” is still incredibly valuable in generating traffic from search engines. As we’ve already discussed, keyword research is still valuable, and technical SEO issues that keep Google and other search engines from understanding and ranking sites’ content are still prevalent. Technical SEO for larger, more complicated sites is really its own discipline, but there are some common mistakes and issues that most sites face that even smaller to mid-sized businesses can benefit from being aware of:
Site Crawlability / XML Sitemap –
Search engines “crawl” websites, going from one page to another incredibly quickly, acting like hyperactive speed-readers. They make copies of your pages that get stored in what’s called an “index,” which is like a massive book of the web. When someone searches, the search engine flips through this big book, finds all the relevant pages and then picks out what it thinks are the very best ones to show first. To be found, you have to be in the book. To be in the book, you have to be crawled. Most sites generally don’t have crawling issues, but there are things that can cause problems. For example, JavaScript or Flash can potentially hide links; making the pages those links lead to hidden from search engines. And both can potentially cause the actual words on pages to be hidden. Each site is given a crawl budget, an approximate amount of time or pages a search engine will crawl each day, based on the relative trust and authority of a site. Larger sites may seek to improve their crawl efficiency to ensure that the ‘right’ pages are being crawled more often. The use of robots.txt, internal link structures and specifically telling search engines not to crawl pages with certain URL parameters can all improve crawl efficiency. However, for most, crawl problems can be easily avoided. In addition, it’s good practice to use sitemaps, both HTML and XML, to make it easy for search engines to crawl your site. You’ll find more about sitemaps and dealing with potential crawling issues in the Search Engine Land categories below. XML sitemaps can help Google and Bing understand your site and find all of its content. Just be sure not to include pages that aren’t useful, and know that submitting a page to a search engine in a sitemap doesn’t insure that the page will actually rank for anything. There are a number of free tools to generate XML sitemaps.
Site Structure
Information architecture refers to how you organize the pages on your website. The way that you organize your website and interlink between your pages can impact how various content on your site ranks in response to searches. The reason for this is that search engines largely perceive links as “votes of confidence” and a means to help understand both what a page is about, and how important it is (and how trusted it should be). Search engines also look at the actual text you use to link to pages, called anchor text – using descriptive text to link to a page on your site helps Google understand what that page is about (but in a post-Penguin world especially, be sure not to be overly aggressive in cramming your keywords into linking text). Information architecture can be an extremely complex topic – particularly for larger sites – and there are a number of great additional resources below with more specific answers listed at the end of this section, but at a high level the most important things to keep in mind are: You want to understand your most linked-to pages (use tools like Ahrefs, Majestic SEO, or Moz and look at “top pages” reports to determine these). Keep your most important search pages (the pages you are using to target your most valuable keywords) “high up” in your information architecture: this means linking to them often in navigation elements and linking to them whenever possible from your most linked-to pages (e.g., make sure your home page and your site’s version of our hit snow study are linking to the most valuable pages on your site from a search perspective – your “money pages”). In general you want to have a “flat information architecture” for your site – meaning that you keep any pages that you want to have rank in search engines as few clicks as possible from your home page and most linked-to pages. See this older video for a more in-depth explanation of how to flatten your site’s structure.
Duplication / Canonicalization
Sometimes that big book, the search index, gets messy. Flipping through it, a search engine might find page after page after page of what looks like virtually the same content, making it more difficult for it to figure out which of those many pages it should return for a given search. This is not good. It gets even worse if people are actively linking to different versions of the same page. Those links, an indicator of trust and authority, are suddenly split between those versions. The result is a distorted (and lower) perception of the true value users have assigned that page. That’s why canonicalization is so important. You only want one version of a page to be available to search engines. There are many ways duplicate versions of a page can creep into existence. A site may have www and non-www versions of the site instead of redirecting one to the other. An e-commerce site may allow search engines to index their paginated pages. But no one is search for “page 9 red dresses”. Or filtering parameters might be appended to a URL, making it look (to a search engine) like a different page. For as many ways as there are to create URL bloat inadvertently, there are ways to address it. Proper implementation of 301 redirects, the use of rel=canonical tags, managing URL parameters and effective pagination strategies can all help ensure you’re running a tight ship. Thin and duplicated content is another area of emphasis with Google’s recent Panda updates. By duplicating content (putting the same or near-identical content on multiple pages), you’re diluting link equity between two pages instead of concentrating it on one page, giving you less of a chance of ranking for competitive phrases with sites that are consolidating their link equity into a single document. Having large quantities of duplicated content makes your site look like it is cluttered with lower-quality (and possibly manipulative) content in the eyes of search engines. There are a number of things that can cause duplicate or thin content. These problems can be difficult to diagnose, but you can look at Webmaster Tools under Search Appearance > HTML Improvements to get a quick diagnosis.
Mobile Friendliness –
More Google searches happen on mobile devices than on desktop. Given this, it’s no wonder that Google is rewarding sites that are mobile friendly with a chance of better rankings on mobile searches while those that aren’t might have a harder time appearing. Bing, too, is doing the same. So get your site mobile friendly. You’ll increase your chance of success with search rankings as making your mobile visitors happy. In addition, if you have an app, consider making use of app indexing and linking, which both search engines offer. If your site is driving (or could be driving) significant search engine traffic from mobile searches, how “mobile friendly” your site is will impact your rankings on mobile devices, which is a fast-growing segment. In some niches, mobile traffic already outweighs desktop traffic. Google recently announced an algorithm update focused on this specifically. You can find out more about how to see what kind of mobile search engine traffic is coming to your site along with some specific recommendations for things to update in my recent post, and here again Google offers a very helpful free tool to get recommendations on how to make your site more mobile-friendly.
Page Speed –
Google wants to make the web a faster place and has declared that speedy sites get a small ranking advantage over slower sites. However, making your site blistering fast isn’t a guaranteed express ride to the top of search results. Speed is a minor factor that impacts just 1 in 100 queries according to Google. But speed can reinforce other factors and may actually improve others. We’re an impatient bunch of folks these days, especially when we’re on our mobile devices! So engagement (and conversion) on a site may improve based on a speedy load time. Speed up your site! Search engines and humans will both appreciate it. Search engines are placing an increasing emphasis on having fast-loading sites – the good news is this is not only beneficial for search engines, but also for your users and your site’s conversion rates. Google has actually created a useful tool here to give you some specific suggestions on what to change on your site to address page speed issues.
Redirects –
Improperly implementing redirects on your site can have a serious impact on search results. Whenever you can avoid it, you want to keep from moving your site’s content from one URL to another; in other words: if your content is on example.com/page, and that page is getting search engine traffic, you want to avoid moving all of the content to example.com/different-url/newpage.html, unless there is an extremely strong business reason that would outweigh a possible short-term or even long-term loss in search engine traffic. If you do need to move content, you want to make sure that you implement permanent (or 301) redirects for content that is moving permanently, as temporary (or 302) redirects (which are frequently used by developers) indicate to Google that the move may not be permanent, and that they shouldn’t move all of the link equity and ranking power to the new URL. (Further, changing your URL structure could create broken links, hurting your referral traffic streams and making it difficult for visitors to navigate your site.)
HTTPS / Secure Site –
Google would like to see the entire web running HTTPS servers, in order to provide better security to web surfers. To help make this happen, it rewards sites that use HTTPS with a small ranking boost. As with the site speed boost, this is just one of many factors Google uses when deciding if a web page should rank well. It alone doesn’t guarantee getting into the top results.
Robots.txt, Meta No Index, & Meta No Follow –
Finally, you can indicate to search engines how you want them to handle certain content on your site (for instance if you’d like them not to crawl a specific section of your site) in a robots.txt file. This file likely already exists for your site at yoursite.com/robots.txt. You want to make sure this file isn’t currently blocking anything you’d want a search engine to find from being added to their index, and you also can use the robots file to keep things like staging servers or swaths of thin or duplicate content that are valuable for internal use or customers from being indexed by search engines. You can use the meta noindex and meta no follow tags for similar purposes, though each functions differently from one another.