With the exception of Halloween, a spooky-looking house isn’t the most inviting place. You may be picturing a tall, rusted wrought iron gate with a “BEWARE” sign in front of a decrepit Victorian manor riddled with cobwebs. While looks may be deceiving, we’re going to stay far away.

You want content on your website to be as inviting as possible so people will find it and benefit from it. It’s crucial that before visitors can find you, search engines must be able to crawl through your site and do the same. SEMrush is showing us 11 crawlability problems and how to fix them so you can tear down those cobwebs and spruce up your site to help search engines lead visitors right to your front door.

• Pages Blocked in Robots.txt: This is where search engines look to see which pages they can and can’t crawl. Check your robots.txt file to ensure that you’re not blocking them from accessing your website. Replace “disallow” directives with “allow” to grant access.
• Nofollow links: The “nofollow” tag (<meta name=”robots” content=”nofollow”>) tells search engines not to crawl specific links. Check your site for these links and remove the tag if it’s not necessary.
• Bad Site Architecture: Put simply, organization is key. Improving your site architecture involves organizing pages in a logical hierarchy with internal links. This will help crawlers easily access all pages.
• Lack of Internal Links: Check to ensure you don’t have pages with no internal links pointing to them. A page without any links pointing to it is like an uncharted island – it’s isolated from the rest of the site and there’s no clear path to get there.
• “Noindex Tags”: Google treats “noindex” tags (<meta name=”robots” content=”noindex”>)as it does “nofollow” tags, so leaving these tags on pages you want the crawler to access will render your page off-limits to the crawler.

If you’ve noticed pages of yours aren’t showing up in Google when you’re trying to earn a top-ranking SERP position, these issues may be to blame. SEMrush lists another 6 potential problems your pages may be facing, so read on and stay diligent.

When you hop in your car and drive somewhere unfamiliar, you likely rely on directions of some sort. Perhaps Google Maps on your phone, verbal directions from a kind stranger, or maybe you look to the stars to navigate – more power to you. Some sense of direction is needed for you to arrive at your destination.

XML Sitemaps is a free tool that provides search engines with a map to navigate your website. You could leave Google to explore the pages of your site on its own, but this tool creates a simple map formatted specifically for search engines so you can lend site crawlers a helping hand. Just enter your site’s URL and give it a whirl.

Think about all the blood, sweat, and tears that went into your most ambitious marketing campaign. Okay, maybe that’s a little dramatic, but at the very least you probably spent a pretty significant chunk of time on it. The payoff comes when you see all the impressions, clicks, sales, etc., However, things are rarely ever guaranteed.

Amazon shows us the amount of coordination, expertise, and dedication that can go into an advertising campaign only for it to fall flat. Where one may just see a tree – the professionals in this video see their worst nightmare (again, probably a little dramatic, but we’ll let you decide).