Checking on the status of your website in Google Search Console (formerly Webmaster Tools) is a best practice for all businesses and individuals with websites. This valuable tool makes it easy for you to ensure Google can view your website properly and that there aren’t any major errors present within your website structure. But no website is absolutely perfect, and one of the commonly seen URL error types is known as an access denied error.
What is an Access Denied Error?
The way that Google is able to discover all the content that exists within your website is by following links from one page to the next. Googlebot will continue to follow the links until all of the pages have been crawled, which then allows them to be indexed. However, an access denied error means exactly that – Googlebot was unable to properly access a page. You absolutely want Google to see all of your pages, so the good news is that this error is relatively easy to correct.
How Do You Fix an Access Denied Error?
In order to remedy an access denied error, there are a few questions you need to ask yourself.
- Does the URL with an error require a login? If a username and/or password are required to view all or some of the content on a page, then Googlebot will not be able to access it. If you want Google to crawl and index the page, then you must remove the login requirement.
- Is your robots.txt file blocking Google? Implementing a robots.txt file incorrectly may prohibit Googlebot from accessing important URLs or even your entire website. While you can still use this feature to provide Google with instructions on how to crawl your website, you do need to make sure it is working as expected. The Test robots.txt tool allows you to quickly and easily see exactly how Google will interpret your file.
- Is your hosting provider blocking Google? If this is the root of your problem, you may need to contact your hosting provider. Sometimes, security features or firewalls can block Google from having full access to your website. Talk with your hosting provider to find out what the issue is and how it can be resolved.