What Are Soft 404s and How to Fix Them?
As a reader, have you ever stumbled upon a webpage displaying an enormous error message, blocking your path to further reading? This can be a frustrating experience as you’re left directionless and forced to abandon the page. From the website owner’s perspective, this could spell trouble. Although error pages do serve a purpose, they can become detrimental to your website if used inappropriately.
Example of an error message used in the wrong context is a “soft 404 error.” These errors can significantly damage your website’s search engine rankings, website traffic, and even credibility.
What is a Soft 404 errors?
A soft 404 error occurs when a user requests a page that cannot be found or is invalid. Instead of returning the appropriate HTTP error code (404 or 410 not found), the server mistakenly responds with an HTTP status code 200 OK (success). In simpler terms, this means that the server reports a page as valid when it is not, which leads search engines to crawl and index these pages. Consequently, these pages appear in search results despite their invalidity. Let us explore the most frequent causes of soft 404 errors. When there is nothing wrong with the page, use “404”.
The difference between Soft 404 and Hard 404 errors
The dissimilarity between the (hard) 404 and soft 404 errors is primarily in the manner by which they indicate their status to search engines. A hard 404 error transmits the 404 error code to both page visitors and search engines, signifying that the page does not exist.
On the other hand, a soft 404 error is exhibited to visitors, but not to search engines. Instead, search engines are given a 200 OK status, thereby enabling them to continue to crawl the page. Therefore, if soft 404 errors appear on the Google Search Console, it suggests that some pages on your site are possibly returning 404 errors even if they exist in reality.
Soft 404 errors usually happen when
Soft 404 errors often occur due to various reasons, one of which is having pages that contain meager or no content. This can result in Google misinterpreting the page’s intended response status code as a 404/410 instead of a 200 OK code. For instance, this can happen on empty tag pages that present no content.
Another reason for soft 404 errors is when there is a temporary problem with crawling. During the process, Google may encounter difficulty loading some of the page resources such as CSS and JS, which may cause the page to appear without content. In turn, this can make Google conclude that the page should be a 404 error. Soft 404 errors can occur due to Google’s misjudgment, where it mistakenly identifies a page as ‘seems to be a 404’ even though there is nothing wrong with it.
Soft 404 Errors and Their Impact on Your Site Ranking
Indeed, it is unequivocally true that soft 404 errors can indeed have a significant impact on the ranking of your website on Search Engine Results Pages (SERPs). To fathom the reasons for this, it is crucial to gain a profound understanding of how Google crawls a website. To mitigate the strain on a website’s server and alleviate the workload for crawl bots, Google employs a crawl budget to determine the frequency and the extent of a site’s crawling. Crawl budget, the amalgamation of crawl rate and crawl demand, is the number of pages that Googlebot will peruse on your site during each visit.
Crawl Rate refers to the frequency at which Googlebot sends requests to your website when it is crawling it. In contrast, Crawl Demand indicates the extent to which Google desires to crawl your site based on its popularity, the interval since the last crawl, and other such factors. When your website has an abundance of soft 404 errors, it utilizes a significant portion of your crawl budget. As a result, fewer of your website’s pre-existing pages are crawled, which leads to a diminished presence and a lowered ranking in SERPs.
To illustrate, if your website comprises 100 pages, and Google allocates a crawl budget of 70, only 70% of your pages will be crawled. If 10 pages are soft 404 errors, there is a risk that some or all of those pages will fall within the crawl budget of 70. Consequently, you forfeit the valuable opportunity for the search engines to crawl your pages
What difficulties do Soft 404s Cause?
Apart from ranking lower in search engine results pages (SERPs), soft 404 errors can cause a variety of issues for your website. For instance, if Googlebot comes across a soft 404 error on a page and discovers that the content is not truly missing, it may presume that your website is serving fake 404 errors. This may result in Google penalizing your website. A major problem associated with soft 404 errors is poor user experience. As soft 404 URLs still appear in search results, users may be directed to non-existent pages.
When a user clicks on a link to a page that returns a soft 404 error, they may infer that the page does not exist and leave your website. This can lead to a high bounce rate and reduced user engagement on your website. Soft 404 errors can also affect a website’s performance and operations. While 404 error pages do not occupy as much server space as pages with content, they still consume bandwidth. If Google and other search engines direct traffic to a non-existent page, it may slow down the website’s speed and performance, causing it to take hits.
Importance of fixing soft 404 errors
It is paramount to ensure that a webpage returns the appropriate HTTP status code. If a page is missing, invalid, or non-existent, the server must return a 404 or 410 (not found) error code or a 301 (moved) redirect code. Using a 200 success code for such pages is a substandard practice and should be avoided. It’s essential to offer a seamless and satisfactory user experience to website visitors. Providing a user with a page that has little or no value is detrimental to the website’s credibility and reputation.
It is imperative to ensure that users who click on links in search engine results land on pages with relevant and valuable content. Soft 404 errors waste the website’s crawl budget and search engine resources. Rather than indexing important pages, search engines expend resources indexing soft 404 pages, which is highly undesirable. Therefore, it is crucial to find and fix any soft 404 errors promptly to enhance the website’s search engine visibility and user experience.
How to find and fix Soft 404 errors?
As they lack official status, identifying soft 404 errors is not an overt task for website proprietors. A mechanism such as Google Search Console, however, offers assistance. According to Google, Search Console enables the user to “measure the traffic and performance of your site, address issues, and elevate your site’s status in Google Search outcomes.” One of the concerns that Search Console can aid in resolving is the soft 404 error.
Discovering Soft 404 Errors through Google Search Console : The process of signing up for an account and connecting your domain with Google Search Console is a straightforward one that only takes a few minutes to complete. Once connected, following information collection, Google Search Console will display soft 404 errors under the Coverage tab. Soft 404 errors can manifest in two ways: as an error or as excluded (as shown above).
Focusing on the soft 404s that appear with an error status is paramount. Those with an excluded status require attention, yet Google has ceased to crawl those pages for a reason. If no soft 404 errors are visible in Google Search Console, utilizing a crawler, such as Screaming Frog, to examine your site and scrutinize pages that return 404 errors is an alternative. By accessing the “Response Codes” section, the user can identify pages that return 404 errors.
Set up effective 404 or 410 Error if page doesn’t exists : If the website visitor encounters a soft 404 error, it is essential to establish a proper 404 (not found) or 410 (content deleted) error on the server to display the correct error to the users and crawl bots. The 404 and 410 error codes function similarly when it comes to search engines, according to Google’s John Mueller. Both these codes instruct Google to stop indexing the URL, which is the desired result in both cases.
Most Content Management Systems (CMSs) provide built-in error pages for 404 and 410. To optimize the 404 pages, it is recommended to add relevant blog posts or offer a search feature or tool. For instance, on Neil Patel’s webpage, users can analyze their website SEO from the 404 page. However, if your CMS does not have a pre-built error page, then you must configure your server and design the page with the assistance of a website developer. Although creating a personalized 404 error page may take a considerable amount of time and money, it is also an excellent branding opportunity.
Improve the Content and Re-index : Assuming that the page is necessary, it is essential to enhance the quality of your website’s content if it is mistakenly identified as a soft 404 error. Although Google is proficient in detecting soft 404 errors, it may inaccurately flag a page even if the content exists on the page, primarily when the content is short or irrelevant.
To prevent this, relevant and engaging content needs to be added to the page to ensure that it is not misinterpreted as a soft 404. This content must not include any fluff or filler, but it should instead enhance the quality of your website. Data-driven content that can be supported by external references, diagrams, and images can be included to substantiate your website’s credibility. This is crucial, mainly if the page was initially returning a soft 404 error. Detecting 404 errors can be beneficial for the user experience and the backend and front-facing sides of your website. Thus, it is imperative to optimize the content and re-index the page if it exists.
Set up 301 redirect for moved pages : When amalgamating content pages or relocating them, it is crucial to establish a redirect without delay to avoid website visitors being led to a 404 page. In the event that this causes soft 404 errors, a solution is readily available. A permanent 301 redirect to the latest page on your website is all that’s necessary. A majority of CMSs include a built-in redirect tool. Input the previous URL and the new one, and the traffic is automatically redirected. This manual provides step-by-step instructions on configuring redirects for WordPress. Upon being set up, the redirect is communicated to search engine crawlers, ensuring that they crawl the appropriate pages.
Conclusion
When a particular webpage is removed from your website, it is necessary to properly signal this removal to both human visitors and search engine bots. The most conventional technique for this is to set up a 404 error page. However, if not configured appropriately, the 404 error page may not be properly communicated to search engine bots, resulting in what is known as a “soft 404 error.” While soft 404 errors can cause complications with both search engine rankings and user experience, they are relatively easy to detect and address. Vispan solutions has a team of beast web developers to solve all your site errors.