SemRush has very nice and useful web site audit reports. It gives you a lot of different details how to improve your website and informs you if something got broken on your site.
I won’t get in to details about what you can see in SemRush audit report, please read here about it. But then you run it you will probably get a lot of errors if you have outbound links to external websites. It will be 406 error – Not Acceptable. It will happen not because there are actual errors on your site or you made error in writting url.
The problem is in web hosting. Some web hosting companies doesn’t allow SemRush robots to access web sites hosted on their servers. So you will get broken external link error in audit report if you have an external link to domain which hosting has disabled access for SemRush robots.
How to fix it? One way is to contact the host of domain you are referring to and ask access for SemRush robots. But it can be long procedure as it is not your website hoster and they can just decline your request.
But fortunately there is another and much easier way to fix it.
By default SemRush uses their own robot to crawl your website, but you can change it and your site can be crawled as Google agent.
How to change crawler settings? It is very easy:
1. Go to Settings on your top right
2. Find User Agent in setting list:
3. Change it from SEMrushBot-Desktop to GoogleBot-Desktop
That’s it. Re-run your audit and wait for a new report with less errors hopefully.
Leave a Reply