Ask HN: How to stop Google indexing dynamic search pages
37 by scalesolved | 23 comments on Hacker News.
Hey HN folks, A few months ago I received a manual action penalty from Google as they detected spam pages on our domain. The problem was that when people were searching on our site they are directed to a page with the following: https://$domain/search?query=$QUERY Some users (most likely bots) are generating huge spam searches on our search page and somehow Google is indexing these and there are no inbound links to these pages (at least I cannot find any). To resolve this I did the following: * On our search page I set the following header: X-Robots-Tag: noindex (based off of the documentation here https://developers.google.com/search/reference/robots_meta_tag). * Submitted URLs to be dropped from Google Index via Webmaster console * Submitted 3 reconsideration requests to Google to avoid the penalties In theory this should stop all search pages being indexed (as they all contain the noindex header) and it has helped drop the number of indexed pages marked as spam by 99% however we still have a significant number of urls marked as spam and so our site has a penalty from Google. Has anyone had this issue before? How can I stop these pages becoming indexed when I have the noindex header set _and_ if you search the spam urls there are no inbound links to them? Any help appreciated folks!
Middlesbrough
Middlesbrough News Featured post
Kim Jong-un: fragile and under pressure but he won’t give up
The Observer The North Korean leader’s surprise freeze of his nuclear programme is less a genuine move than a diplomatic manoeuvre,...
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment