How to block bad bots from accessing the website using .htaccess

Question 1

In the log file of my website, I see many bad bots and crawlers accessing my website. I have disallowed some of them in the robots.txt, but it seems that they are not obeying robots.txt. How can I block them using the .htaccess file?

Question 2

You can use the HTTP_USER_AGENT to check those bots or spiders and block them. Here I am blocking some of the well-known bad bots. When these bots will try to access the page, 403 will be returned.

## block these bots
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} (AhrefsBot|SemrushBot|Barkrowler|MJ12bot|DotBot|Java|DataForSeoBot|BLEXBot) [NC]
RewriteRule .* - [F,L]

pkumar81 · Answer 1 · 2022-09-29T08:56:45+0000

You can use the HTTP_USER_AGENT to check those bots or spiders and block them. Here I am blocking some of the well-known bad bots. When these bots will try to access the page, 403 will be returned.

## block these bots
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} (AhrefsBot|SemrushBot|Barkrowler|MJ12bot|DotBot|Java|DataForSeoBot|BLEXBot) [NC]
RewriteRule .* - [F,L]

How to block bad bots from accessing the website using .htaccess

Please log in or register to add a comment.

Please log in or register to answer this question.

1 Answer

Please log in or register to add a comment.

Related questions

Categories