[Apache] How to allow or disallow every pages to all bots using robots.txt

Question

What code should I use in the robots.txt to allow or disallow all pages of my website to all bots? I am using the Apache web server.

Answer 1

If you want to allow every page to all bots, you can use the following code in the robots.txt file.

User-agent: *
Disallow:

If you want to disallow every page to all bots, you can use the following code in the robots.txt file.

User-agent: *
Disallow: /