What is Robot.txt file in SEO

Robot.txt is a text file that tells search engine robots which pages on your sites to crawl.

Its also tells web robots which pages not to crawl.


Steps to create Robot.txt file for blog
1.) Open website https://ctrlq.org/blogger/
2.)  Put website in generate sitemap option and press enter

                                                                                                                                                                                                                                                                                                                      

3.)  After clicking enter your robot.txt file will generate

Add caption

4.)  you can copy  all that blue portion and paste in blogger and click on save option 
      go in blogger setting option - Search Preference - Click on custom robot.txt   Edit
      and paste all that blue portion text there and click on save button. 

Steps to Submit Updated robot.txt to Google
1.Put in google robots.txt tester and press enter

2. After press enter you will see interface like this
3. Click on 2 option which marked in blue after you will see this interface

4.Paste in that box generated robot.txt data and click on submit.This submit function of robot.txt tester allow you to ask google to more quickly crawl and index a new robot.txt file for your site.

5.you can also test your website weather its block or not by putting your URL below 
and click on Test.

Blocking all web crawlers from all content

User agent : *

Disallow : /

Allowing all web crawlers to access all content

User agent : *

Disallow :

Robot.txt File Generator
https://www.seoptimer.com/robots-txt-generator
https://smallseotools.com/robots-txt-generator/