Robots.txt is a file that exists on the foundation directory of each website and may be used to instruct search engines like yahoo on which directories/data files of the website they will crawl and include inside their index. I am joyful to get messages from institutions, media associates and also https://news.rafeeg.ae