A robots.txt file is an important tool which tells search engines which parts of your