In this edition of our Beginner SEO series we discuss the Robots.txt file. When it comes to crawling your site, the robots.txt file is one of your biggest assets. It exists primarily to inform search engine spiders and other bots which parts of your site they can and can’t access. It’s a file that is […]