Robots file is to prevent certain crawlers. Which content google search engine crawl and index. XML sitemap is a like Roadmap of your blogger. It's inform search engines to how many URL have them. Custom robots.txt and xml sitemap is a part of advance SEO setting for quick index in