Robotnik

Overview

Robotnik allows you to serve up a different robots.txt file for each domain. That's useful if you don't want search engines crawling some of your sites (e.g., your staging site), or if you want to point to different sitemaps from within your robots.txt.

Installation

Simply download the ZIP file and install it using "Install local package" under the "Packages" node in the "Developer" section of your Umbraco installation.

Configuration

You just need to add some app settings to let Robotnik know which files to serve up for which domains. Refer to the repository for more details: https://github.com/rhythmagency/robotnik