Sign in Register
Copied to clipboard

Flag this post as spam?

This post will be reported to the moderators as potential spam to be looked at


  • Jakob Bagterp 13 posts 115 karma points c-trib
    May 17, 2023 @ 15:37
    Jakob Bagterp
    0

    What is best practise to disallow in robots.txt Umbraco v11 and .Net Core?

    In the past and in older Umbraco installations, it was recommended to disallow several directories in robots.txt to prevent search engines to crawl. For instance:

    Disallow: /aspnet_client/
Disallow: /bin/
Disallow: /config/
Disallow: /css/
Disallow: /data/
Disallow: /install/
Disallow: /masterpages/
Disallow: /python/
Disallow: /scripts/
Disallow: /umbraco/
Disallow: /umbraco_client/
Disallow: /usercontrols/
Disallow: /xslt/

    Is this still the case with Umbraco v11 and .Net Core installations, or is it different?

    As an example, Umbraco.com has the follow setting, though most of these paths aren't exposed:

    Sitemap: https://umbraco.com/sitemap/
User-agent: *
Disallow: /error.html
Disallow: /app_data/
Disallow: /app_plugins/
Disallow: /bin/
Disallow: /config/
Disallow: /data/
Disallow: /umbraco/
Disallow: /views/
    Copy Link
  • Yaco Zaragoza 88 posts 362 karma points
    May 18, 2023 @ 14:08
    Yaco Zaragoza
    0

    I also want to know the answer to this

    Copy Link
  • Huw Reddick 1929 posts 6697 karma points MVP 2x c-trib
    May 20, 2023 @ 12:17
    Huw Reddick
    0

    https://ukad-group.com/blog/your-pocket-guide-to-umbraco-seo/

    Copy Link
  • Jakob Bagterp 13 posts 115 karma points c-trib
    May 20, 2023 @ 13:43
    Jakob Bagterp
    0

    I'm not sure how your link answers the question?

    Copy Link
  • Huw Reddick 1929 posts 6697 karma points MVP 2x c-trib
    May 20, 2023 @ 14:15
    Huw Reddick
    0

    There is a section on roots.txt with an image.

    Copy Link
  • Jakob Bagterp 13 posts 115 karma points c-trib
    May 29, 2023 @ 12:13
    Jakob Bagterp
    0

    After a little research, maybe the answer can be found in the Constants.SystemDirectories section. It lists the following root folders:

    /bin/
/config/
/umbraco/
/install/
/app_plugins/
/backoffice/
/views/

    And GlobalSettings lists the following reserved paths:

    /app_plugins/
/install/
/mini-profiler-resources/
/umbraco/

    As there may be more relevant directories to block, can anyone confirm that this is correct?

    Copy Link
Please Sign in or register to post replies

Write your reply to:

Draft