Ive been having issues when I access my robots.txt. Its effecting my indexing as well. I have been sending Sebastiaan Janssen so many messages through his site and no reply. Quiet disappointing though.
I installed his package Cultiv.DynamicRobots and then uninstalled it because I didn't need it. Since then I am having the below issue with my just robots.txt which I generated through robots.txt editor.
HTTP Error 500.21 - Internal Server Error Handler "Cultiv.DynamicRobots" has a bad module "ManagedPipelineHandler" in its module list
I am running Umbraco 4.7. With ms sql db.
Could one of you good souls help me with the above.
Okay, that looks like a typical web.config and the robots handler is registered correctly. Is your application pool configured to run in Integrated Pipeline mode and is it a .Net 4.0 application pool?
ASP .Net pool wasn't set to "Integrated" hence the Janssen's Cultiv.DynamicRobots package wasn't working. Error resulting was (although I removed the package via Umbraco package manager)
ERROR : Handler "Cultiv.DynamicRobots" has a bad module "ManagedPipelineHandler" in its module list
Solution : Change the Pipeline Mode to "Integrated" then reinstalling the package showed me the robots.txt again. Perfect!
Summary : Be sure to keep the Pipline mode : "Integrated" if you want to use the Cultiv.DynamicRobots
"Could not load file or assembly 'Cultiv.DynamicRobots' or one of its dependencies. The system cannot find the file specified."
Exception Details: System.IO.FileNotFoundException: Could not load file or assembly 'Cultiv.DynamicRobots' or one of its dependencies. The system cannot find the file specified.
Handler "Cultiv.DynamicRobots" has a bad module "ManagedPipelineHandler" in its module list
Hello!
Ive been having issues when I access my robots.txt. Its effecting my indexing as well. I have been sending Sebastiaan Janssen so many messages through his site and no reply. Quiet disappointing though.
I installed his package Cultiv.DynamicRobots and then uninstalled it because I didn't need it. Since then I am having the below issue with my just robots.txt which I generated through robots.txt editor.
HTTP Error 500.21 - Internal Server Error
Handler "Cultiv.DynamicRobots" has a bad module "ManagedPipelineHandler" in its module list
I am running Umbraco 4.7. With ms sql db.
Could one of you good souls help me with the above.
Thanks,
m.
Cultiv is always quick when it comes to answering questions on our, so there has to be something else, like messages not arriving.
I notified him by Twitter, so he should answer here soon.
Thanks Man.
Thanks Stefan.
@m: Wow, have not seen this before. Can you show your web.config file (make sure to edit out any passwords that I should not see)?
Sebastiaan, Thanks for the message. Please find below my web.config
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<configSections>
<section name="urlrewritingnet" restartOnExternalChanges="true" requirePermission="false" type="UrlRewritingNet.Configuration.UrlRewriteSection, UrlRewritingNet.UrlRewriter" />
<section name="microsoft.scripting" type="Microsoft.Scripting.Hosting.Configuration.Section, Microsoft.Scripting, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" requirePermission="false" />
<section name="clientDependency" type="ClientDependency.Core.Config.ClientDependencySection, ClientDependency.Core" />
<section name="Examine" type="Examine.Config.ExamineSettings, Examine" />
<section name="ExamineLuceneIndexSets" type="UmbracoExamine.Config.ExamineLuceneIndexes, UmbracoExamine" />
<!-- Added in Umbraco 4.6.2 -->
<sectionGroup name="system.web.webPages.razor" type="System.Web.WebPages.Razor.Configuration.RazorWebSectionGroup, System.Web.WebPages.Razor, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35">
<section name="host" type="System.Web.WebPages.Razor.Configuration.HostSection, System.Web.WebPages.Razor, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" requirePermission="false" />
<section name="pages" type="System.Web.WebPages.Razor.Configuration.RazorPagesSection, System.Web.WebPages.Razor, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" requirePermission="false" />
</sectionGroup>
<!-- End of added in Umbraco 4.6.2 -->
</configSections>
<urlrewritingnet configSource="config\UrlRewriting.config" />
<microsoft.scripting configSource="config\scripting.config" />
<clientDependency configSource="config\ClientDependency.config" />
<Examine configSource="config\ExamineSettings.config" />
<ExamineLuceneIndexSets configSource="config\ExamineIndex.config" />
<appSettings>
<add key="umbracoDbDSN" value="server=xxxx;database=xxxx;user id=xxxx;password=xxxx" />
<add key="umbracoConfigurationStatus" value="4.7.0" />
<add key="umbracoReservedUrls" value="~/config/splashes/booting.aspx,~/install/default.aspx,~/config/splashes/noNodes.aspx" />
<add key="umbracoReservedPaths" value="~/umbraco,~/install/" />
<add key="umbracoContentXML" value="~/App_Data/umbraco.config" />
<add key="umbracoStorageDirectory" value="~/App_Data" />
<add key="umbracoPath" value="~/umbraco" />
<add key="umbracoEnableStat" value="false" />
<add key="umbracoHideTopLevelNodeFromPath" value="true" />
<add key="umbracoEditXhtmlMode" value="true" />
<add key="umbracoUseDirectoryUrls" value="false" />
<add key="umbracoDebugMode" value="true" />
<add key="umbracoTimeOutInMinutes" value="20" />
<add key="umbracoVersionCheckPeriod" value="7" />
<add key="umbracoDisableXsltExtensions" value="true" />
<add key="umbracoDefaultUILanguage" value="en" />
<add key="umbracoProfileUrl" value="profiler" />
<add key="umbracoUseSSL" value="false" />
<add key="umbracoUseMediumTrust" value="false" />
<!--
Set this to true to enable storing the xml cache locally to the IIS server even if the app files are stored centrally on a SAN/NAS
Alex Norcliffe 2010 02 for 4.1 -->
<add key="umbracoContentXMLUseLocalTemp" value="false" />
<!-- Added in Umbraco 4.6.2 -->
<add key="webpages:Enabled" value="false" />
<add key="enableSimpleMembership" value="false" />
<add key="autoFormsAuthentication" value="false" />
<!-- End of added in Umbraco 4.6.2 -->
</appSettings>
<system.net>
<mailSettings>
<smtp>
<network host="xxxx" userName="xxxxx" password="xxxxx" />
</smtp>
</mailSettings>
</system.net>
<connectionStrings>
<remove name="LocalSqlServer" />
<!--<add name="LocalSqlServer" connectionString="server=.\sqlexpress;database=aspnetdb;user id=DBUSER;password=DBPASSWORD" providerName="System.Data.SqlClient"/>-->
</connectionStrings>
<system.web>
<customErrors mode="Off" />
<trace enabled="true" requestLimit="10" pageOutput="false" traceMode="SortByTime" localOnly="true" />
<sessionState mode="InProc" stateConnectionString="tcpip=127.0.0.1:42424" sqlConnectionString="data source=127.0.0.1;Trusted_Connection=yes" cookieless="false" timeout="20" />
<globalization requestEncoding="UTF-8" responseEncoding="UTF-8" />
<xhtmlConformance mode="Strict" />
<httpRuntime requestValidationMode="2.0" />
<pages enableEventValidation="false">
<!-- ASPNETAJAX -->
<controls>
<add tagPrefix="asp" namespace="System.Web.UI" assembly="System.Web.Extensions, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" />
<add tagPrefix="umbraco" namespace="umbraco.presentation.templateControls" assembly="umbraco" />
<add tagPrefix="asp" namespace="System.Web.UI.WebControls" assembly="System.Web.Extensions, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" />
</controls>
</pages>
<httpModules>
<!-- URL REWRTIER -->
<add name="UrlRewriteModule" type="UrlRewritingNet.Web.UrlRewriteModule, UrlRewritingNet.UrlRewriter" />
<add name="umbracoRequestModule" type="umbraco.presentation.requestModule" />
<!-- UMBRACO -->
<add name="viewstateMoverModule" type="umbraco.presentation.viewstateMoverModule" />
<add name="umbracoBaseRequestModule" type="umbraco.presentation.umbracobase.requestModule" />
<!-- ASPNETAJAX -->
<add name="ScriptModule" type="System.Web.Handlers.ScriptModule, System.Web.Extensions, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" />
<!-- CLIENT DEPENDENCY -->
<add name="ClientDependencyModule" type="ClientDependency.Core.Module.ClientDependencyModule, ClientDependency.Core" />
</httpModules>
<httpHandlers>
<remove verb="*" path="*.asmx" />
<!-- ASPNETAJAX -->
<add verb="*" path="*.asmx" type="System.Web.Script.Services.ScriptHandlerFactory, System.Web.Extensions, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" validate="false" />
<add verb="*" path="*_AppService.axd" type="System.Web.Script.Services.ScriptHandlerFactory, System.Web.Extensions, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" validate="false" />
<add verb="GET,HEAD" path="ScriptResource.axd" type="System.Web.Handlers.ScriptResourceHandler, System.Web.Extensions, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" validate="false" />
<!-- UMBRACO CHANNELS -->
<add verb="*" path="umbraco/channels.aspx" type="umbraco.presentation.channels.api, umbraco" />
<add verb="*" path="umbraco/channels/word.aspx" type="umbraco.presentation.channels.wordApi, umbraco" />
<add verb="*" path="DependencyHandler.axd" type="ClientDependency.Core.CompositeFiles.CompositeDependencyHandler, ClientDependency.Core " />
<add verb="GET,HEAD,POST" path="GoogleSpellChecker.ashx" type="umbraco.presentation.umbraco_client.tinymce3.plugins.spellchecker.GoogleSpellChecker,umbraco" />
</httpHandlers>
<compilation defaultLanguage="c#" debug="false" batch="false" targetFramework="4.0">
<assemblies>
<!-- ASP.NET 4.0 Assemblies -->
<add assembly="System.Design, Version=4.0.0.0, Culture=neutral, PublicKeyToken=B03F5F7F11D50A3A" />
<add assembly="System.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=B77A5C561934E089" />
<add assembly="System.Web.Extensions, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" />
<add assembly="System.Xml.Linq, Version=4.0.0.0, Culture=neutral, PublicKeyToken=B77A5C561934E089" />
<add assembly="System.Data.DataSetExtensions, Version=4.0.0.0, Culture=neutral, PublicKeyToken=B77A5C561934E089" />
<add assembly="System.Web.Extensions.Design, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" />
<add assembly="System.Web.Abstractions, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
</assemblies>
<!-- Added in Umbraco 4.6.2 -->
<buildProviders>
<add extension=".cshtml" type="umbraco.MacroEngines.RazorBuildProvider, umbraco.MacroEngines" />
<add extension=".vbhtml" type="umbraco.MacroEngines.RazorBuildProvider, umbraco.MacroEngines" />
<add extension=".razor" type="umbraco.MacroEngines.RazorBuildProvider, umbraco.MacroEngines" />
</buildProviders>
<!-- End of added in Umbraco 4.6.2 -->
</compilation>
<authentication mode="Forms">
<forms name="yourAuthCookie" loginUrl="login.aspx" protection="All" path="/" />
</authentication>
<authorization>
<allow users="?" />
</authorization>
<!-- Membership Provider -->
<membership defaultProvider="UmbracoMembershipProvider" userIsOnlineTimeWindow="15">
<providers>
<clear />
<add name="UmbracoMembershipProvider" type="umbraco.providers.members.UmbracoMembershipProvider" enablePasswordRetrieval="false" enablePasswordReset="false" requiresQuestionAndAnswer="false" defaultMemberTypeAlias="Another Type" passwordFormat="Hashed" />
<add name="UsersMembershipProvider" type="umbraco.providers.UsersMembershipProvider" enablePasswordRetrieval="false" enablePasswordReset="false" requiresQuestionAndAnswer="false" passwordFormat="Hashed" />
</providers>
</membership>
<!-- added by NH to support membership providers in access layer -->
<roleManager enabled="true" defaultProvider="UmbracoRoleProvider">
<providers>
<clear />
<add name="UmbracoRoleProvider" type="umbraco.providers.members.UmbracoRoleProvider" />
</providers>
</roleManager>
<!-- Sitemap provider-->
<siteMap defaultProvider="UmbracoSiteMapProvider" enabled="true">
<providers>
<clear />
<add name="UmbracoSiteMapProvider" type="umbraco.presentation.nodeFactory.UmbracoSiteMapProvider" defaultDescriptionAlias="description" securityTrimmingEnabled="true" />
</providers>
</siteMap>
</system.web>
<!-- ASPNETAJAX -->
<system.web.extensions>
<scripting>
<scriptResourceHandler enableCompression="true" enableCaching="true" />
</scripting>
</system.web.extensions>
<system.webServer>
<validation validateIntegratedModeConfiguration="false" />
<modules runAllManagedModulesForAllRequests="true">
<remove name="ScriptModule" />
<remove name="UrlRewriteModule" />
<remove name="umbracoRequestModule" />
<remove name="viewstateMoverModule" />
<remove name="umbracoBaseRequestModule" />
<remove name="ClientDependencyModule" />
<!-- Needed for login/membership to work on homepage (as per http://stackoverflow.com/questions/218057/httpcontext-current-session-is-null-when-routing-requests) -->
<remove name="FormsAuthentication" />
<add name="UrlRewriteModule" type="UrlRewritingNet.Web.UrlRewriteModule, UrlRewritingNet.UrlRewriter" />
<add name="umbracoRequestModule" type="umbraco.presentation.requestModule" />
<add name="viewstateMoverModule" type="umbraco.presentation.viewstateMoverModule" />
<add name="umbracoBaseRequestModule" type="umbraco.presentation.umbracobase.requestModule" />
<add name="ScriptModule" preCondition="managedHandler" type="System.Web.Handlers.ScriptModule, System.Web.Extensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" />
<add name="ClientDependencyModule" type="ClientDependency.Core.Module.ClientDependencyModule, ClientDependency.Core" />
<!-- Needed for login/membership to work on homepage (as per http://stackoverflow.com/questions/218057/httpcontext-current-session-is-null-when-routing-requests) -->
<add name="FormsAuthentication" type="System.Web.Security.FormsAuthenticationModule" />
</modules>
<handlers accessPolicy="Read, Write, Script, Execute">
<remove name="WebServiceHandlerFactory-Integrated" />
<remove name="ScriptHandlerFactory" />
<remove name="ScriptHandlerFactoryAppServices" />
<remove name="ScriptResource" />
<remove name="Channels" />
<remove name="Channels_Word" />
<remove name="ClientDependency" />
<remove name="SpellChecker" />
<add name="ScriptHandlerFactory" verb="*" path="*.asmx" preCondition="integratedMode" type="System.Web.Script.Services.ScriptHandlerFactory, System.Web.Extensions, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" />
<add name="ScriptHandlerFactoryAppServices" verb="*" path="*_AppService.axd" preCondition="integratedMode" type="System.Web.Script.Services.ScriptHandlerFactory, System.Web.Extensions, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" />
<add name="ScriptResource" verb="GET,HEAD" path="ScriptResource.axd" preCondition="integratedMode" type="System.Web.Handlers.ScriptResourceHandler, System.Web.Extensions, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" />
<add verb="*" name="Channels" preCondition="integratedMode" path="umbraco/channels.aspx" type="umbraco.presentation.channels.api, umbraco" />
<add verb="*" name="Channels_Word" preCondition="integratedMode" path="umbraco/channels/word.aspx" type="umbraco.presentation.channels.wordApi, umbraco" />
<add verb="*" name="ClientDependency" preCondition="integratedMode" path="DependencyHandler.axd" type="ClientDependency.Core.CompositeFiles.CompositeDependencyHandler, ClientDependency.Core " />
<add verb="GET,HEAD,POST" preCondition="integratedMode" name="SpellChecker" path="GoogleSpellChecker.ashx" type="umbraco.presentation.umbraco_client.tinymce3.plugins.spellchecker.GoogleSpellChecker,umbraco" />
<add path="robots.txt" verb="*" type="Cultiv.DynamicRobots.RobotsTxt, Cultiv.DynamicRobots" name="Cultiv.DynamicRobots" /></handlers>
<!-- Adobe AIR mime type -->
<staticContent>
<mimeMap fileExtension=".air" mimeType="application/vnd.adobe.air-application-installer-package+zip" />
</staticContent>
</system.webServer>
<system.codedom>
<compilers>
<compiler language="c#;cs;csharp" extension=".cs" type="Microsoft.CSharp.CSharpCodeProvider,System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" warningLevel="4">
<providerOption name="CompilerVersion" value="v4.0" />
<providerOption name="WarnAsError" value="false" />
</compiler>
</compilers>
</system.codedom>
<runtime>
<!-- Old asp.net ajax assembly bindings -->
<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
<dependentAssembly>
<assemblyIdentity name="System.Web.Extensions" publicKeyToken="31bf3856ad364e35" />
<bindingRedirect oldVersion="1.0.0.0-1.1.0.0" newVersion="4.0.0.0" />
</dependentAssembly>
<dependentAssembly>
<assemblyIdentity name="System.Web.Extensions.Design" publicKeyToken="31bf3856ad364e35" />
<bindingRedirect oldVersion="1.0.0.0-1.1.0.0" newVersion="4.0.0.0" />
</dependentAssembly>
</assemblyBinding>
</runtime>
<!-- Added in Umbraco 4.6.2 -->
<system.web.webPages.razor>
<host factoryType="umbraco.MacroEngines.RazorUmbracoFactory, umbraco.MacroEngines" />
<pages pageBaseType="umbraco.MacroEngines.DynamicNodeContext">
<namespaces>
<add namespace="Microsoft.Web.Helpers" />
<add namespace="umbraco" />
<add namespace="Examine" />
</namespaces>
</pages>
</system.web.webPages.razor>
<!-- End of added in Umbraco 4.6.2 -->
</configuration>
Okay, that looks like a typical web.config and the robots handler is registered correctly. Is your application pool configured to run in Integrated Pipeline mode and is it a .Net 4.0 application pool?
Yes It is.
Editted: I just checked the setting again and it wasnt set to Integrated. Which I did now. Let me get back to you in a bit.
Can you run this command and see if it helps:
%windir%\Microsoft.NET\Framework\v4.0.30319\aspnet_regiis.exe -i
And maybe afterwards do iisreset as well (run both by pressing WIN+R).
Ok mate we've got somewhere. Thanks Janssen.
ASP .Net pool wasn't set to "Integrated" hence the Janssen's Cultiv.DynamicRobots package wasn't working. Error resulting was (although I removed the package via Umbraco package manager)
ERROR : Handler "Cultiv.DynamicRobots" has a bad module "ManagedPipelineHandler" in its module list
Solution : Change the Pipeline Mode to "Integrated" then reinstalling the package showed me the robots.txt again. Perfect!
Summary : Be sure to keep the Pipline mode : "Integrated" if you want to use the Cultiv.DynamicRobots
Ok Janssen Issue No 2 is?
If I UNinstall the Cultiv.DynamicRobots package I get the below error.
-----------------------------------------------------------------------------------------------------------------
"Could not load file or assembly 'Cultiv.DynamicRobots' or one of its dependencies. The system cannot find the file specified."
Exception Details: System.IO.FileNotFoundException: Could not load file or assembly 'Cultiv.DynamicRobots' or one of its dependencies. The system cannot find the file specified.
-----------------------------------------------------------------------------------------------------------------
Hmm, well, it looks like the handler is not removed during uninstall then.
Just remove this line from the web.config:
Thanks for the Quick reply Janssen.
Yes thats perfect its working fine now.
Excellent, good to hear. Would you be so kind as to mark on of the answers as "Solved" to help the next person with this problem? Thanks!
is working on a reply...