Hi again, with U5 abandoned, I have shifted my porting efforts over to the U 4.7.2 branch. I was really hoping to get something done by the end of Codegarden 12, but there was just way too much to do...
However, what I can say now is that I am pretty confident that we will be able to do a good port over to Linux. I have been working on this, and most of the code changes are pretty minor (or, looking back now seem minor) - but there are a fair number of them. In a nutshell, I have a got a partially working version now, and am working towards, let's say an alpha release. It is probably too early to share a version of the code b/c there are still a lot of errors, and it may end up frustrating people.
The good news: In the past two years, since I started on looking at porting Umbraco over to Linux, mono has developed by leaps and bounds - this is now making a successful port possible.
The bad news: Linux is case sensitive. There are a lot of minor changes to be done here, and some discipline to be had in the future in casing conventions. Mono does have the MONO_IOMAP=all option - but this is too slow for any production system. Hence, all casing needs to be vetted for Linux. Mono offers some inspection tools but it is still work, lots of it, and requires consistency for future releases. I had hoped to have a univeral release of Umbraco but we will probably end up with two branches - one for Windows and another for Linux.
The small print: I have had success compiling Umbraco with mono 2.11.2 (using a parallel development environment), and forcing all assemblies to .NET 4.0. I have not had any success under .NET 3.5 - indeed with 3.5 you get a lot of resource not found errors (that are not case related).
The even smaller print: Other than case errors, I am at present dealing with some MYSQL errors, and other unspecified oddities (E.g. as you can get it in London: "The train is delayed, and the cause is unknown, and under investigation")
But the overall outlook remains optimistic. I will check in again or also place an item in my blog
In the past week, I have been busy getting the unit tests working. Porting the tests over to NUnit was easy enough - just a lot of search and replace, and the results compiled succesfully. Running the tests, however, required some modifications to the core code. The gist of the changes was to be able to inject configuration values, which are available in the .NET runtime, but not necessarily so at times when NUnit is running.
Usually we would put these values in through a mocking mechanism (e.g. NMock2 - which is what I m using). But the Umbraco Web App contains a long static chain, which NMock cannot deal with. An option would have been to use TypeMock - but TypeMock is quite pricey, and in my opinion would have created a high entry price for contributing to the source code and tests.
Having had a quick perusal of the tests, it seemed that the easiest approach was to Interface out the .NET 'ConfigurationManager' class, then using the facade pattern create a couple of derived classes - one for testing and one for live use. To break the static chain, I replaced all references to 'ConfigurationManager' with a Singleton that instantiates the required ConfigurationManager class through dependency injection. This approach kept the changes to the original source to a minimum (hence, minimised the possibility of introducing bugs), but required the additionof the new ConfigManager classes. The approach is quite standart. At some point, it would be nice to revisit the long static chain - and make it non-static, e.g. initialize in global.asax - but that would require pretty big changes.
In sum, using this approach I can make the unit tests pass. At present some units are passing, and I am working through the remaining ones.
OK, I ll put up the code on Friday. About the casing issues - which is probably the most tedious aspect, I m in the process of writing a small tool which will run through all the files in the solution, and make sure that the casing of the files matches that of any references to those files in the code. Additionally, the tool will let us apply some casing conventions (e.g. pascal versus camel case, default.aspx vs Default.aspx. That should get us upto speed very quickly. Also this should be far more robust than my manual approach. Exciting stuff...
Hi again Casey, the Umbraco core team, and all Linux / mono fans, let me first of all say how excited I am to be porting the Umbraco CMS over to Linux. We are finally at a stage where this porting can become practical reality. I believe that the Umbraco CMS is one of the best out there, and porting it over to mono / Linux and beyond will be great step for our little CMS.
What follows are a series of posts looking at the porting process. I am going to kick off with methodology and considerations.
Methodology
At this stage I am working on a proof of concept. The idea is to get Umbraco 4.7.2 up and running on Linux in the quickest possible way with the fewest possible changes. The methodology combines Agile techniques with signal extraction techniques - where the released stable Umbraco 4.7.2 source constitutes the signal. I am doing multiple passes (iterations) over the same signal - as it were to tune into the Linux channel. We can say that the Umbraco signal is arriving to the Linux channel in a degraded manner resulting in incomplete reconstitution. Our goal is to create a clean signal that will give us the clean "HD" CMS experience.
As with all signal extraction problems the GIGO, Garbage In - Garbage Out, principle applies. The rationale for choosing 4.7.2 is its high stable quality.
Here are the basic customer stories with the present status (the customer(s) being developer(s).
Get 4.7.2 compiling on mono. Status: yes with mono 2.11.2, .NET 4.
Get the application started and loaded without any 404 Resource XYZ not found errors. Minor code changes. Status: partially completed.
Verify all UI functions. SQL errors, and further 404 Resource errors. Status: in progress.
Get Unit tests working. Requires serious code changes. Status: partially completed.
Considerations
An important consideration is whether when the application becomes multi-platform, will the code for the other platforms be compiled and tested on each target platform? Or whether the code will be compiled and tested only on the source platform? I would recommend that the application is always compiled and tested on each target platform. Msunit, Pex & Moles do not work with mono, and unit tests would have to be ported and run on the target platform.
OK. let's have a look at the individual customer stories in detail.
Getting 4.7.2 compiling on mono
Hmm... Cannot seem to get the numbered lists working... Oh well...
The current stable release of mono is 2.10.x. The next stable version will be 2.12. The current SVN version is 2.11.2. All my builds are against 2.11.2.
Errorsresolved: umbraco/datalayer/SqlHelpers/MySql/SqlResources.resx - Position: Line 123, Column 5. Inner exception: Could not find a part of the path ".../umbraco/datalayer/SqlHelpers/MySql/sql\total.sql". Inner exception: Could not find a part of the path ".../umbraco/datalayer/SqlHelpers/MySql/sql\version4_upgrade.sql.sql".
Change all: sql\total.sql references to Sql\Total.sql, Sql\Version4_Upgrade.sql (mono can deal with the '\')
This also happens under the SqlHelpers/SqlServer folder.
umbraco/cms/businesslogic/Packager/FileResources/PackageFiles.resx: Error: Error: Invalid ResX input. Position: Line 123, Column 5. Inner exception: Could not find file ".../umbraco/cms/businesslogic/Packager/FileResources/packages.config". Another casing issue - use Packages.config.
umbraco/presentation/umbraco/templateControls/Resources.resx: Error: Error: Invalid ResX input. Position: Line 123, Column 5. Inner exception: Could not find file ".../umbraco/templateControls/inlinexslt.xsltTemplate". Another casing issue - use InlineXslt.xsltTemplate.
components/SQLCE4Umbraco/SqlCEResources.resx: Error: Error: Invalid ResX input. Position: Line 123, Column 5. Inner exception: Could not find a part of the path ".../components/SQLCE4Umbraco/sql\total.sql". As above Sql\Total.sql
umbraco/presentation/umbraco/Search/ExamineEvents.cs(7,7): Error CS0246: The type or namespace name `Lucene' could not be found. Are you missing a using directive or an assembly reference? (CS0246) Re-set reference from Shannon's c drive : ) to foreign dlls/Lucene.Net.dll
Edit: I forgot the following...
.../umbraco/actions/delete.aspx.designer.cs(21,21): Error CS0234: The type or namespace name `uicontrols' does not exist in the namespace `umbraco.presentation.umbraco'. Are you missing an assembly reference? (CS0234) You will get about a 170!! of these - apparently there are some differences in how mono resolves references.
The error is easily fixed: Replace protected umbraco.uicontrols... with protected global::umbraco.uicontrols
Then you might need to do the same for: (about 8-9 occurences) protected umbraco.controls... to protected global::umbraco.controls.
Compile now builds all projects (Tests ignored at the moment)
OK. let's have a look at the individual customer stories in detail.
Getting the application started and loaded without any 404 Resource XYZ not found errors
Let's have a look at what happens here.
System.IO.FileNotFoundException Could not load file or assembly 'System.Web.Entity' or one of its dependencies. The system cannot find the file specified. Remove reference from umbraco.presentation and rebuild.
System.Configuration.ConfigurationErrorsException Unable to open configSource file '.../umbraco/presentation/config\UrlRewriting.config. Unfortunately mono cannot deal with the '\' paths in web.config. Set all such paths from '\' to '/'.
System.Web.Configuration.nBrowser.Exception Parent not found with id = default. Make sure the .NET 4 default.browser file is available and in the application path.. e.g. in ../umbraco/presentation/App_Browsers This file is not available in mono, you will need to get it from your windows .NET 4 files.
umbraco.businesslogic.Exceptions.UserAuthorizationException There was attempt to redirect to '/umbraco/' which is another domain than where you've logged in. Cross site scripting barrier kicks in. Not sure why - could be a localdomain issue. Comment out css barrier for now, .../umbraco/presentation/umbraco/login.aspx.cs lines 137:144 Fix is deferred for now
The installer screen does not appear In web.config set the umbracoConfigurationStatus key to ""
The installer appears but no styles / scripts are loaded. Change ../umbraco_client/installer references to ../umbraco_client/Installer
Script resources (*.axd) not found. In web.config change all 3.5.0.0 assembly references to 4.0.0.0
Cannot remember why I did this anymore... In .../umbraco/presentation/install/default.aspx added references to: System.Web.Configuration, System.Text, and System.Drawing
System.Web.HttpException The file '/install/steps/defaultuser.ascx' does not exist. This is the case for these install steps again it is a casing issue In umbraco.presentation/install/steps/Definitions/*.cs set the UserControl property value casing to match referenced file name casing. I.e., /steps/defaultuser.ascx to /steps/defaultUser.ascx, and /steps/validatepermissions.ascx to /steps/validatePermissions.ascx
Starter Kits step 'No Thanks' image does not show In umbraco.presentation/install/Skinning/locadStarterKits.aspx line 35 change installer to Installer.
System.InvalidCastException Cannot cast from source type to destination type at mono/mcs/class/System.Web.Extensions/System.Web.Script.Serialization/JavaScriptSerializer.cs:67
Login panel appears on the left javascript and files do no load In .../components/umbraco.controls/Panel.cs, replace references to panel/ with Panel/ OK, now we can login.
But... upon logging in you are greeted with: System.InvalidCastException at System.Web.Script.Serialization.JavaScriptSerializer..ctor in mono/mcs/class/System.Web.Extensions/System.Web.Script.Serialization/JavaScriptSerializer.cs:67 This is a particularly tough error to sort out. The trick is to look at the Application output. System.Web.Extensions is loaded twice as .NET 4 (by the web site) and as .NET 3.5 (by the Our.Umbraco.uGoLive.dll). The latter gains precendence and causes a crash. This is a documented mono bug with workarounds. You can choose from one of these: (1) recompile the Our.Umbraco.uGoLive.dll in .NET4 but the source is not available, (2) disable Our.Umbraco.uGoLive.dll, or (3) use an assembly redirect to redirect 3.5.0.0 to 4.0.0.0. We will do (3).
Whew, that was something... Now we log in and are greeted by An empty screen with a loading bar Using Firebug, focus on the GET language.asp 500 internal server error. The error results because language.asp cannot find the appropriate language xml file to load. In umbraco/businesslogic/IO/IOHelper.cs change in line 99, path[1] to path[0]. This is a small but significant change and may need revisiting.
The language file now loads but the screen still looks the same In umbraco.presentation/umbraco/umbraco.aspx, and in umbraco.presentation/umbraco/masterpages/umbracoPage.master change Application/jQuery to Application/JQuery. Do a search and replace for this pattern to correct all solution occurrences.
Now, the UI loads but the right hand side Dasboard panel gives a file not found error In umbraco/businesslogic/IO/SystemFiles.cs change dashboard.config to Dashboard.config. In the same file also change these: xsltExtensions to xsltExtensions restExtensions to restExtensions
We ll be proactive and clear a few more casing mismatches: In umbraco.presentation/umbraco/umbraco.aspx change all occurrences of UmbracoSpeechBubbleBackend to UmbracoSpeechBubbleBackEnd, and also in the whole solution replace tabView with tabview (for umbraco_client/tabview/images) Also do a search and replace changing speechBubble_close to speechbubble_close (for the speech bubble close image) Now we are left with two ScriptResource not found errors, and some Dashboard load errors.
Fix the dashboard In umbraco.presentation/config/Dashboard.config, replace file references to Pascal casing so as to match the file names. There are a few images fixed in the same manner: tv.png to TV.png, and listitemorange.gif to listItemOrange.gif, feedproxy.aspx to FeedProxy.aspx This completes the fixes for the Dashboard. The last two tabs are not in working order, and I will ignore them for now.
SUMMARY
This completes the changes needed to make the interface load. Most of the changes have to do with mismatches between actual file name casing and the hard coded references to those files. There are a number of approaches possible, e.g., all file names can be abstracted into a separate file - where possible. This would help. Fixes do not need to be applied in code, but can be applied to file and folder names - even symbolic links can be used. I have gone the hard way - replacing hard coded values - to illustrate the extent of the issue.
Before we put the UI through its paces, let's look at the TDD aspects of our port.
Getting Unit tests working
Right... After a good nights's sleep, back to the business...
From a TDD perspective, with NUnit & NMock, we cannot escape the conclusion that major parts of the source need to be re-written. In particular, the 4.7.2 source code contains a long static chain - a chain of objects with static variables & functions, that render testing with NUnit & NMock impossible. In general, many testing frameworks do not allow for mocking of static chain elements, and this style of coding has consequently come into dis-favor.
It would be preferable for example to have a very small static application loader class, that instantiates all relevant variables & constants, and initializes data services. In .NET, this could be done in the global.asax file. Then for testing, we isolate all database read / write calls, which are tested separately. The functions that use database services are tested separately using a test data service and mocks. This has the benefit of testing applications functionality and database access in isolation of each other and is especially helpful for diagnosing any multi-platform issues.
Of course, a major re-write of 4.7.2 is beyond the scope of this porting exercise - which is all about porting an existing application with minimal changes. But these issues can be addressed by the next 4.x version.
In this section, I will show what minimum changes are needed to get unit tests working in Linux. The changes rely on some clever minor re-writing of the existing code to inject test values through dependency injection. It is not the prettiest approach, but it works.
Monodevelop does not recognize the Umbraco.Test project Open the Umbraco.Test.csproj file and delete the {3AC096D0-A1C2-E12C-1390-A8335801FDAB};{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC} line The Umbraco.Test project will not load in the solution. Now in the Solution Options > Build Configurations, Configuration Mappings, add Umbraco.Test to the build.
Setting up Nunit tests Download the NUnit dlls. In the project references, delete the 'Microsoft.VisualStudio.QualityTools.UnitTestFramework, and add nunit.framework.dll
Test Conversion Using search replace tools, for every test replace 'using Microsoft.VisualStudio.TestTools.UnitTesting;' with 'using NUnit.Framework;' Using this link convert all test calls. The tests now compile.
Getting Tests to Pass All tests run but fail. I have started fixing these tests. Fix 1: establish database access. NUnit does not have access to the .NET Application environment, and calls to Configuration.Appsettings[key] return null. This causes tests requring database access to fail. Solution: inject database settings.
Code re-writes for injecting test Appsettings I target the static umbraco/businesslogic/GlobalSettings.cs class DbDSN property (line 139), and replace the static system call with a lazy loading singleton that mimics static behavior: //return ConfigurationManager.AppSettings["umbracoDbDSN"]; return ConfigurationManagerService.Instance.AppSettings["umbracoDbDSN"];
The ConfigurationManagerService class creates and returns an instance of the default or a test configurator. In tests we can set the test configurator with: ConfigurationManagerService.ConfigManager = new ConfigurationManagerTest(SetUpUtilities.GetAppSettings()); If the 'ConfigManager' property is null, then the service defaults to the runtime, non-test configurator.
All of this is probably not making much sense, and seeing the whole code will help.
1. First we create an interface for the ConfigurationManager calls that we are using.:
// .../umbraco/interfaces/IConfigurationManager.cs using System;
using System.Collections.Specialized;
using System.Web;
namespace umbraco.interfaces
{
public interface IConfigurationManager
{
NameValueCollection AppSettings {get;}
Object GetSection(string SectionName);
void RefreshSection(string SectionName);
}
}
2. Next we create our wrappers (aka Facade pattern)
// .../umbraco/businesslogic/ConfigurationManager/ConfigurationManagerDefault.cs using System;
using System.Collections.Specialized;
using System.Configuration;
using System.Web;
using umbraco.interfaces;
namespace umbraco.BusinessLogic
{
public class ConfigurationManagerDefault : IConfigurationManager
{
public ConfigurationManagerDefault (){}
public NameValueCollection AppSettings
{
get
{
return ConfigurationManager.AppSettings;
}
}
public Object GetSection(string SectionName)
{
return ConfigurationManager.GetSection(SectionName);
}
public void RefreshSection(string SectionName)
{
ConfigurationManager.RefreshSection(SectionName);
}
}
}
// .../umbraco/businesslogic/ConfigurationManager/ConfigurationManagerTest.cs using System;
using System.Collections.Specialized;
using System.Configuration;
using System.Web;
using umbraco.interfaces;
namespace umbraco.BusinessLogic
{
public class ConfigurationManagerTest : IConfigurationManager
{
public ConfigurationManagerTest(NameValueCollection appSettings)
{
_appSettings = new NameValueCollection();
_appSettings.Add(appSettings);
}
private NameValueCollection _appSettings;
public NameValueCollection AppSettings
{
get
{
return MergeAppSettings(ConfigurationManager.AppSettings);
}
}
public Object GetSection(string SectionName)
{
return ConfigurationManager.GetSection(SectionName);
}
public void RefreshSection(string SectionName)
{
ConfigurationManager.RefreshSection(SectionName);
}
private NameValueCollection MergeAppSettings(NameValueCollection appSettings)
{
NameValueCollection mergedAppSettings;
mergedAppSettings = new NameValueCollection();
if (appSettings.HasKeys())
foreach (string key in appSettings)
{
if (_appSettings[key] != null)
mergedAppSettings.Add(key.ToString(), _appSettings[key].ToString());
else
mergedAppSettings.Add(key.ToString(), appSettings[key].ToString());
}
else
mergedAppSettings.Add(_appSettings);
return mergedAppSettings;
}
}
}
3. We create the injector
// .../umbraco/businesslogic/ConfigurationManager/ConfigurationManagerFactory.cs using System;
using System.Collections.Specialized;
using System.Configuration;
using System.Web;
using umbraco.interfaces;
namespace umbraco.BusinessLogic
{
public class ConfigurationManagerFactory
{
private ConfigurationManagerFactory() {}
public static IConfigurationManager GetConfigManager(IConfigurationManager configManager)
{
if (configManager == null)
return new ConfigurationManagerDefault();
else
return configManager;
}
}
}
4. We create the singleton
// .../umbraco/businesslogic/ConfigurationManager/ConfigurationManagerService.cs using System;
using System.Collections.Specialized;
using System.Configuration;
using System.Web;
using umbraco.interfaces;
namespace umbraco.BusinessLogic
{
public sealed class ConfigurationManagerService
{
private static volatile IConfigurationManager _instance;
private static object _syncRoot = new Object();
private static IConfigurationManager _configManager = null;
public static IConfigurationManager ConfigManager
{
get {return _configManager;}
set
{
_configManager = value;
}
}
private ConfigurationManagerService (){}
public static IConfigurationManager Instance
{
get
{
if (_instance == null)
{
lock(_syncRoot)
{
if (_instance == null)
_instance = ConfigurationManagerFactory.GetConfigManager(_configManager);
}
}
return _instance;
}
}
}
}
5. So far, we have not modified any existing source code but now we do that.
// .../umbraco/businesslogic/GlobalSettings.cs line 139
public static string DbDSN
{
get
{
try
{
//return ConfigurationManager.AppSettings["umbracoDbDSN"];
return ConfigurationManagerService.Instance.AppSettings["umbracoDbDSN"];
}
// ...
6. We can use the following quick and dirty test setup (it would be better to read the settings from a file).
// .../umbraco.Test/SetUpUtilities.cs using System;
using System.Collections.Specialized;
namespace umbraco.Test
{
public class SetUpUtilities
{
public SetUpUtilities () {}
private const string _umbracoDbDSN = "server=127.0.0.1;database=someDB;user id=someUser;password=somePwd;datalayer=MySql";
public static NameValueCollection GetAppSettings()
{
NameValueCollection appSettings = new NameValueCollection();
//add application settings
appSettings.Add("umbracoDbDSN", _umbracoDbDSN);
return appSettings;
}
}
}
7. A simple test example
// .../umbraco.Test/ApplicationTest.cs using NUnit.Framework;
using System;
using umbraco.interfaces;
using System.Collections.Generic;
using umbraco.DataLayer;
using System.Linq;
namespace umbraco.Test
{
///
///This is a test class for ApplicationTest and is intended
///to contain all ApplicationTest Unit Tests
///
[TestFixture]
public class ApplicationTest
{
[TestFixtureSetUp]
public void InitTestFixture()
{
ConfigurationManagerService.ConfigManager = new ConfigurationManagerTest(SetUpUtilities.GetAppSettings());
}
[Test]
public void Application_Make_New()
{
var name = Guid.NewGuid().ToString("N");
Application.MakeNew(name, name, "icon.jpg");
//check if it exists
var app = Application.getByAlias(name);
Assert.IsNotNull(app);
//now remove it
app.Delete();
Assert.IsNull(Application.getByAlias(name));
}
}
}
3. Important caveats: if possible avoid using gnome as your window manager. gnome makes heavy use of C# / mono, and you may find that you have to replicate a lot of additional libraries. Actually, avoid using KDE as well, to avoid any potential adverse interactions, use the simplest possible window manager you can find: E.g. openbox, or if you are like me and want your compositing as well, go for Compiz.
4. I found adding the following to the environment setup was necessary:
export MONO_GAC_PREFIX=/opt/mono:/usr
5. Quick description of the Compiz set-up on Arch Linux:
#.xinitrc
exec compiz-alone-session
setxkbmap -layout uk #if using vmware vmware-user-suid-wrapper
# ~/.config/compiz/autostart.sh
# This shell script is run before Compiz launches.
# Environment variables set here are passed to the Compiz session.
# Set a background color
BG=""
if which hsetroot >/dev/null 2>&1; then
BG=hsetroot
else
if which esetroot >/dev/null 2>&1; then
BG=esetroot
else
if which xsetroot >/dev/null 2>&1; then
BG=xsetroot
fi
fi
fi
test -z $BG || $BG -solid "#202020"
# D-bus
if which dbus-launch >/dev/null 2>&1 && test -z "$DBUS_SESSION_BUS_ADDRESS"; then
eval `dbus-launch --sh-syntax --exit-with-session`
fi
# Run XDG autostart things. By default don't run anything desktop-specific
# See xdg-autostart --help more info
DESKTOP_ENV="COMPIZ"
if which /usr/bin/xdg-autostart >/dev/null 2>&1; then
/usr/bin/xdg-autostart $DESKTOP_ENV
fi
xrdb -merge ~/.Xresources
export _JAVA_OPTIONS='-Dawt.useSystemAAFontSettings=lcd'
xrdb -merge ~/.Xresources &
xrandr -s 1280x800 &
thunar --daemon &
emerald --replace &
nitrogen --restore &
cairo-dock &
PS: On Arch Linux with open-vmware-tools you can set in the VMware machine settings, 3D acceleration to yes, and you will get compositing and more.
You can also add archlinux bashrun, and compiz-boxmenu and compiz-deskmenu - though the menu set-up is slightly buggy: I ended up configuring the menu with the compiz-deskmenu tool, and then using that config file with compiz-boxmenu (using option-alt-m to launch the menu through ccsm commands).
Oh, and I m using a pure 32-bit machine:
[mono] ~ @ uname -a
Linux 3.4.4-2-ARCH #1 SMP PREEMPT Sun Jun 24 17:28:37 UTC 2012 i686 GNU/Linux
[mono] ~ @ mono -V
Mono JIT compiler version 2.11.2 (master/a7e01ca Tue Jun 19 21:41:42 BST 2012)
Copyright (C) 2002-2012 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
TLS: __thread
SIGSEGV: altstack
Notifications: epoll
Architecture: x86
Disabled: none
Misc: softdebug
LLVM: supported, not enabled.
GC: Included Boehm (with typed GC and Parallel Mark)
I have started looking at the UI. This looks like it will be the most challenging aspect of the port. We will be looking at various errors and resolving them. In most instances, there are no clear error messages. But so far errors in the UI have been one of the following: javascript which does not load, casing issues, possible mono bugs or idiosyncracies.
But first an addition to getting the application to load.
After switching all projects to .NET 4, we get: .../umbraco/cms/businesslogic/member/Member.cs(43,43): Error CS0246: The type or namespace name `MembershipProvider' could not be found. Are you missing a using directive or an assembly reference? (CS0246) (umbraco.cms) Add References to System.Web.ApplicationServices in umbraco.cms, umbraco.providers
Final ScriptResource not found error. Traces to js/dualSelectBox.js in umbraco.aspx lines 53::63 We have a CompositeScript control with one script. Remove the CompositeScript tags, i.e. delete lines 54 and 58. Not sure why this generates an error but I am suspecting a mono issue.
General casing issues: replace as follows - editNodeTypeNew.aspx to EditNodeTypeNew.aspx settingDataType.gif to settingDatatype.gif /genericProperties to /GenericProperties
Cannot get to DocumentType edit screen .../umbraco/settings/EditNodeTypeNew.aspx?id=... System.Web.HttpException Multiple controls with the same ID 'ctl00' were found. FindControl requires that controls have unique IDs. EditNodeTypeNew.aspx, Lines 9, 15, and give property panels IDs. Generally if you have multiple controls of the same type on the page make sure you explicitly give them distinct ids. E.g.
No save button image: editor/save.gif to editor/Save.GIF
Save bubble: speechbubble/ to speechBubble/
umbraco splash umbracosplash. to umbracoSplash.
In Info and Structure tabs, list item selection does not persist after save. This used to be a pre mono 2.6 bug, but the investigation continues
UI Testing - Templates
System.Web.Compilation.ParseException expecting '>'. Got 'asp' Mono is very particular. Do this - but only for the following sections. .../umbraco/presentation/umbraco/settings/editTemplate.aspx lines 11::244 IF you do it for the whole of the script tag, then inline asp will not execute. This is most likely a mono bug. Search for 'n
I did this too change .master extensions to .Master - this is tricky. Do not do a global S & R. Do it by hand. Will need to revisit but using .Master for compatibility for now.
UI Testing - Media airinstallbadge to AIRInstallBadge
Of course, we are not talking about an Oscar winning movie, rather our three paths simply are "~", "/", and "\\", the last one also known as @"\". All of these three (path) characters play a signifcant role in our linux port.
Let's start with "~"
Changing .../umbraco/businesslogic/IO/IOHelpers.cs line 114: path[1] to path[0] got us going but it also introduced a subtle bug. Here it is:
System.Exception Could not locate TinyMCE by URI:/umbraco_client/tinymce3/tiny_mce_src.js, Physical path:/umbraco_client/tinymce3/tiny_mce_src.js. Make sure that you configured the installPath to a valid location in your web.config. This path should be an relative or site absolute URI to where TinyMCE is located. .../umbraco/components/editorControls/tinyMCE3/webcontrol/TinyMCEWebControl.cs:219
Because we had effectively stripped the ~, the above was not mapped properly.
In fact going from path[1] to path[0] changes application behaviour significantly. As "~" in Linux is resolved as the value of $HOME, stripping the "~" character for any IO operation involving the file system makes sense. In windows, .NET operations which involve e.g. loading a file path with "~" in it resolve well. Not so under Linux.
Going from path[1] to path[0], however, did not work in this case:
Content > Trying to load a content page with a rich text editor System.Exception Could not locate TinyMCE by URI:/umbraco_client/tinymce3/tiny_mce_src.js, Physical path:/umbraco_client/tinymce3/tiny_mce_src.js. Make sure that you configured the installPath to a valid location in your web.config. This path should be an relative or site absolute URI to where TinyMCE is located. .../umbraco/components/editorControls/tinyMCE3/webcontrol/TinyMCEWebControl.cs:219
So, now we apply a better fix
(1) In .../umbraco/businesslogic/IO, we have add a new MultiPlatformHelper class:
using System; using System.Text;
namespace umbraco.IO { public static class MultiPlatformHelper {
public const string PLATFORM_UNIX = "UNIX"; public const string PLATFORM_WIN = "WIN";
public const string WIN_DIRSEP = "\\"; public const string UNIX_DIRSEP = "/";
public static string OSPlatform { get { return System.Environment.OSVersion.Platform.ToString().ToUpper(); } }
public static bool IsWindows() { return OSPlatform.Contains(PLATFORM_WIN); }
public static bool IsUnix() { return OSPlatform.Contains(PLATFORM_UNIX); }
public static string MapUnixPath(string path) { string filePath = path;
if (filePath.StartsWith("~")) filePath = IOHelper.ResolveUrl(filePath);
filePath = IOHelper.MapPath(filePath, true);
return filePath; }
public static string ConvertPathFromUnixToWin(string path) { return path.Replace(MultiPlatformHelper.UNIX_DIRSEP, MultiPlatformHelper.WIN_DIRSEP); }
} }
(2) Then in .../umbraco/businesslogic/IOHelper.cs, modify MapPath as:
public static string MapPath(string path)
{
if (IO.MultiPlatformHelper.IsWindows())
return MapPath(path, true);
Before moving on to the "/" and "\\" characters, let's also fix this error:
System.Web.HttpException The resource cannot be found. Details: Requested URL: /umbraco/~/umbraco/js/umbracoUpgradeChecker.js Path error: should say - /umbraco/js/umbracoUpgradeChecker.js
Solution: in .../umbraco/presentation/umbraco/umbraco.aspx.cs line 104 change
Now we are ready to look at the "/" and "\\" character issues. As we know, the path separator in Linux is "/", while in windows it is "\\".
Mono has two built-in mechanisms for dealing with these differences:
(1) When mono is compiled with IS_PORTABILITY_SET, then in the mono CLR, for a unix system, AltDirectorySeparatorChar is defined as "\\". (Otherwise AltDirectorySeparatorChar is defined as "/")
(2) Running the mono app with the MONO_IOMAP=all environment variable set also sets the AltDirectorySeparatorChar to "\\" on Unix systems. However, running Umbraco with this environment variable set is slow and prone to crashes.
I am going to go for a third option, where I make the path character issues explicit in code (with help from the MultiPlatformHelper class):
(3) Resolving path separator character issues in the Umbraco code itself.
The changes so far impact the downloading and installation of packages.
Preliminaries
Developer > Packages System.IO.DirectoryNotFoundException Could not find a part of the path ".../umbraco/presentation/App_Data/packages/created/createdPackages.config". If this directory / file is missing get it from a working install
cd umbraco/presentation/umbraco/developer ln -s Packages packages
Developer > Packages System.Exception Error - file not found. Could not find file named '.../umbraco/presentation/App_Data/packages\... In .../umbraco/cms/businesslogic/Packager/Repositories/Repository.cs line 209, Replace
Developer > Packages After installation we find under .../umbraco/presentation, a directory called: \home\...\umbraco\presentation\, which contains the package files.
In .../umbraco/cms/businesslogic/Packager/Installer.cs Alter lines 814:815 to
Contents > TinyMCE Editor Save and publish image is not found Search replace saveAndPublish.gif with SaveAndPublish.gif
Developer > xslt files System.IO.DirectoryNotFoundException Directory '.../umbraco/presentation/xslt' not found Solution: Create the missing 'xslt' directory.
Developer > Script Files System.Web.HttpException The resource cannot be found. Details: Requested URL: /umbraco/developer/python/editPython.aspx Replace python/editPython.aspx with Python/editPython.aspx
Developer > Cache Browser System.Web.HttpException The resource cannot be found. Details: Requested URL: /umbraco/developer/cache/viewCacheItem.aspx Replace cache/viewCacheItem.aspx with Cache/viewCacheItem.aspx
Developer > Data Types System.Web.HttpException The resource cannot be found. Details: Requested URL: /umbraco/developer/datatypes/editDataType.aspx Replace datatypes/editDataType.aspx with DataTypes/editDatatype.aspx
Developer > Macros System.IO.DirectoryNotFoundException Directory '.../umbraco/presentation/usercontrols' not found.
Create the usercontrols directory
Developer > Macros System.Web.HttpException The resource cannot be found. Details: Requested URL: /umbraco/developer/macros/editMacro.aspx Replace macros/editMacro.aspx with Macros/editMacro.aspx
Developer > Packages System.Web.HttpException The resource cannot be found. Details: ../packages/... Replace developer/packages with developer/Packages
Compilation Error .../umbraco/presentation/umbraco/developer/Packages/editPackages.aspx Replace protected umbraco.uicontrols with protected global::umbraco.uicontrols
Developer > Packages System.Web.HttpException The resource cannot be found. Details: Requested URL: /umbraco_client/images/progressBar.gif Replace /images/progressBar.gif with /images/progressbar.gif
And, in .../umbraco/presentation/umbraco/dialogs/emptyTrashcan.aspx.designer.cs Replace umbraco.uicontrols with global::umbraco.uicontrols
Developer > Packages Uninstall does not delete .dll's and added directories. But installed packages file etc are updated. Defer for now.
Developer > Macro > Create produces: System.IO.DirectoryNotFoundException Directory '.../umbraco/presentation/umbraco/xslt/templates/schema2' not found. Replace schema2 with Schema2 Replace clean.xslt with Clean.xslt
Developer > Macro System.Web.HttpException The resource cannot be found. Details: Requested URL: /umbraco/developer/xslt/editXslt.aspx Replace xslt/editXslt.aspx with Xslt/editXslt.aspx
Also replace insField.GIF with insField.gif insMemberItem.GIF with insMemberItem.gif insChildTemplateNew.GIF with insChildTemplateNew.gif insFieldByLevel.GIF with insFieldByLevel.gif xslVisualize.GIF with xslVisualize.gif
Members > Member Groups System.Web.HttpException The resource cannot be found. Details: Requested URL: /umbraco/members/editMemberGroup.aspx Replace members/editMemberGroup.aspx with members/EditMemberGroup.aspx Replace membergroup.gif with memberGroup.gif Replace membertype.gif with memberType.gif
System.Web.HttpException The resource cannot be found. Details: Requested URL: /umbraco/members/viewMembers.aspx Replace members/viewMembers.aspx with members/ViewMembers.aspx
System.Web.HttpException The resource cannot be found. Details: Requested URL: /umbraco/members/editMember.aspx Replace members/editMember.aspx with members/EditMember.aspx, then editMember.aspx with EditMember.aspx
Then in search.aspx.designer.cs Replace umbraco.uicontrols with global::umbraco.uicontrols
There are a couple of other important changes, Here they are:
Umbraco > Members > EditMembers > Save
Umbraco > Media > editMedia > Save System.MethodAccessException Method `umbraco.cms.businesslogic.web.Document:FireBeforeSave (umbraco.cms.businesslogic.SaveEventArgs)' is inaccessible from method `umbraco.controls.ContentControl:saveClick (object,System.Web.UI.ImageClickEventArgs) The error originates in line 236 of .../umbraco/presentation/umbraco/controls/ContentControl.cs
doc.FireBeforeSave(docArgs);
this indeed breaks protection. In .../umbraco/cms/businesslogic/web/Document.cs line 1884, we have,
protected internal new virtual void FireBeforeSave(SaveEventArgs e)
change this to,
public new virtual void FireBeforeSave(SaveEventArgs e)
We may need to revisit this.
The other issue involves the '¤' character, which is used as a string splitter.Mono is quite fussy about how this string is defined. Here's what works in mono.
Settings > Scripts > Create Folder Index out of bounds error This is due to issues with "¤" string splitting. In file .../umbraco/presentation/umbraco/create/script.ascx.cs, modify line 40 as:
Umbraco > Members Could not load control: '/umbraco/Members/MemberSearch.ascx replace Members/MemberSearch.ascx with members/MemberSearch.ascx
System.Web.HttpException The resource cannot be found. Details: Requested URL: /umbraco/settings/editMediaType.aspx Replace settings/editMediaType.aspx with settings/EditMediaType.aspx
System.Web.HttpException The resource cannot be found. Details: Requested URL: /umbraco/members/editMemberType.aspx Replace members/editMemberType.aspx with members/EditMemberType.aspx
Umbraco > Member Group > Save [editMemberGroupSaved] key is not not config/lang/en.xml Deferred for now.
System.Web.HttpException The resource cannot be found. Details: Requested URL: /umbraco/ Replace settings/editDictionaryItem.aspx with settings/EditDictionaryItem.aspx
System.Web.HttpException The resource cannot be found. Details: Requested URL: /umbraco/settings/stylesheet/editStylesheet.aspx Replace stylesheet/editStylesheet.aspx with stylesheet/editstylesheet.aspx
System.Web.HttpException The resource cannot be found. Details: Requested URL: /umbraco/images/developer/userControlIcon.png Replace developer/userControlIcon.png with developer/usercontrolIcon.png
System.Web.HttpException The resource cannot be found. Details: Requested URL: /umbraco/users/editUser.aspx Replace users/editUser.aspx with users/editUser.aspx
Can, amazing progress on the Mono port! I'm using this thread as a base for my own work on the 4.8.1 release. Would you be interested in putting your results so far in a github repo or something similar, so other people can contribute?
Some of the fixes are indeed also applicable to the Windows version, but not all of them. One example is the UmbracoExtensions bug where the Mono compiler treats project references a tiny bit differently than the Windows .NET compiler. It would probably be a better solution to run Mono development in it's own source tree for now, as far as I can tell.
Christian, great work! Yes, we should set something up - either in GitHub on in Codeplex. I am about 2 weeks away from completing on 4.7.2, and the source will then be ready to be uploaded and shared with the community for wider testing and scrutiny. I can give you a sneak preview and say that, at present I have 4.7.2 working in Linux! However, this is with XSLT only. Still working on getting razor support going. A few niggles also remain to be cleared.
Sun, still not sure whether to go for Git or Codeplex. Initially I had thought about keeping a branch under the Umbraco trunk on Codeplex, and this is possible but it would not be as clean as having a separate GitHub project - as we also eventually plan to port Umbraco to OSX.
I am also leaning towards giving the mono development branch its own tree. The present porting model is to use well-known Umbraco milestones to create the Linux / OSX ports. In doing so, we are not always going 100% in sync with the windows version. The next set of articles will also publish one Umbraco bug - that was masked in windows but needed to be fixed in the Linux version. In such cases, it is good to have a separate source tree because it gives us quick response time. On the other hand, if the mono version was in the main trunk - it would always have the latest code - but perhaps not be as stable in mono. These are the tradeoffs I think.
Dealing with dynamic checkboxlists losing their state after postback
This is probably the biggest mono oddity that we have. It could be a bug, but I am classifying it as an oddity. As far as I have found out in the MS specifications, the checkbox state of dynamic checkbox list controls is set during the prerender phase. Hence technically speaking the state of the controls is not available during Page_Load, or validation, which follows page load. While the state is not directly available in the controls, it is available in the viewstate, and can be accessed in the Request collection. We use this principle to provide a fix for lost checkboxlist states during postback. The following examples will illustrate.
(1) Users > Users > Save Error saving user (check log) This error occurs because the "Sections" Checkbox list items lose their selected state. I think this is a mono oddity. In fact, the checkbox list state is intact during postback in Page_Load but is lost during validation, and during page rendering. Here's the fix: it is a little clunky but it does work. In .../umbraco/presentation/umbraco/users/EditUser.aspx.cs, modify sectionValidator_ServerValidate(...), and add after line 218,
setCheckBoxStates(lapps);
Then also add these:
//mono fix for lost checkboxlist states private void setCheckBoxStates(CheckBoxList cbl) { if (IsPostBack) { string cblFormID = cbl.ClientID.Replace("_","$"); int i = 0; foreach (var item in cbl.Items) { string itemSelected = Request.Form[cblFormID + "$" + i]; if (itemSelected != null && itemSelected != String.Empty) ((ListItem)item).Selected = true; i++; } } }
(2) Settings > Document Types > Structure > Allowed child nodetypes Checkboxes state is not retained during save. This is the same issue as above. In .../umbraco/presentation/umbraco/controls/ContentTypeControlNew.aspx insert before line 285,
setCheckBoxStates(lstAllowedContentTypes);
and also add
//mono fix for lost checkboxlist states private void setCheckBoxStates(CheckBoxList cbl) { if (IsPostBack) { string cblFormID = cbl.ClientID.Replace("_","$"); int i = 0; foreach (var item in cbl.Items) { string itemSelected = Request.Form[cblFormID + "$" + i]; if (itemSelected != null && itemSelected != String.Empty) ((ListItem)item).Selected = true; i++; } } }
(3) Settings > Document Types > Info > Allowed Templates Checkboxes state is not retained during save. This looks like a mono oddity. When OnBubbleSave fires in .../umbraco/presentation/umbraco/settings/EditNodeTypeNew.aspx, the state of the checkbox is not available. We use the setCheckBoxStates(CheckBoxList cbl) fix. In .../umbraco/presentation/umbraco/settings/EditNodeTypeNew.aspx insert before line 120,
setCheckBoxStates(templateList);
and also add
//mono fix for lost checkboxlist states
private void setCheckBoxStates(CheckBoxList cbl)
{
if (IsPostBack)
{
string cblFormID = cbl.ClientID.Replace("_","$");
int i = 0;
foreach (var item in cbl.Items)
{
string itemSelected = Request.Form[cblFormID + "$" + i];
if (itemSelected != null && itemSelected != String.Empty)
((ListItem)item).Selected = true;
i++;
}
}
}
I ll close off tonight's set with another list of small fixes:
System.Web.HttpException The resource cannot be found. Details: Requested URL: /umbraco/WebServices/NodeSorter.asmx Replace WebServices/NodeSorter.asmx with webservices/nodeSorter.asmx
System.Web.HttpException The resource cannot be found. Details: Requested URL: /umbraco_client/tableSorting/img/bg.gif Replace tableSorting/img/bg.gif with tablesorting/img/bg.gif
System.Web.HttpException The resource cannot be found. Replace webservices/cmsnode.asmx with webservices/CMSNode.asmx
System.Web.HttpException The resource cannot be found. Replace dialogs/assignDomain.aspx with dialogs/AssignDomain.aspx
System.Web.HttpException The resource cannot be found. Details: Requested URL: /umbraco/dialogs/rollback.aspx Replace dialogs/rollback.aspx with dialogs/rollBack.aspx
System.Web.HttpException The resource cannot be found. Details: Requested URL: /umbraco_client/propertyPane/images/propertyBackground.gif Replace propertyPane/images/propertyBackground.gif with propertypane/images/propertyBackground.gif
System.Web.HttpException The resource cannot be found. Details: Requested URL: /umbraco/dialogs/preview.aspx Replace dialogs/preview.aspx with dialogs/Preview.aspx
Settings > Stylesheets > Save [cssErrorHandler] make sure that you have permissions set correctly. In .../umbraco/presentation/, create the 'css'. Make sure the user (running the web server) has write permissions to it.
I will be putting up the next iteration changes soon. Things have gone a bit slower than expected. There jave been a few Umbraco bugs, and also one old Mono bug, that has been lingering about since a couple of years or so. Also, there are differences in how mono handles XPathNodeIterator. In particular, one must call MoveNext() prior to dereferencing 'Current' - otherwise, Current is always null. This has necessitated touching up large areas of code - in particular with Razor, and library.cs, and it is not quite finished yet. Still working towards the alpha release. Ok that's it for now...
due to casing... In web.config setting, umbracoUseDirectoryUrls to true, produces errors like this: System.Web.HttpException The resource cannot be found. Details: Requested URL: /umbraco/webservices/legacyajaxcalls.asmx Note: the url is in all lower-case. In, .../umbraco/presentation/requestModule.cs lines 127:130, replace occurences of 'path' with 'requestPath' to get this:
And... another Umbraco minority: Content > Any Document > Publish > Publish completes with: System.Configuration.Provider.ProviderException Unrecognized attribute: securityTrimmingEnabled At .../umbraco/presentation/umbraco/nodeFactory/UmbracoSiteMapProvider.cs:47 In file, .../umbraco/presentation/umbraco/nodeFactory/UmbracoSiteMapProvider.cs, after line 38 add:
Content > Any Document > RTE (Rich Text Editor) > enter any text and publish: ...Content is not in a correct format Empty Multiline Textboxes are pre-appended with '\t' everytime an editor control with a Textbox loads, this leads to corrupted RegEx filters, etc. upon saving.
This is a mono rendering bug. After the textbox opening tag is rendered, a new line is issued, and then, System.Web/System.Web.UI/HtmlTextWriter.cs > OutputTabs() renders a single tab leading to corruption. The TextBox, MultiLine generated <textarea> tag increases the Indent count by one. However, this is wrong and when the opening <textarea> tag is closed a tab is issued inside the textarea which is of course rendered.
The culprit is line 214 in .../mono/mcs/class/System.Web/System.Web.UI.WebControls/Textbox.cs. The simple fix is to comment out the line, and re-compile mono. There are also workarounds using a client side script that scrubs the textboxes. For now, I have locally re-compiled mono. But in the alpha 4.7.2 linux release I will probably be doing either a client side script, or a wrapper class that overrides the mono textbox (multiline) behaviour.
RAZOR: Earlier this year, Microsoft made ASP.NET MVC, Web API, and Razor open source, and this source is now also used by mono. However, since we are using the .NET4 implementation, we still need the original MS dlls. Make sure the following dlls are in the bin folder of .../umbraco/presentation/bin (and in the bin folder of the binary app distro) External: System.Web.Razor.dll, System.Web.WebPages.dll, System.Web.WebPages.Razor.dll, Microsoft.Web.Helpers.dll .NET 4 versions Umbraco: From the folder, umbraco.MacroEngines.dll (mono develop will not set a refernce to this to avoid circular references, so you can copy it manually or add a post build step)
Note: If you do not have the 4 external, and umbraco macro dll's in the bin folder, you will get a variety of errors: No umbraco.MacroEngines.dll - language selection is create script dialog is empty No Syste.Web.Razor, System.Web.WebPages, Sysem.Web.WebPages.Razor - error language extension file is not recognised No Microsoft.Web.Helpers - File build failed message (due to CS0234: The type or namespace name `Web' does not exist in the namespace `Microsoft'), this error will only report after applying the 'MoveNext' fix below.
Adding Razor to Scripting: To .../umbraco/presentation/config/umbracoSettings.config, after the developer section, add
<scripting>
<razor>
<!-- razor DynamicNode typecasting detects XML and returns DynamicXml - Root elements that won't convert to DynamicXml -->
<notDynamicXmlDocumentElements>
<element>p</element>
<element>div</element>
<element>ul</element>
<element>span</element>
</notDynamicXmlDocumentElements>
<dataTypeModelStaticMappings>
<!--
<mapping dataTypeGuid="00000000-0000-0000-0000-000000000000">Fully.Qualified.Type.Name.For.ModelBinder,Assembly.Name.Excluding.Dot.Dll</mapping>
<mapping documentTypeAlias="textPage" nodeTypeAlias="propertyAlias">Fully.Qualified.Type.Name.For.ModelBinder,Assembly.Name.Excluding.Dot.Dll</mapping>
<mapping dataTypeGuid="00000000-0000-0000-0000-000000000000" documentTypeAlias="textPage" nodeTypeAlias="propertyAlias">Fully.Qualified.Type.Name.For.ModelBinder,Assembly.Name.Excluding.Dot.Dll</mapping>
<mapping dataTypeGuid="00000000-0000-0000-0000-000000000000" documentTypeAlias="textPage">Fully.Qualified.Type.Name.For.ModelBinder,Assembly.Name.Excluding.Dot.Dll</mapping>
<mapping dataTypeGuid="00000000-0000-0000-0000-000000000000" nodeTypeAlias="propertyAlias">Fully.Qualified.Type.Name.For.ModelBinder,Assembly.Name.Excluding.Dot.Dll</mapping>
<mapping nodeTypeAlias="propertyAlias">Fully.Qualified.Type.Name.For.ModelBinder,Assembly.Name.Excluding.Dot.Dll</mapping>
-->
</dataTypeModelStaticMappings>
</razor>
</scripting>
This is enough to get you to create Razor scripts. Making them run, however, requires more work
XPathNodeIterator MoveNext() issue, and more Razor
Upon trying to save a file under Scripting Files, you get 'Scripting file could not be saved' But the file is saved, And, there is an extra temporary script file left behind. This does not happen when 'skip file testing' is checked. Error occurs in SaveDLRScript(codeEditorSave.asmx.cs 291) and new Node(id) yields null, and breaks things.
This highlights another mono subtlety: in mono, usage for an XPathNodeIterator is: ('MoveNext' issue)
XPathNodeIterator iter = Select (expression);
if (iter.MoveNext ())
return iter.Current;
else
return null;
Where the return type is 'XPathNavigator'. We need to update this for all relevant cases in umbraco. This is because initially the XpathNodeIterator's Position is at '0', and the 'Current' property is always at null.
And, therefore any immediate reference to iter.Current is always null.
First add a function to .../umbraco/businesslogic/xmlHelper.cs:
//this is very mono specific at the moment
public static XmlNode GetCurrentNodeFromIterator(XPathNodeIterator xpi)
{
if (xpi != null)
{
if (xpi.MoveNext())
return ((IHasXmlNode)xpi.Current).GetNode();
}
return null;
}
Next use the following examples for the replacement. ** Loading razor script into rich text editor yields (insert macro): 'Error loading MacroEngine script (file: TestRazorSiteMap.cshtml)' So, in .../umbraco/presentation/umbraco/nodeFactory/Page.cs replace, line 593,
XmlNode n = ((IHasXmlNode)library.GetXmlNodeCurrent().Current).GetNode();
with
XmlNode n = xmlHelper.GetCurrentNodeFromIterator(library.GetXmlNodeCurrent());
** Above saveDLRScript error. So, in .../umbraco/presentation/umbraco/nodeFactory/Page.cs replace, line 298,
_pageXmlNode = ((IHasXmlNode)library.GetXmlNodeById(NodeId.ToString()).Current).GetNode();
with
_pageXmlNode = xmlHelper.GetCurrentNodeFromIterator(library.GetXmlNodeById(NodeId.ToString()));
ExamineBackedMedia.cs
48:51, from,
var media = umbraco.library.GetMedia(id, true);
if (media != null && media.Current != null)
{
media.MoveNext();
return new ExamineBackedMedia(media.Current);
}
to
XPathNodeIterator media = umbraco.library.GetMedia(id, true);
if (media != null)
{
if (media.MoveNext())
return new ExamineBackedMedia(media.Current);
}
Note: we need to type media, otherwise 'MoveNext()' will return false.
62:76, from,
XPathNodeIterator result = xpath.SelectChildren(XPathNodeType.Element);
//add the attributes e.g. id, parentId etc
if (result.Current.HasAttributes)
{
if (result.Current.MoveToFirstAttribute())
{
Values.Add(result.Current.Name, result.Current.Value);
while (result.Current.MoveToNextAttribute())
{
Values.Add(result.Current.Name, result.Current.Value);
}
result.Current.MoveToParent();
}
}
while (result.MoveNext())
to
XPathNodeIterator xpi = xpath.Select(".");
//add the attributes e.g. id, parentId etc
if (xpi.MoveNext())
if (xpi.Current.HasAttributes)
{
if (xpi.Current.MoveToFirstAttribute())
{
Values.Add(xpi.Current.Name, xpi.Current.Value);
while (xpi.Current.MoveToNextAttribute())
{
Values.Add(xpi.Current.Name, xpi.Current.Value);
}
}
}
XPathNodeIterator result = xpath.SelectChildren(XPathNodeType.Element);
while (result.MoveNext())
108:110, from,
if (media != null && media.Current != null)
{
media.MoveNext();
XPathNavigator xpath = media.Current;
to
if (media != null && media.MoveNext())
{
XPathNavigator xpath = media.Current;
370:373, from,
if (media != null && media.Current != null)
{
media.MoveNext();
var children = media.Current.SelectChildren(XPathNodeType.Element);
to
if (media != null && media.MoveNext())
{
var children = media.Current.SelectChildren(XPathNodeType.Element);
DynamicXml.cs, 31:33
from,
if (xpni.Current != null)
{
var xml = xpni.Current.OuterXml;
to
if (xpni.MoveNext())
{
var xml = xpni.Current.OuterXml;
Media Picker not found: System.Web.HttpException The resource cannot be found. Details: Requested URL: /umbraco/dialogs/treepicker.aspx Replace treepicker.aspx with treePicker.aspx
Fixing bugs, which I introduced with the 'MoveNext' issue fixes
Last week's MoveNext() fix actually turns out to be too restrictive and yielded a subtle bug.
Symptom:
Razor Code of this format:
@{
var image = @Model.Media("relatedMedia");
}
<img src=".../@image.UmbracoFile" alt="@image.Name" />
only returns the media path with first application launch. Subsequent calls return null. I introduced this as a bug in the last set up 'MoveNext issue' fixes.
In ExamineBackedMedia.cs change
48:51, from,
if (media != null)
{
if (media.MoveNext())
return new ExamineBackedMedia(media.Current);
to
if (media != null)
{
media.MoveNext();
if (media.Current != null)
return new ExamineBackedMedia(media.Current);
This is because the relevant XPathNodeIterator is cached, and once the index is advanced its state is retained.
I am also going to relax the remaining restrictions involving MoveNext as follows:
Again in ExamineBackedMedia.cs change:
64:67, from,
XPathNodeIterator xpi = xpath.Select(".");
//add the attributes e.g. id, parentId etc
if (xpi.MoveNext())
if (xpi.Current.HasAttributes)
to
XPathNodeIterator xpi = xpath.Select(".");
//add the attributes e.g. id, parentId etc
xpi.MoveNext ();
if (xpi.Current != null)
if (xpi.Current.HasAttributes)
110:113 from,
var media = umbraco.library.GetMedia(this.Id, true);
if (media != null && media.MoveNext())
{
XPathNavigator xpath = media.Current;
...
to
var media = umbraco.library.GetMedia(this.Id, true);
if (media != null)
{
media.MoveNext();
if (media.Current != null)
{
XPathNavigator xpath = media.Current;
var result = xpath.SelectChildren(XPathNodeType.Element);
while (result.MoveNext())
{
if (result.Current != null && !result.Current.HasAttributes)
{
if (string.Equals(result.Current.Name, alias))
{
string value = result.Current.Value;
if (string.IsNullOrEmpty(value))
{
value = result.Current.OuterXml;
}
Values.Add(result.Current.Name, value);
propertyExists = true;
return new PropertyResult(alias, value, Guid.Empty);
}
}
}
}
}
370:373, from,
var media = umbraco.library.GetMedia(ParentId, true);
if (media != null && media.MoveNext())
{
var children = media.Current.SelectChildren(XPathNodeType.Element);
...
to
if (media != null)
{
media.MoveNext();
if (media.Current != null)
{
var children = media.Current.SelectChildren(XPathNodeType.Element);
List mediaList = new List();
while (children != null && children.Current != null)
{
if (children.MoveNext())
{
if (children.Current.Name != "contents")
{
//make sure it's actually a node, not a property
if (!string.IsNullOrEmpty(children.Current.GetAttribute("path", "")) &&
!string.IsNullOrEmpty(children.Current.GetAttribute("id", "")) &&
!string.IsNullOrEmpty(children.Current.GetAttribute("version", "")))
{
mediaList.Add(new ExamineBackedMedia(children.Current));
}
}
}
else
{
break;
}
}
return mediaList;
}
}
DynamicXml.cs, 31:33
from,
if (xpni != null)
{
if (xpni.MoveNext())
{
var xml = xpni.Current.OuterXml;
to
if (xpni != null)
{
xpni.MoveNext();
if (xpni.Current != null)
{
var xml = xpni.Current.OuterXml;
Also change function GetCurrentNodeFromIterator in .../umbraco/businesslogic/xmlHelper.cs to:
//this is very mono specific at the moment
public static XmlNode GetCurrentNodeFromIterator(XPathNodeIterator xpi)
{
if (xpi != null)
{
xpi.MoveNext();
if (xpi.Current != null)
return ((IHasXmlNode)xpi.Current).GetNode();
}
return null;
}
Insert image through RTE: System.NotSupportedException The type System.Collections.Generic.Dictionary`2[[System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089],[System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]] is not supported because it implements IDictionary Description: HTTP 500.Error processing request. Details: Non-web exception. Exception origin (name of application or object): System.Xml. http://127.0.0.1:8080/umbraco/controls/Images/ImageViewerUpdater.asmx/UpdateImage at System.Xml.Serialization.TypeData.get_ListItemType () [0x000cf] in /home/kol3/Development/mono/mcs/class/System.XML/System.Xml.Serialization/TypeData.cs:331 This error can be traced to .../umbraco/presentation/umbraco/controls/images/imageViewer.ascx.cs line 96, which calls umbraco.IO.IOHelper.ResolveUrl(...) in .../umbraco/businesslogic/IO/IOHelper.cs 44:50 In line 49, note the call to: VirtualPathUtility.ToAbsolute(string virtualPath, string AppPath). In the mono implementation AppPath cannot be null. We resolve as follows: .../umbraco/businesslogic/IO/MultiplatformHelper.cs, add
public static string EnsureRootAppPath (string path)
{
if (IsUnix && (path == String.Empty))
return "/";
return path;
}
Then in .../umbraco/businesslogic/IO/IOHelper.cs, change line 49 from,
While we are at it we'll also refactor in as follows (and update all references as needed)
public static bool IsWindows
{
get
{
return OSPlatform.Contains(PLATFORM_WIN);
}
}
public static bool IsUnix
{
get
{
return OSPlatform.Contains(PLATFORM_UNIX);
}
}
Upload png (any image) media: uploaded but not correct format error issued. In .../umbraco/presentation/umbraco/controls/ContentControl.cs, change line 447 from
if (p.PropertyType.ValidationRegExp != "")
to
if (p.PropertyType.ValidationRegExp != null && p.PropertyType.ValidationRegExp != "")
The conditional should work, but mono is converting "" to null, and this casues the conditional to fail and give a false error message.
After publish xml cache is not properly refreshed. The same doctype element is re-added, and for example, an XSLT Menu macro shows too many pages. .../editContent.aspx.cs (315) .../umbraco/presentation/content/content.cs line 392 calls GetElementById -> attr.attr.OwnerElement.IsRooted (604) -> XmlLinkedNode (52) returns false for the published document. This is a mono issue. There is no IsRooted in the MS .net documentation. In content.cs change 393 from,
XmlNode x = xmlContentCopy.GetElementById(id.ToString())
to,
//Deal with IsRooted being false in mono for the published node
string xpathId = UmbracoSettings.UseLegacyXmlSchema ?
String.Format ("//node[@id = '{0}'], id.ToString()") :
String.Format ("//*[@isDoc][@id='{0}']", id.ToString());
XmlNode x = xmlContentCopy.SelectSingleNode(xpathId);
(Not so sure about the legacy syntax...) & did not vet load balancing. As far as I can tell, there is an 'IsRooted' property on Xml documents, which is set to false during the operations in getPreviewOrPublishNode(...), and the consequent call to GetElementById in AppendDocumentXml (content.cs, 393) returns null and breaks the code. That's why we do not use XmlNode x = xmlContentCopy.GetElementById(id.ToString()) here.
After the above fix, xml cache is still not properly refreshed. We now have after publish, one node but non @isDoc children are duplicated. The xml going into the XML Cache file is corrupted. Look at TransferValuesFromDocumentXmlToPublishedXml (323) For some reason, this loop does not loop through all elements. Change from this (328:329):
foreach (XmlNode n in PublishedNode.SelectNodes(xpath))
PublishedNode.RemoveChild(n);
To:
XmlNode[] NodesToRemove =
(new List(PublishedNode.SelectNodes(xpath).OfType())).ToArray();
for (int i = 0; i < NodesToRemove.Length; i++)
PublishedNode.RemoveChild(NodesToRemove[i]);
This uses Linq, and we will need to add this as well: using System.Linq; (+ a reference to System.Core)
I have fixed this throughout the code but this would need to be tested.
content.cs 451:452
from
foreach (XmlNode child in parentNode.SelectNodes(xpath))
parentNode.RemoveChild(child);
to
XmlNode[] NodesToRemove =
(new List(parentNode.SelectNodes(xpath).OfType())).ToArray();
for (int i = 0; i < NodesToRemove.Length; i++)
parentNode.RemoveChild(NodesToRemove[i]);
macro.cs 953:954
from
foreach (XmlNode n in currentNode.SelectNodes("./node"))
currentNode.RemoveChild(n);
to
XmlNode[] NodesToRemove =
(new List(currentNode.SelectNodes("./node").OfType())).ToArray();
for (int i = 0; i < NodesToRemove.Length; i++)
currentNode.RemoveChild(NodesToRemove[i]);
StandardPackageActions.cs 493:500
from
foreach (XmlNode ext in xn.SelectNodes("//ext"))
{
if (ext.Attributes["alias"] != null && ext.Attributes["alias"].Value == _alias)
{
xn.RemoveChild(ext);
inserted = true;
}
}
to
XmlNode[] NodesToRemove =
(new List(xn.SelectNodes("//ext").OfType())).ToArray();
for (int j = 0; j < NodesToRemove.Length; j++)
{
if (NodesToRemove[j].Attributes["alias"] != null && NodesToRemove[j].Attributes["alias"].Value == _alias)
{
xn.RemoveChild(NodesToRemove[j]);
inserted = true;
}
}
StandardPackageActions.cs 588:595
from
foreach (XmlNode node in xn.SelectNodes("//allow"))
{
if (node.Attributes["host"] != null && node.Attributes["host"].Value == hostname)
{
xn.RemoveChild(node);
inserted = true;
}
}
to
XmlNode[] NodesToRemove =
(new List(xn.SelectNodes("//allow").OfType())).ToArray();
for (int j = 0; j < NodesToRemove.Length; j++)
{
if (NodesToRemove[j].Attributes["host"] != null && NodesToRemove[j].Attributes["host"].Value == hostname)
{
xn.RemoveChild(NodesToRemove[j]);
inserted = true;
}
}
Document.cs 1446:1447
from
foreach (XmlNode xDel in x.SelectNodes("./data"))
x.RemoveChild(xDel);
to
XmlNode[] NodesToRemove =
(new List(x.SelectNodes("./data").OfType())).ToArray();
for (int i = 0; i < NodesToRemove.Length; i++)
x.RemoveChild(NodesToRemove[i]);
A quick update on unit testing is in order: of the 100 unit tests in the 4.7.2 solution, 79 are now passing. 5 Fail. And, a further 16 are throwing errors.
Here is a brief overview of what I have done to get tests passing. In most cases, we are dealing with the absence of the HttpContext or Application Domain values that are normally available during the application run time. I won't repeat what I have said earlier in the post, but I am hoping that there is enough stuff here to point the interested reader in the right direction. It is also important that I have gone for quick fixes, and in some instances these come with caveats, such as subtly altering application behaviour.
//umbraco.Test/SetUpUtilities.cs
using System;
using System.Collections.Specialized;
using System.Xml;
using System.Web;
using System.Web.Caching;
using umbraco.BusinessLogic;
namespace umbraco.Test
{
public class SetUpUtilities
{
public SetUpUtilities () {}
private const string _umbracoDbDSN = "server=127.0.0.1;database=UMBRACO_DB;user id=USER_ID;password=PASSWORD;datalayer=MySql";
private const string _umbracoConfigFile = "<PATH-TO-APPLICATION>/config/umbracoSettings.config";
private const string _dynamicBase = "<PATH-TO-ASSEMBLY-CACHE (e.g., /tmp/USER_ID-temp-aspnet-0)";
public static NameValueCollection GetAppSettings()
{
NameValueCollection appSettings = new NameValueCollection();
//add application settings
appSettings.Add("umbracoDbDSN", _umbracoDbDSN);
return appSettings;
}
public static void AddUmbracoConfigFileToHttpCache()
{
XmlDocument temp = new XmlDocument();
XmlTextReader settingsReader = new XmlTextReader(_umbracoConfigFile);
temp.Load(settingsReader);
HttpRuntime.Cache.Insert("umbracoSettingsFile", temp,
new CacheDependency(_umbracoConfigFile));
}
public static void RemoveUmbracoConfigFileFromHttpCache()
{
HttpRuntime.Cache.Remove("umbracoSettingsFile");
}
public static void InitConfigurationManager()
{
ConfigurationManagerService.ConfigManager = new ConfigurationManagerTest(SetUpUtilities.GetAppSettings());
}
public static void InitAppDomainDynamicBase()
{
AppDomain.CurrentDomain.SetDynamicBase(_dynamicBase); //(Obsolete but works...)
//AppDomain.CurrentDomain.SetupInformation.DynamicBase = _dynamicBase;
}
}
}
//sample test file set-up
...
private User m_User;
[TestFixtureSetUp]
public void InitTestFixture()
{
SetUpUtilities.InitConfigurationManager();
m_User = new User(0);
SetUpUtilities.InitAppDomainDynamicBase();
}
[SetUp]
public void MyTestInitialize()
{
SetUpUtilities.AddUmbracoConfigFileToHttpCache();
...
}
[TearDown]
public void MyTestCleanup()
{
...
SetUpUtilities.RemoveUmbracoConfigFileFromHttpCache();
}
...
Language_Get_By_Culture_Code terminates with error:
linq operation is not valid due to the current state of the object
do:
Language.cs (161) Replace SingleOrDefault() with FirstOrDefault()
templates, stylesheet tests fail: create directories (masterpages, css) in test project.
When trying to logout, MySql.Data.MySqlClient.MySqlException You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'WHERE CONTEXTID = '...'' at line 1
In .../umbraco/businesslogic/BasePages/BasePage.cs (237),
change
"DELETE umbracoUserLogins WHERE contextId = @contextId"
to
"DELETE FROM umbracoUserLogins WHERE contextId = @contextId"
Set-up of Test Database Tests are highly dependant on a database being set up with some basic values: One document type with texstring and richtext editor fields. Otherwise some Document tests will fail with id: xxxx not found error.
Some tests are no longer used in the code base, I have commented these out. Here they are:
LanguageTest.cs
Language_Delete_Default_Language() - commented out,
the code it tests is commented out as well, so no need to test.
TaskTypeTest.cs
TaskType_Make_Duplicate
Fails in line 104
This is because in the MySQL version of the CMSTASKTYPE table, there are no
constraints that would throw an SQL exception when a duplicate alias
is inserted. And there are no checks in code. I disable this test for now.
UserTest.cs
User_Make_New_Duplicate_Login
Fails in line 124
This is because in the MySQL version of the UMBRACOUSER table, there are no
constraints that would throw an SQL exception when a duplicate userlogin
is inserted. And there are no checks in code. I disable this test for now.
LanguageTest.cs
Language_Make_Duplicate
Fails becase no SQL exception is thrown
This is because in the MySQL version of the UMBRACOLANGUAGE table, there are no
constraints that would throw an SQL exception when a duplicate languageISOCode
is inserted. And there are no checks in code. I disable this test for now.
There are a few remaining fixes to be applied:
Test: Document_Save_And_Publish_Then_Roll_Back fails
DocumentTest.cs line 287:289
change
Thread.Sleep(1000);
doc.Save();
Assert.IsTrue(doc.HasPendingChanges());
to
Thread.Sleep(3000);
doc.Save();
Assert.IsTrue(doc.HasPendingChanges());
Thread.Sleep(3000);
That is increase the sleep timeout. But the test does still fail on
occassion so needs another look.
RelationTest.cs failures: You have an error in your SQL syntax:
In file .../umbraco/cms/businesslogic/relation/RelationType.cs (131)
change,
"select id, dual, name, alias from umbracoRelationType"
to
"select id, [dual], name, alias from umbracoRelationType"
Also replace in line 44.
Tests using or referencing "GetMemberFromLoginName(string loginName)"
fail because the HttpContext is null. A proper fix for this is lengthy.
For now, we will do this:
In file .../umbraco/cms/businesslogic/member/Member.cs line 286,
change
else
to
else if (HttpContext.Current != null)
LanguageTest.cs
Language_Delete_With_Assigned_Domain
Errors with object reference not set to an instance of an object.
In DocumentTest.cs, change function signature as follows:
From
internal static Document CreateNewUnderRoot(DocumentType dt)
To
internal static Document CreateNewUnderRoot(DocumentType dt, User m_user)
and adjust calls accordingly.
With the above, the port is essentially complete. I have pretty much gone for the quickest solutions where possible - there are not perfect but they get us sufficiently going.
Concerning testing, the approach I have taken is even simpler. It may even be worthwhile, exploring an approach where essential config settings come from the nunit config file.
Also, I applied some hacks to the code to get some tests working. Ideally, we would have re-written the whole application to allow for HttpContext wrapping - but that was beyond the scope of the present porting exercise - perhaps we will do this for the next port.
Next steps: Look for a release of the completed port project - coming soon. The details will be posted here.
I have started deployment testing on Linux kernel 3.0.x / mysql 5 / nginx / fastcgi / mono 2.11. This approach requires that mono 2.11 be downloaded and built from the git repository, and preferably set-up in a parallel environment. There are a couple of errors - which i am investigating at the moment. Hopefully, I ll have them resolved and will be uploading the source in 2 weeks or so. And also post instructions on how to configure the linux server...
@Can, everything path related can (and should) be delegated to the System.IO.Path class, which has a DirectorySeparatorChar field defined, plus utility methods like Combine() for concatenating paths together with the correct, system-specific separator char. Awesome stuff you're doing! :)
@Asbjørn: Thanks for the tip. If I remember correctly, mono's implementation (of System.IO.Path) is slightly different (again if I remember correctly running mono in debug mode mimics windows behaviour, but running in debug mode is not a good choice for production systems), and this had necessitated the additional lines of logic in multiplatform... I ll probably revisit this at some point.
The multiplatform... class at the moment bridges the gap between mono specific and windows specific implementation differences...
The beta-1 release of Umbraco 4.7.2 for mono is now available on github. I am dedicating this release to my uncle Erik Laksberg, with whom I started my computing journey, and who has sadly passed away last night.
The release is fairly stable, and ready for some road testing. If you encounter any issues please submit them on git.
There is one important known issue at this point: Lucene in membership searches do not return anything. Also, there is a mono bug, that will lead to tabs being inserted progressively into any textarea - this is quite annoying but easily fixed in mono itself. However, I am likely to apply a patch for this before we leave the beta phase.
I will add a wiki page that talks about how to set up a site in the near future.
Binaries are not available at the moment, and the solution will only reliably build in debug mode (requires mono > 2.11)
Umbraco 4.7.2 on Linux
Hi again, with U5 abandoned, I have shifted my porting efforts over to the U 4.7.2 branch. I was really hoping to get something done by the end of Codegarden 12, but there was just way too much to do...
However, what I can say now is that I am pretty confident that we will be able to do a good port over to Linux. I have been working on this, and most of the code changes are pretty minor (or, looking back now seem minor) - but there are a fair number of them. In a nutshell, I have a got a partially working version now, and am working towards, let's say an alpha release. It is probably too early to share a version of the code b/c there are still a lot of errors, and it may end up frustrating people.
The good news: In the past two years, since I started on looking at porting Umbraco over to Linux, mono has developed by leaps and bounds - this is now making a successful port possible.
The bad news: Linux is case sensitive. There are a lot of minor changes to be done here, and some discipline to be had in the future in casing conventions. Mono does have the MONO_IOMAP=all option - but this is too slow for any production system. Hence, all casing needs to be vetted for Linux. Mono offers some inspection tools but it is still work, lots of it, and requires consistency for future releases. I had hoped to have a univeral release of Umbraco but we will probably end up with two branches - one for Windows and another for Linux.
The small print: I have had success compiling Umbraco with mono 2.11.2 (using a parallel development environment), and forcing all assemblies to .NET 4.0. I have not had any success under .NET 3.5 - indeed with 3.5 you get a lot of resource not found errors (that are not case related).
The even smaller print: Other than case errors, I am at present dealing with some MYSQL errors, and other unspecified oddities (E.g. as you can get it in London: "The train is delayed, and the cause is unknown, and under investigation")
But the overall outlook remains optimistic. I will check in again or also place an item in my blog
Great work, Hope you can help Umbraco core to make 4.8 with mono.
In the past week, I have been busy getting the unit tests working. Porting the tests over to NUnit was easy enough - just a lot of search and replace, and the results compiled succesfully. Running the tests, however, required some modifications to the core code. The gist of the changes was to be able to inject configuration values, which are available in the .NET runtime, but not necessarily so at times when NUnit is running.
Usually we would put these values in through a mocking mechanism (e.g. NMock2 - which is what I m using). But the Umbraco Web App contains a long static chain, which NMock cannot deal with. An option would have been to use TypeMock - but TypeMock is quite pricey, and in my opinion would have created a high entry price for contributing to the source code and tests.
Having had a quick perusal of the tests, it seemed that the easiest approach was to Interface out the .NET 'ConfigurationManager' class, then using the facade pattern create a couple of derived classes - one for testing and one for live use. To break the static chain, I replaced all references to 'ConfigurationManager' with a Singleton that instantiates the required ConfigurationManager class through dependency injection. This approach kept the changes to the original source to a minimum (hence, minimised the possibility of introducing bugs), but required the additionof the new ConfigManager classes. The approach is quite standart. At some point, it would be nice to revisit the long static chain - and make it non-static, e.g. initialize in global.asax - but that would require pretty big changes.
In sum, using this approach I can make the unit tests pass. At present some units are passing, and I am working through the remaining ones.
Hi Casey,
OK, I ll put up the code on Friday. About the casing issues - which is probably the most tedious aspect, I m in the process of writing a small tool which will run through all the files in the solution, and make sure that the casing of the files matches that of any references to those files in the code. Additionally, the tool will let us apply some casing conventions (e.g. pascal versus camel case, default.aspx vs Default.aspx. That should get us upto speed very quickly. Also this should be far more robust than my manual approach. Exciting stuff...
Hi again Casey, the Umbraco core team, and all Linux / mono fans, let me first of all say how excited I am to be porting the Umbraco CMS over to Linux. We are finally at a stage where this porting can become practical reality. I believe that the Umbraco CMS is one of the best out there, and porting it over to mono / Linux and beyond will be great step for our little CMS.
What follows are a series of posts looking at the porting process. I am going to kick off with methodology and considerations.
Methodology
At this stage I am working on a proof of concept. The idea is to get Umbraco 4.7.2 up and running on Linux in the quickest possible way with the fewest possible changes. The methodology combines Agile techniques with signal extraction techniques - where the released stable Umbraco 4.7.2 source constitutes the signal. I am doing multiple passes (iterations) over the same signal - as it were to tune into the Linux channel. We can say that the Umbraco signal is arriving to the Linux channel in a degraded manner resulting in incomplete reconstitution. Our goal is to create a clean signal that will give us the clean "HD" CMS experience.
As with all signal extraction problems the GIGO, Garbage In - Garbage Out, principle applies. The rationale for choosing 4.7.2 is its high stable quality.
Here are the basic customer stories with the present status (the customer(s) being developer(s).
Considerations
An important consideration is whether when the application becomes multi-platform, will the code for the other platforms be compiled and tested on each target platform? Or whether the code will be compiled and tested only on the source platform? I would recommend that the application is always compiled and tested on each target platform. Msunit, Pex & Moles do not work with mono, and unit tests would have to be ported and run on the target platform.
OK. let's have a look at the individual customer stories in detail.
Getting 4.7.2 compiling on mono
Hmm... Cannot seem to get the numbered lists working... Oh well...
The current stable release of mono is 2.10.x. The next stable version will be 2.12. The current SVN version is 2.11.2. All my builds are against 2.11.2.
Errors resolved:
umbraco/datalayer/SqlHelpers/MySql/SqlResources.resx - Position: Line 123, Column 5.
Inner exception: Could not find a part of the path ".../umbraco/datalayer/SqlHelpers/MySql/sql\total.sql".
Inner exception: Could not find a part of the path ".../umbraco/datalayer/SqlHelpers/MySql/sql\version4_upgrade.sql.sql".
Change all: sql\total.sql references to Sql\Total.sql, Sql\Version4_Upgrade.sql (mono can deal with the '\')
This also happens under the SqlHelpers/SqlServer folder.
umbraco/cms/businesslogic/Packager/FileResources/PackageFiles.resx: Error: Error: Invalid ResX input.
Position: Line 123, Column 5.
Inner exception: Could not find file ".../umbraco/cms/businesslogic/Packager/FileResources/packages.config".
Another casing issue - use Packages.config.
umbraco/presentation/umbraco/templateControls/Resources.resx: Error: Error: Invalid ResX input.
Position: Line 123, Column 5.
Inner exception: Could not find file ".../umbraco/templateControls/inlinexslt.xsltTemplate".
Another casing issue - use InlineXslt.xsltTemplate.
components/SQLCE4Umbraco/SqlCEResources.resx: Error: Error: Invalid ResX input.
Position: Line 123, Column 5.
Inner exception: Could not find a part of the path ".../components/SQLCE4Umbraco/sql\total.sql".
As above Sql\Total.sql
umbraco/presentation/umbraco/Search/ExamineEvents.cs(7,7): Error CS0246: The type or namespace name `Lucene' could not be found. Are you missing a using directive or an assembly reference? (CS0246)
Re-set reference from Shannon's c drive : ) to foreign dlls/Lucene.Net.dll
Edit: I forgot the following...
.../umbraco/actions/delete.aspx.designer.cs(21,21): Error CS0234: The type or namespace name `uicontrols' does not exist in the namespace `umbraco.presentation.umbraco'. Are you missing an assembly reference? (CS0234)
You will get about a 170!! of these - apparently there are some differences in how mono resolves references.
The error is easily fixed: Replace protected umbraco.uicontrols... with protected global::umbraco.uicontrols
Then you might need to do the same for: (about 8-9 occurences)
protected umbraco.controls... to protected global::umbraco.controls.
Compile now builds all projects (Tests ignored at the moment)
OK. let's have a look at the individual customer stories in detail.
Getting the application started and loaded without any 404 Resource XYZ not found errors
Let's have a look at what happens here.
System.IO.FileNotFoundException
Could not load file or assembly 'System.Web.Entity' or one of its dependencies. The system cannot find the file specified.
Remove reference from umbraco.presentation and rebuild.
System.Configuration.ConfigurationErrorsException
Unable to open configSource file '.../umbraco/presentation/config\UrlRewriting.config.
Unfortunately mono cannot deal with the '\' paths in web.config. Set all such paths from '\' to '/'.
System.Web.Configuration.nBrowser.Exception
Parent not found with id = default.
Make sure the .NET 4 default.browser file is available and in the application path.. e.g. in ../umbraco/presentation/App_Browsers
This file is not available in mono, you will need to get it from your windows .NET 4 files.
umbraco.businesslogic.Exceptions.UserAuthorizationException
There was attempt to redirect to '/umbraco/' which is another domain than where you've logged in.
Cross site scripting barrier kicks in. Not sure why - could be a localdomain issue.
Comment out css barrier for now, .../umbraco/presentation/umbraco/login.aspx.cs lines 137:144
Fix is deferred for now
The installer screen does not appear
In web.config set the umbracoConfigurationStatus key to ""
The installer appears but no styles / scripts are loaded.
Change ../umbraco_client/installer references to ../umbraco_client/Installer
Script resources (*.axd) not found.
In web.config change all 3.5.0.0 assembly references to 4.0.0.0
Cannot remember why I did this anymore...
In .../umbraco/presentation/install/default.aspx added references to: System.Web.Configuration, System.Text, and System.Drawing
System.Web.HttpException
The file '/install/steps/defaultuser.ascx' does not exist.
This is the case for these install steps again it is a casing issue
In umbraco.presentation/install/steps/Definitions/*.cs set the UserControl property value casing to match referenced file name casing. I.e., /steps/defaultuser.ascx to /steps/defaultUser.ascx, and /steps/validatepermissions.ascx to /steps/validatePermissions.ascx
Starter Kits step 'No Thanks' image does not show
In umbraco.presentation/install/Skinning/locadStarterKits.aspx line 35 change installer to Installer.
System.InvalidCastException
Cannot cast from source type to destination type at mono/mcs/class/System.Web.Extensions/System.Web.Script.Serialization/JavaScriptSerializer.cs:67
Login panel appears on the left javascript and files do no load
In .../components/umbraco.controls/Panel.cs, replace references to panel/ with Panel/
OK, now we can login.
But... upon logging in you are greeted with: System.InvalidCastException
at System.Web.Script.Serialization.JavaScriptSerializer..ctor in mono/mcs/class/System.Web.Extensions/System.Web.Script.Serialization/JavaScriptSerializer.cs:67
This is a particularly tough error to sort out. The trick is to look at the Application output. System.Web.Extensions is loaded twice as .NET 4 (by the web site) and as .NET 3.5 (by the Our.Umbraco.uGoLive.dll). The latter gains precendence and causes a crash. This is a documented mono bug with workarounds.
You can choose from one of these: (1) recompile the Our.Umbraco.uGoLive.dll in .NET4 but the source is not available, (2) disable Our.Umbraco.uGoLive.dll, or (3) use an assembly redirect to redirect 3.5.0.0 to 4.0.0.0. We will do (3).
Whew, that was something... Now we log in and are greeted by An empty screen with a loading bar
Using Firebug, focus on the GET language.asp 500 internal server error.
The error results because language.asp cannot find the appropriate language xml file to load.
In umbraco/businesslogic/IO/IOHelper.cs change in line 99, path[1] to path[0].
This is a small but significant change and may need revisiting.
The language file now loads but the screen still looks the same
In umbraco.presentation/umbraco/umbraco.aspx, and in umbraco.presentation/umbraco/masterpages/umbracoPage.master change Application/jQuery to Application/JQuery. Do a search and replace for this pattern to correct all solution occurrences.
Now, the UI loads but the right hand side Dasboard panel gives a file not found error
In umbraco/businesslogic/IO/SystemFiles.cs change dashboard.config to Dashboard.config.
In the same file also change these:
xsltExtensions to xsltExtensions
restExtensions to restExtensions
We ll be proactive and clear a few more casing mismatches: In umbraco.presentation/umbraco/umbraco.aspx change all occurrences of UmbracoSpeechBubbleBackend to UmbracoSpeechBubbleBackEnd, and also in the whole solution replace tabView with tabview (for umbraco_client/tabview/images)
Also do a search and replace changing speechBubble_close to speechbubble_close (for the speech bubble close image)
Now we are left with two ScriptResource not found errors, and some Dashboard load errors.
Fix the dashboard
In umbraco.presentation/config/Dashboard.config, replace file references to Pascal casing so as to match the file names.
There are a few images fixed in the same manner: tv.png to TV.png, and listitemorange.gif to listItemOrange.gif, feedproxy.aspx to FeedProxy.aspx
This completes the fixes for the Dashboard. The last two tabs are not in working order, and I will ignore them for now.
SUMMARY
This completes the changes needed to make the interface load. Most of the changes have to do with mismatches between actual file name casing and the hard coded references to those files. There are a number of approaches possible, e.g., all file names can be abstracted into a separate file - where possible. This would help. Fixes do not need to be applied in code, but can be applied to file and folder names - even symbolic links can be used. I have gone the hard way - replacing hard coded values - to illustrate the extent of the issue.
Before we put the UI through its paces, let's look at the TDD aspects of our port.
Getting Unit tests working
Right... After a good nights's sleep, back to the business...
From a TDD perspective, with NUnit & NMock, we cannot escape the conclusion that major parts of the source need to be re-written. In particular, the 4.7.2 source code contains a long static chain - a chain of objects with static variables & functions, that render testing with NUnit & NMock impossible. In general, many testing frameworks do not allow for mocking of static chain elements, and this style of coding has consequently come into dis-favor.
It would be preferable for example to have a very small static application loader class, that instantiates all relevant variables & constants, and initializes data services. In .NET, this could be done in the global.asax file. Then for testing, we isolate all database read / write calls, which are tested separately. The functions that use database services are tested separately using a test data service and mocks. This has the benefit of testing applications functionality and database access in isolation of each other and is especially helpful for diagnosing any multi-platform issues.
Of course, a major re-write of 4.7.2 is beyond the scope of this porting exercise - which is all about porting an existing application with minimal changes. But these issues can be addressed by the next 4.x version.
In this section, I will show what minimum changes are needed to get unit tests working in Linux. The changes rely on some clever minor re-writing of the existing code to inject test values through dependency injection. It is not the prettiest approach, but it works.
Monodevelop does not recognize the Umbraco.Test project
Open the Umbraco.Test.csproj file and delete the {3AC096D0-A1C2-E12C-1390-A8335801FDAB};{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC} line
The Umbraco.Test project will not load in the solution.
Now in the Solution Options > Build Configurations, Configuration Mappings, add Umbraco.Test to the build.
Setting up Nunit tests
Download the NUnit dlls. In the project references, delete the 'Microsoft.VisualStudio.QualityTools.UnitTestFramework, and add nunit.framework.dll
Test Conversion
Using search replace tools, for every test replace 'using Microsoft.VisualStudio.TestTools.UnitTesting;' with 'using NUnit.Framework;'
Using this link convert all test calls.
The tests now compile.
Getting Tests to Pass
All tests run but fail. I have started fixing these tests. Fix 1: establish database access. NUnit does not have access to the .NET Application environment, and calls to Configuration.Appsettings[key] return null. This causes tests requring database access to fail.
Solution: inject database settings.
Code re-writes for injecting test Appsettings
I target the static umbraco/businesslogic/GlobalSettings.cs class DbDSN property (line 139), and replace the static system call with a lazy loading singleton that mimics static behavior:
//return ConfigurationManager.AppSettings["umbracoDbDSN"];
return ConfigurationManagerService.Instance.AppSettings["umbracoDbDSN"];
The ConfigurationManagerService class creates and returns an instance of the default or a test configurator.
In tests we can set the test configurator with: ConfigurationManagerService.ConfigManager = new ConfigurationManagerTest(SetUpUtilities.GetAppSettings());
If the 'ConfigManager' property is null, then the service defaults to the runtime, non-test configurator.
All of this is probably not making much sense, and seeing the whole code will help.
1. First we create an interface for the ConfigurationManager calls that we are using.:
2. Next we create our wrappers (aka Facade pattern)
3. We create the injector
4. We create the singleton
5. So far, we have not modified any existing source code but now we do that.
// .../umbraco/businesslogic/GlobalSettings.cs line 139 public static string DbDSN { get { try { //return ConfigurationManager.AppSettings["umbracoDbDSN"]; return ConfigurationManagerService.Instance.AppSettings["umbracoDbDSN"]; } // ...6. We can use the following quick and dirty test setup (it would be better to read the settings from a file).
7. A simple test example
I will digress a little bit and write a few notes about setting up a Linux test system.
Setting up a Linux test system
1. At present this requires setting up a parallel mono development environment with mono 2.11.2
2. There are great instructions here.
3. Important caveats: if possible avoid using gnome as your window manager. gnome makes heavy use of C# / mono, and you may find that you have to replicate a lot of additional libraries. Actually, avoid using KDE as well, to avoid any potential adverse interactions, use the simplest possible window manager you can find: E.g. openbox, or if you are like me and want your compositing as well, go for Compiz.
4. I found adding the following to the environment setup was necessary:
5. Quick description of the Compiz set-up on Arch Linux:
Install the following:
community/ccsm 0.8.4-3
community/compiz-bcop 0.8.8-2
community/compiz-core 0.8.8-3
community/compiz-fusion-plugins-main 0.8.8-2
community/compiz-manager 0.6.0-5
community/compizconfig-python 0.8.4-4
community/emerald 0.8.8-2
community/emerald-themes 0.6.0-4
community/fusion-icon 1:0.1-1
community/libcompizconfig 0.8.8-2
PS: On Arch Linux with open-vmware-tools you can set in the VMware machine settings, 3D acceleration to yes, and you will get compositing and more.
You can also add archlinux bashrun, and compiz-boxmenu and compiz-deskmenu - though the menu set-up is slightly buggy: I ended up configuring the menu with the compiz-deskmenu tool, and then using that config file with compiz-boxmenu (using option-alt-m to launch the menu through ccsm commands).
Oh, and I m using a pure 32-bit machine:
[mono] ~ @ uname -a Linux 3.4.4-2-ARCH #1 SMP PREEMPT Sun Jun 24 17:28:37 UTC 2012 i686 GNU/Linux [mono] ~ @ mono -V Mono JIT compiler version 2.11.2 (master/a7e01ca Tue Jun 19 21:41:42 BST 2012) Copyright (C) 2002-2012 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com TLS: __thread SIGSEGV: altstack Notifications: epoll Architecture: x86 Disabled: none Misc: softdebug LLVM: supported, not enabled. GC: Included Boehm (with typed GC and Parallel Mark)I have started looking at the UI. This looks like it will be the most challenging aspect of the port. We will be looking at various errors and resolving them. In most instances, there are no clear error messages. But so far errors in the UI have been one of the following: javascript which does not load, casing issues, possible mono bugs or idiosyncracies.
But first an addition to getting the application to load.
After switching all projects to .NET 4, we get:
.../umbraco/cms/businesslogic/member/Member.cs(43,43): Error CS0246: The type or namespace name `MembershipProvider' could not be found. Are you missing a using directive or an assembly reference? (CS0246) (umbraco.cms)
Add References to System.Web.ApplicationServices in umbraco.cms, umbraco.providers
Final ScriptResource not found error.
Traces to js/dualSelectBox.js in umbraco.aspx lines 53::63
We have a CompositeScript control with one script. Remove the CompositeScript tags, i.e. delete lines 54 and 58.
Not sure why this generates an error but I am suspecting a mono issue.
Now let's move on to the UI
UI testing - Document types
General casing issues: replace as follows -
editNodeTypeNew.aspx to EditNodeTypeNew.aspx
settingDataType.gif to settingDatatype.gif
/genericProperties to /GenericProperties
Cannot get to DocumentType edit screen
.../umbraco/settings/EditNodeTypeNew.aspx?id=...
System.Web.HttpException Multiple controls with the same ID 'ctl00' were found. FindControl requires that controls have unique IDs.
EditNodeTypeNew.aspx, Lines 9, 15, and give property panels IDs. Generally if you have multiple controls of the same type on the page make sure you explicitly give them distinct ids. E.g.
No save button image:
editor/save.gif to editor/Save.GIF
Save bubble:
speechbubble/ to speechBubble/
umbraco splash
umbracosplash. to umbracoSplash.
In Info and Structure tabs, list item selection does not persist after save.
This used to be a pre mono 2.6 bug, but the investigation continues
UI Testing - Templates
System.Web.Compilation.ParseException expecting '>'. Got 'asp'
Mono is very particular. Do this - but only for the following sections. .../umbraco/presentation/umbraco/settings/editTemplate.aspx lines 11::244 IF you do it for the whole of the script tag, then inline asp will not execute. This is most likely a mono bug. Search for 'n
I did this too
change .master extensions to .Master - this is tricky. Do not do a global S & R. Do it by hand. Will need to revisit but using .Master for compatibility for now.
UI Testing - Media
airinstallbadge to AIRInstallBadge
This installment is entitled:
"A tale of three paths..."
Of course, we are not talking about an Oscar winning movie, rather our three paths simply are "~", "/", and "\\", the last one also known as @"\".
All of these three (path) characters play a signifcant role in our linux port.
Let's start with "~"
Changing .../umbraco/businesslogic/IO/IOHelpers.cs line 114: path[1] to path[0] got us going but it also introduced
a subtle bug. Here it is:
System.Exception
Could not locate TinyMCE by URI:/umbraco_client/tinymce3/tiny_mce_src.js, Physical path:/umbraco_client/tinymce3/tiny_mce_src.js. Make sure that you configured the installPath to a valid location in your web.config. This path should be an relative or site absolute URI to where TinyMCE is located.
.../umbraco/components/editorControls/tinyMCE3/webcontrol/TinyMCEWebControl.cs:219
Because we had effectively stripped the ~, the above was not mapped properly.
In fact going from path[1] to path[0] changes application behaviour significantly. As "~" in Linux is resolved as the value of $HOME, stripping the "~" character for any IO operation involving the file system makes sense. In windows, .NET operations which involve e.g. loading a file path with "~" in it resolve well. Not so under Linux.
Going from path[1] to path[0], however, did not work in this case:
Content > Trying to load a content page with a rich text editor
System.Exception
Could not locate TinyMCE by URI:/umbraco_client/tinymce3/tiny_mce_src.js, Physical path:/umbraco_client/tinymce3/tiny_mce_src.js. Make sure that you configured the installPath to a valid location in your web.config. This path should be an relative or site absolute URI to where TinyMCE is located.
.../umbraco/components/editorControls/tinyMCE3/webcontrol/TinyMCEWebControl.cs:219
So, now we apply a better fix
(1) In .../umbraco/businesslogic/IO, we have add a new MultiPlatformHelper class:
(2) Then in .../umbraco/businesslogic/IOHelper.cs, modify MapPath as:
This fixes the tinyMCE load error and also makes the "~" path issues clear.
Again, this will keep us for now, but we will probably re-visit it in the future.
Before moving on to the "/" and "\\" characters, let's also fix this error:
System.Web.HttpException
The resource cannot be found.
Details: Requested URL: /umbraco/~/umbraco/js/umbracoUpgradeChecker.js
Path error: should say - /umbraco/js/umbracoUpgradeChecker.js
Solution: in .../umbraco/presentation/umbraco/umbraco.aspx.cs line 104 change
to
Now we are ready to look at the "/" and "\\" character issues. As we know, the path separator in Linux is "/", while in windows it is "\\".
Mono has two built-in mechanisms for dealing with these differences:
(1) When mono is compiled with IS_PORTABILITY_SET, then in the mono CLR, for a unix system, AltDirectorySeparatorChar is defined as "\\". (Otherwise AltDirectorySeparatorChar is defined as "/")
(2) Running the mono app with the MONO_IOMAP=all environment variable set also sets the AltDirectorySeparatorChar to "\\" on Unix systems. However, running Umbraco with this environment variable set is slow and prone to crashes.
I am going to go for a third option, where I make the path character issues explicit in code (with help from the MultiPlatformHelper class):
(3) Resolving path separator character issues in the Umbraco code itself.
The changes so far impact the downloading and installation of packages.
Preliminaries
Developer > Packages
System.IO.DirectoryNotFoundException
Could not find a part of the path ".../umbraco/presentation/App_Data/packages/created/createdPackages.config".
If this directory / file is missing get it from a working install
Developer > Packages
You get this error after clicking "OK" to install a package
System.Web.HttpException
The resource cannot be found.
Details: Requested URL: /umbraco/developer/packages/proxy.htm
Why? Umbraco referrer url is:
http://our.umbraco.org/repo_viewproject?repoguid=65194810-1f85-11dd-bd0b-0800200c9a66&callback=127.0.0.1:8080/umbraco/developer/packages/proxy.htm?/umbraco/developer/packages/installer.aspx?repoguid=65194810-1f85-11dd-bd0b-0800200c9a66&version=v45&fullversion=4.7.2&uselegacyschema=false&dotnetversion=4.0.30319.17020&trustlevel=unrestricted&project_id=8189
Notice the casing.
Solutions: rename local Packages folder to packages, or set a symbolic link.
We ll do the symbolic link for now:
Developer > Packages
System.Exception
Error - file not found. Could not find file named '.../umbraco/presentation/App_Data/packages\...
In .../umbraco/cms/businesslogic/Packager/Repositories/Repository.cs line 209,
Replace
with
return String.Format ("packages{0}{0}{1}.umb", Path.DirectorySeparatorChar, packageGuid);Now we are ready to proceed with the alterations:
Developer > Packages
System.IO.FileNotFoundException
Could not find file ".../umbraco/presentation/App_Data/fc9f3959-3764-4678-a14e-139974cbfe30/package.xml
Decompressed File names look like: 09b6762e-67b1-4cba-b0d9-14652ddcbb30\package
Replace .../umbraco/cms/businesslogic/Packager/Installer.cs line 963:964 with:
Developer > Packages
After installation we find under .../umbraco/presentation, a directory called:
\home\...\umbraco\presentation\, which contains the package files.
In .../umbraco/cms/businesslogic/Packager/Installer.cs
Alter lines 814:815 to
And... a number of small fixes follow:
Contents > TinyMCE Editor
Save and publish image is not found
Search replace saveAndPublish.gif with SaveAndPublish.gif
Developer > xslt files
System.IO.DirectoryNotFoundException
Directory '.../umbraco/presentation/xslt' not found
Solution: Create the missing 'xslt' directory.
Developer > Script Files
System.Web.HttpException
The resource cannot be found.
Details: Requested URL: /umbraco/developer/python/editPython.aspx
Replace python/editPython.aspx with Python/editPython.aspx
Developer > Cache Browser
System.Web.HttpException
The resource cannot be found.
Details: Requested URL: /umbraco/developer/cache/viewCacheItem.aspx
Replace cache/viewCacheItem.aspx with Cache/viewCacheItem.aspx
Developer > Data Types
System.Web.HttpException
The resource cannot be found.
Details: Requested URL: /umbraco/developer/datatypes/editDataType.aspx
Replace datatypes/editDataType.aspx with DataTypes/editDatatype.aspx
Developer > Macros
System.IO.DirectoryNotFoundException
Directory '.../umbraco/presentation/usercontrols' not found.
Create the usercontrols directory
Developer > Macros
System.Web.HttpException
The resource cannot be found.
Details: Requested URL: /umbraco/developer/macros/editMacro.aspx
Replace macros/editMacro.aspx with Macros/editMacro.aspx
Developer > Packages
System.Web.HttpException
The resource cannot be found.
Details: ../packages/...
Replace developer/packages with developer/Packages
Compilation Error
.../umbraco/presentation/umbraco/developer/Packages/editPackages.aspx
Replace protected umbraco.uicontrols with protected global::umbraco.uicontrols
Developer > Packages
System.Web.HttpException
The resource cannot be found.
Details: Requested URL: /umbraco_client/images/progressBar.gif
Replace /images/progressBar.gif with /images/progressbar.gif
And, in .../umbraco/presentation/umbraco/dialogs/emptyTrashcan.aspx.designer.cs
Replace umbraco.uicontrols with global::umbraco.uicontrols
Developer > Packages
Uninstall does not delete .dll's and added directories.
But installed packages file etc are updated.
Defer for now.
Developer > Macro > Create produces:
System.IO.DirectoryNotFoundException
Directory '.../umbraco/presentation/umbraco/xslt/templates/schema2' not found.
Replace schema2 with Schema2
Replace clean.xslt with Clean.xslt
Developer > Macro
System.Web.HttpException
The resource cannot be found.
Details: Requested URL: /umbraco/developer/xslt/editXslt.aspx
Replace xslt/editXslt.aspx with Xslt/editXslt.aspx
Also replace
insField.GIF with insField.gif
insMemberItem.GIF with insMemberItem.gif
insChildTemplateNew.GIF with insChildTemplateNew.gif
insFieldByLevel.GIF with insFieldByLevel.gif
xslVisualize.GIF with xslVisualize.gif
Members > Member Groups
System.Web.HttpException
The resource cannot be found.
Details: Requested URL: /umbraco/members/editMemberGroup.aspx
Replace members/editMemberGroup.aspx with members/EditMemberGroup.aspx
Replace membergroup.gif with memberGroup.gif
Replace membertype.gif with memberType.gif
System.Web.HttpException
The resource cannot be found.
Details: Requested URL: /umbraco/members/viewMembers.aspx
Replace members/viewMembers.aspx with members/ViewMembers.aspx
System.Web.HttpException
The resource cannot be found.
Details: Requested URL: /umbraco/members/editMember.aspx
Replace members/editMember.aspx with members/EditMember.aspx, then editMember.aspx with EditMember.aspx
Then in search.aspx.designer.cs
Replace umbraco.uicontrols with global::umbraco.uicontrols
We are continuing right on... In the last post, I had made some changes to IOHelper.MapPath(...).
After the IOHelper.MapPath modification, .../umbraco/presentation/umbraco/dashboard/FeedProxy.aspx no longer loads feeds.
Modify line 27 from,
to
The xmlHelper class already uses IOHelper.MapPath, and calling it twice produces an incorect path.
This probably can be fixed in other ways as well. But we will do this for now.
There are a couple of other important changes, Here they are:
Umbraco > Members > EditMembers > Save
Umbraco > Media > editMedia > Save
System.MethodAccessException
Method `umbraco.cms.businesslogic.web.Document:FireBeforeSave (umbraco.cms.businesslogic.SaveEventArgs)' is inaccessible from method `umbraco.controls.ContentControl:saveClick (object,System.Web.UI.ImageClickEventArgs)
The error originates in line 236 of .../umbraco/presentation/umbraco/controls/ContentControl.cs
this indeed breaks protection. In .../umbraco/cms/businesslogic/web/Document.cs line 1884, we have,
change this to,
We may need to revisit this.
The other issue involves the '¤' character, which is used as a string splitter.Mono is quite fussy about how this string is defined. Here's what works in mono.
Settings > Scripts > Create Folder
Index out of bounds error
This is due to issues with "¤" string splitting.
In file .../umbraco/presentation/umbraco/create/script.ascx.cs, modify line 40 as:
char c = '\u00A4'; string returnUrl = presentation.create.dialogHandler_temp.Create( helper.Request("nodeType"), createFolder, relativepath + c.ToString() + rename.Text + c.ToString() + scriptType.SelectedValue);and in file .../umbraco/presentation/umbraco/create/ScriptTasks.cs, modify line 44 as:
char c = '\u00A4'; string[] scriptFileAr = _alias.Split(c);And another set of smaller fixes follow:
Umbraco > Members
Could not load control: '/umbraco/Members/MemberSearch.ascx
replace Members/MemberSearch.ascx with members/MemberSearch.ascx
System.Web.HttpException
The resource cannot be found.
Details: Requested URL: /umbraco/settings/editMediaType.aspx
Replace settings/editMediaType.aspx with settings/EditMediaType.aspx
System.Web.HttpException
The resource cannot be found.
Details: Requested URL: /umbraco/members/editMemberType.aspx
Replace members/editMemberType.aspx with members/EditMemberType.aspx
Umbraco > Member Group > Save
[editMemberGroupSaved] key is not not config/lang/en.xml
Deferred for now.
System.Web.HttpException
The resource cannot be found.
Details: Requested URL: /umbraco/
Replace settings/editDictionaryItem.aspx with settings/EditDictionaryItem.aspx
System.Web.HttpException
The resource cannot be found.
Details: Requested URL: /umbraco/settings/stylesheet/editStylesheet.aspx
Replace stylesheet/editStylesheet.aspx with stylesheet/editstylesheet.aspx
System.Web.HttpException
The resource cannot be found.
Details: Requested URL: /umbraco/images/developer/userControlIcon.png
Replace developer/userControlIcon.png with developer/usercontrolIcon.png
System.Web.HttpException
The resource cannot be found.
Details: Requested URL: /umbraco/users/editUser.aspx
Replace users/editUser.aspx with users/editUser.aspx
Can, amazing progress on the Mono port! I'm using this thread as a base for my own work on the 4.8.1 release. Would you be interested in putting your results so far in a github repo or something similar, so other people can contribute?
If this do not effect Umbraco on IIS, why not combine it into core?
Some of the fixes are indeed also applicable to the Windows version, but not all of them. One example is the UmbracoExtensions bug where the Mono compiler treats project references a tiny bit differently than the Windows .NET compiler. It would probably be a better solution to run Mono development in it's own source tree for now, as far as I can tell.
Hi guys,
Christian, great work! Yes, we should set something up - either in GitHub on in Codeplex. I am about 2 weeks away from completing on 4.7.2, and the source will then be ready to be uploaded and shared with the community for wider testing and scrutiny. I can give you a sneak preview and say that, at present I have 4.7.2 working in Linux! However, this is with XSLT only. Still working on getting razor support going. A few niggles also remain to be cleared.
Sun, still not sure whether to go for Git or Codeplex. Initially I had thought about keeping a branch under the Umbraco trunk on Codeplex, and this is possible but it would not be as clean as having a separate GitHub project - as we also eventually plan to port Umbraco to OSX.
I am also leaning towards giving the mono development branch its own tree. The present porting model is to use well-known Umbraco milestones to create the Linux / OSX ports. In doing so, we are not always going 100% in sync with the windows version. The next set of articles will also publish one Umbraco bug - that was masked in windows but needed to be fixed in the Linux version. In such cases, it is good to have a separate source tree because it gives us quick response time. On the other hand, if the mono version was in the main trunk - it would always have the latest code - but perhaps not be as stable in mono. These are the tradeoffs I think.
Dealing with dynamic checkboxlists losing their state after postback
This is probably the biggest mono oddity that we have. It could be a bug, but I am classifying it as an oddity. As far as I have found out in the MS specifications, the checkbox state of dynamic checkbox list controls is set during the prerender phase. Hence technically speaking the state of the controls is not available during Page_Load, or validation, which follows page load. While the state is not directly available in the controls, it is available in the viewstate, and can be accessed in the Request collection. We use this principle to provide a fix for lost checkboxlist states during postback. The following examples will illustrate.
(1)
Users > Users > Save
Error saving user (check log)
This error occurs because the "Sections" Checkbox list items lose their selected state.
I think this is a mono oddity. In fact, the checkbox list state is intact during
postback in Page_Load but is lost during validation, and during page rendering.
Here's the fix: it is a little clunky but it does work.
In .../umbraco/presentation/umbraco/users/EditUser.aspx.cs, modify sectionValidator_ServerValidate(...), and add after line 218,
Then also add these:
(2)
Settings > Document Types > Structure > Allowed child nodetypes Checkboxes state is not retained during save. This is the same issue as above. In .../umbraco/presentation/umbraco/controls/ContentTypeControlNew.aspx insert before line 285,
and also add
(3)
Settings > Document Types > Info > Allowed Templates
Checkboxes state is not retained during save.
This looks like a mono oddity. When OnBubbleSave fires in
.../umbraco/presentation/umbraco/settings/EditNodeTypeNew.aspx,
the state of the checkbox is not available. We use the setCheckBoxStates(CheckBoxList cbl) fix.
In .../umbraco/presentation/umbraco/settings/EditNodeTypeNew.aspx insert before line 120,
and also add
//mono fix for lost checkboxlist states private void setCheckBoxStates(CheckBoxList cbl) { if (IsPostBack) { string cblFormID = cbl.ClientID.Replace("_","$"); int i = 0; foreach (var item in cbl.Items) { string itemSelected = Request.Form[cblFormID + "$" + i]; if (itemSelected != null && itemSelected != String.Empty) ((ListItem)item).Selected = true; i++; } } }I ll close off tonight's set with another list of small fixes:
System.Web.HttpException
The resource cannot be found.
Details: Requested URL: /umbraco/WebServices/NodeSorter.asmx
Replace WebServices/NodeSorter.asmx with webservices/nodeSorter.asmx
System.Web.HttpException
The resource cannot be found.
Details: Requested URL: /umbraco_client/tableSorting/img/bg.gif
Replace tableSorting/img/bg.gif with tablesorting/img/bg.gif
System.Web.HttpException
The resource cannot be found.
Replace webservices/cmsnode.asmx with webservices/CMSNode.asmx
System.Web.HttpException
The resource cannot be found.
Replace dialogs/assignDomain.aspx with dialogs/AssignDomain.aspx
System.Web.HttpException
The resource cannot be found.
Details: Requested URL: /umbraco/dialogs/rollback.aspx
Replace dialogs/rollback.aspx with dialogs/rollBack.aspx
System.Web.HttpException
The resource cannot be found.
Details: Requested URL: /umbraco_client/propertyPane/images/propertyBackground.gif
Replace propertyPane/images/propertyBackground.gif with propertypane/images/propertyBackground.gif
System.Web.HttpException
The resource cannot be found.
Details: Requested URL: /umbraco/dialogs/preview.aspx
Replace dialogs/preview.aspx with dialogs/Preview.aspx
Settings > Stylesheets > Save
[cssErrorHandler] make sure that you have permissions set correctly.
In .../umbraco/presentation/, create the 'css'.
Make sure the user (running the web server) has write permissions to it.
A quick update:
I will be putting up the next iteration changes soon. Things have gone a bit slower than expected. There jave been a few Umbraco bugs, and also one old Mono bug, that has been lingering about since a couple of years or so. Also, there are differences in how mono handles XPathNodeIterator. In particular, one must call MoveNext() prior to dereferencing 'Current' - otherwise, Current is always null. This has necessitated touching up large areas of code - in particular with Razor, and library.cs, and it is not quite finished yet. Still working towards the alpha release. Ok that's it for now...
An Umbraco bug that affects Linux
due to casing...
In web.config setting, umbracoUseDirectoryUrls to true, produces errors like this: System.Web.HttpException
The resource cannot be found.
Details: Requested URL: /umbraco/webservices/legacyajaxcalls.asmx
Note: the url is in all lower-case.
In, .../umbraco/presentation/requestModule.cs lines 127:130, replace occurences of
'path' with 'requestPath' to get this:
if (asmxPos >= 0) context.RewritePath(requestPath.Substring(0, asmxPos + 5), requestPath.Substring(asmxPos + 5), context.Request.QueryString.ToString());And... another Umbraco minority:
Content > Any Document > Publish > Publish completes with:
System.Configuration.Provider.ProviderException
Unrecognized attribute: securityTrimmingEnabled
At .../umbraco/presentation/umbraco/nodeFactory/UmbracoSiteMapProvider.cs:47
In file, .../umbraco/presentation/umbraco/nodeFactory/UmbracoSiteMapProvider.cs,
after line 38 add:
config.Remove("securityTrimmingEnabled");And now... For a mono bug
Content > Any Document > RTE (Rich Text Editor) > enter any text and publish:
...Content is not in a correct format
Empty Multiline Textboxes are pre-appended with '\t' everytime an editor control with a Textbox
loads, this leads to corrupted RegEx filters, etc. upon saving.
This is a mono rendering bug. After the textbox opening tag is rendered, a new line is issued, and then,
System.Web/System.Web.UI/HtmlTextWriter.cs > OutputTabs() renders a single tab leading to corruption.
The TextBox, MultiLine generated <textarea> tag increases the Indent count by one. However, this is wrong and
when the opening <textarea> tag is closed a tab is issued inside the textarea which is of course rendered.
The culprit is line 214 in .../mono/mcs/class/System.Web/System.Web.UI.WebControls/Textbox.cs.
The simple fix is to comment out the line, and re-compile mono. There are also workarounds using a client side
script that scrubs the textboxes. For now, I have locally re-compiled mono.
But in the alpha 4.7.2 linux release I will probably be doing either a client side script, or a wrapper class that overrides the
mono textbox (multiline) behaviour.
Adding Razor Support
RAZOR: Earlier this year, Microsoft made ASP.NET MVC, Web API, and Razor open source, and this source is now also used by mono. However, since we are using the .NET4 implementation, we still need the original MS dlls. Make sure the following dlls are in the bin folder of .../umbraco/presentation/bin (and in the bin folder of the binary app distro)
External: System.Web.Razor.dll, System.Web.WebPages.dll, System.Web.WebPages.Razor.dll, Microsoft.Web.Helpers.dll .NET 4 versions Umbraco: From the folder, umbraco.MacroEngines.dll (mono develop will not set a refernce to this to avoid circular references, so you can copy it manually or add a post build step)
Note: If you do not have the 4 external, and umbraco macro dll's in the bin folder, you will get a variety of errors:
No umbraco.MacroEngines.dll - language selection is create script dialog is empty
No Syste.Web.Razor, System.Web.WebPages, Sysem.Web.WebPages.Razor - error language extension file is not recognised
No Microsoft.Web.Helpers - File build failed message (due to CS0234: The type or namespace name `Web' does not exist in the namespace `Microsoft'), this error will only report after applying the 'MoveNext' fix below.
Adding Razor to Scripting:
To .../umbraco/presentation/config/umbracoSettings.config,
after the developer section, add
<scripting> <razor> <!-- razor DynamicNode typecasting detects XML and returns DynamicXml - Root elements that won't convert to DynamicXml --> <notDynamicXmlDocumentElements> <element>p</element> <element>div</element> <element>ul</element> <element>span</element> </notDynamicXmlDocumentElements> <dataTypeModelStaticMappings> <!-- <mapping dataTypeGuid="00000000-0000-0000-0000-000000000000">Fully.Qualified.Type.Name.For.ModelBinder,Assembly.Name.Excluding.Dot.Dll</mapping> <mapping documentTypeAlias="textPage" nodeTypeAlias="propertyAlias">Fully.Qualified.Type.Name.For.ModelBinder,Assembly.Name.Excluding.Dot.Dll</mapping> <mapping dataTypeGuid="00000000-0000-0000-0000-000000000000" documentTypeAlias="textPage" nodeTypeAlias="propertyAlias">Fully.Qualified.Type.Name.For.ModelBinder,Assembly.Name.Excluding.Dot.Dll</mapping> <mapping dataTypeGuid="00000000-0000-0000-0000-000000000000" documentTypeAlias="textPage">Fully.Qualified.Type.Name.For.ModelBinder,Assembly.Name.Excluding.Dot.Dll</mapping> <mapping dataTypeGuid="00000000-0000-0000-0000-000000000000" nodeTypeAlias="propertyAlias">Fully.Qualified.Type.Name.For.ModelBinder,Assembly.Name.Excluding.Dot.Dll</mapping> <mapping nodeTypeAlias="propertyAlias">Fully.Qualified.Type.Name.For.ModelBinder,Assembly.Name.Excluding.Dot.Dll</mapping> --> </dataTypeModelStaticMappings> </razor> </scripting>This is enough to get you to create Razor scripts. Making them run, however, requires more work
XPathNodeIterator MoveNext() issue, and more Razor
Upon trying to save a file under Scripting Files, you get 'Scripting file could not be saved'
But the file is saved,
And, there is an extra temporary script file left behind.
This does not happen when 'skip file testing' is checked.
Error occurs in SaveDLRScript(codeEditorSave.asmx.cs 291) and new Node(id) yields null, and breaks things.
This highlights another mono subtlety: in mono, usage for an XPathNodeIterator is:
('MoveNext' issue)
XPathNodeIterator iter = Select (expression); if (iter.MoveNext ()) return iter.Current; else return null;Where the return type is 'XPathNavigator'. We need to update this for all relevant cases in umbraco.
This is because initially the XpathNodeIterator's Position is at '0', and the 'Current' property is always at null.
And, therefore any immediate reference to iter.Current is always null.
First add a function to .../umbraco/businesslogic/xmlHelper.cs:
//this is very mono specific at the moment public static XmlNode GetCurrentNodeFromIterator(XPathNodeIterator xpi) { if (xpi != null) { if (xpi.MoveNext()) return ((IHasXmlNode)xpi.Current).GetNode(); } return null; }Next use the following examples for the replacement.
** Loading razor script into rich text editor yields (insert macro):
'Error loading MacroEngine script (file: TestRazorSiteMap.cshtml)'
So, in .../umbraco/presentation/umbraco/nodeFactory/Page.cs replace, line 593,
** Above saveDLRScript error. So, in .../umbraco/presentation/umbraco/nodeFactory/Page.cs replace, line 298,
** Total Changes: 14
p.aspx.cs, 42, 46
library.cs, 96, 1207, 1266, 1340
baseHttpModule.cs, 182
UmbracoPage.cs, 22
Page_Legacy.cs, 219, 269, 510
Page.cs, 247, 298, 591
** RAZOR RELATED: **
ExamineBackedMedia.cs 48:51, from, var media = umbraco.library.GetMedia(id, true); if (media != null && media.Current != null) { media.MoveNext(); return new ExamineBackedMedia(media.Current); } to XPathNodeIterator media = umbraco.library.GetMedia(id, true); if (media != null) { if (media.MoveNext()) return new ExamineBackedMedia(media.Current); } Note: we need to type media, otherwise 'MoveNext()' will return false.62:76, from, XPathNodeIterator result = xpath.SelectChildren(XPathNodeType.Element); //add the attributes e.g. id, parentId etc if (result.Current.HasAttributes) { if (result.Current.MoveToFirstAttribute()) { Values.Add(result.Current.Name, result.Current.Value); while (result.Current.MoveToNextAttribute()) { Values.Add(result.Current.Name, result.Current.Value); } result.Current.MoveToParent(); } } while (result.MoveNext()) to XPathNodeIterator xpi = xpath.Select("."); //add the attributes e.g. id, parentId etc if (xpi.MoveNext()) if (xpi.Current.HasAttributes) { if (xpi.Current.MoveToFirstAttribute()) { Values.Add(xpi.Current.Name, xpi.Current.Value); while (xpi.Current.MoveToNextAttribute()) { Values.Add(xpi.Current.Name, xpi.Current.Value); } } } XPathNodeIterator result = xpath.SelectChildren(XPathNodeType.Element); while (result.MoveNext())108:110, from, if (media != null && media.Current != null) { media.MoveNext(); XPathNavigator xpath = media.Current; to if (media != null && media.MoveNext()) { XPathNavigator xpath = media.Current;370:373, from, if (media != null && media.Current != null) { media.MoveNext(); var children = media.Current.SelectChildren(XPathNodeType.Element); to if (media != null && media.MoveNext()) { var children = media.Current.SelectChildren(XPathNodeType.Element);DynamicXml.cs, 31:33 from, if (xpni.Current != null) { var xml = xpni.Current.OuterXml; to if (xpni.MoveNext()) { var xml = xpni.Current.OuterXml;And... one for the road...
Media Picker not found:
System.Web.HttpException
The resource cannot be found.
Details: Requested URL: /umbraco/dialogs/treepicker.aspx
Replace treepicker.aspx with treePicker.aspx
Fixing bugs, which I introduced with the 'MoveNext' issue fixes
Last week's MoveNext() fix actually turns out to be too restrictive and yielded a subtle bug.
Symptom:
Razor Code of this format:
@{ var image = @Model.Media("relatedMedia"); } <img src=".../@image.UmbracoFile" alt="@image.Name" />only returns the media path with first application launch. Subsequent calls return null. I introduced this as a bug in the last set up 'MoveNext issue' fixes.
In ExamineBackedMedia.cs change 48:51, from, if (media != null) { if (media.MoveNext()) return new ExamineBackedMedia(media.Current); to if (media != null) { media.MoveNext(); if (media.Current != null) return new ExamineBackedMedia(media.Current);This is because the relevant XPathNodeIterator is cached, and once the index is advanced its state is retained.
I am also going to relax the remaining restrictions involving MoveNext as follows:
Again in ExamineBackedMedia.cs change: 64:67, from, XPathNodeIterator xpi = xpath.Select("."); //add the attributes e.g. id, parentId etc if (xpi.MoveNext()) if (xpi.Current.HasAttributes) to XPathNodeIterator xpi = xpath.Select("."); //add the attributes e.g. id, parentId etc xpi.MoveNext (); if (xpi.Current != null) if (xpi.Current.HasAttributes) 110:113 from, var media = umbraco.library.GetMedia(this.Id, true); if (media != null && media.MoveNext()) { XPathNavigator xpath = media.Current; ... to var media = umbraco.library.GetMedia(this.Id, true); if (media != null) { media.MoveNext(); if (media.Current != null) { XPathNavigator xpath = media.Current; var result = xpath.SelectChildren(XPathNodeType.Element); while (result.MoveNext()) { if (result.Current != null && !result.Current.HasAttributes) { if (string.Equals(result.Current.Name, alias)) { string value = result.Current.Value; if (string.IsNullOrEmpty(value)) { value = result.Current.OuterXml; } Values.Add(result.Current.Name, value); propertyExists = true; return new PropertyResult(alias, value, Guid.Empty); } } } } } 370:373, from, var media = umbraco.library.GetMedia(ParentId, true); if (media != null && media.MoveNext()) { var children = media.Current.SelectChildren(XPathNodeType.Element); ... to if (media != null) { media.MoveNext(); if (media.Current != null) { var children = media.Current.SelectChildren(XPathNodeType.Element); List mediaList = new List(); while (children != null && children.Current != null) { if (children.MoveNext()) { if (children.Current.Name != "contents") { //make sure it's actually a node, not a property if (!string.IsNullOrEmpty(children.Current.GetAttribute("path", "")) && !string.IsNullOrEmpty(children.Current.GetAttribute("id", "")) && !string.IsNullOrEmpty(children.Current.GetAttribute("version", ""))) { mediaList.Add(new ExamineBackedMedia(children.Current)); } } } else { break; } } return mediaList; } } DynamicXml.cs, 31:33 from, if (xpni != null) { if (xpni.MoveNext()) { var xml = xpni.Current.OuterXml; to if (xpni != null) { xpni.MoveNext(); if (xpni.Current != null) { var xml = xpni.Current.OuterXml;Also change function GetCurrentNodeFromIterator in .../umbraco/businesslogic/xmlHelper.cs to:
//this is very mono specific at the moment public static XmlNode GetCurrentNodeFromIterator(XPathNodeIterator xpi) { if (xpi != null) { xpi.MoveNext(); if (xpi.Current != null) return ((IHasXmlNode)xpi.Current).GetNode(); } return null; }Some Media related issues
Insert image through RTE: System.NotSupportedException The type System.Collections.Generic.Dictionary`2[[System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089],[System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]] is not supported because it implements IDictionary
Description: HTTP 500.Error processing request.
Details: Non-web exception. Exception origin (name of application or object): System.Xml. http://127.0.0.1:8080/umbraco/controls/Images/ImageViewerUpdater.asmx/UpdateImage at System.Xml.Serialization.TypeData.get_ListItemType () [0x000cf] in /home/kol3/Development/mono/mcs/class/System.XML/System.Xml.Serialization/TypeData.cs:331
This error can be traced to .../umbraco/presentation/umbraco/controls/images/imageViewer.ascx.cs line 96,
which calls umbraco.IO.IOHelper.ResolveUrl(...) in .../umbraco/businesslogic/IO/IOHelper.cs 44:50
In line 49, note the call to: VirtualPathUtility.ToAbsolute(string virtualPath, string AppPath).
In the mono implementation AppPath cannot be null.
We resolve as follows: .../umbraco/businesslogic/IO/MultiplatformHelper.cs, add
public static string EnsureRootAppPath (string path) { if (IsUnix && (path == String.Empty)) return "/"; return path; }Then in .../umbraco/businesslogic/IO/IOHelper.cs, change line 49 from,
to
While we are at it we'll also refactor in as follows (and update all references as needed)
public static bool IsWindows { get { return OSPlatform.Contains(PLATFORM_WIN); } } public static bool IsUnix { get { return OSPlatform.Contains(PLATFORM_UNIX); } }Upload png (any image) media: uploaded but not correct format error issued.
In .../umbraco/presentation/umbraco/controls/ContentControl.cs, change line 447
from
to
The conditional should work, but mono is converting "" to null, and this casues the conditional to fail and give a false error message.
Issues relating to XML Caching - 1
After publish xml cache is not properly refreshed. The same doctype element is re-added, and for example, an XSLT Menu macro shows too many pages.
.../editContent.aspx.cs (315)
.../umbraco/presentation/content/content.cs line 392 calls GetElementById
-> attr.attr.OwnerElement.IsRooted (604)
-> XmlLinkedNode (52) returns false for the published document.
This is a mono issue. There is no IsRooted in the MS .net documentation.
In content.cs change
393 from,
XmlNode x = xmlContentCopy.GetElementById(id.ToString()) to, //Deal with IsRooted being false in mono for the published node string xpathId = UmbracoSettings.UseLegacyXmlSchema ? String.Format ("//node[@id = '{0}'], id.ToString()") : String.Format ("//*[@isDoc][@id='{0}']", id.ToString()); XmlNode x = xmlContentCopy.SelectSingleNode(xpathId);(Not so sure about the legacy syntax...) & did not vet load balancing. As far as I can tell, there is an 'IsRooted' property on Xml documents, which is set to false during the operations in getPreviewOrPublishNode(...), and the consequent call to GetElementById in AppendDocumentXml (content.cs, 393) returns null and breaks the code. That's why we do not use XmlNode x = xmlContentCopy.GetElementById(id.ToString()) here.
Issues relating to XML Caching - 2
After the above fix, xml cache is still not properly refreshed. We now have after publish, one node but non @isDoc children are duplicated.
The xml going into the XML Cache file is corrupted.
Look at TransferValuesFromDocumentXmlToPublishedXml (323)
For some reason, this loop does not loop through all elements.
Change from this (328:329):
foreach (XmlNode n in PublishedNode.SelectNodes(xpath)) PublishedNode.RemoveChild(n); To: XmlNode[] NodesToRemove = (new List(PublishedNode.SelectNodes(xpath).OfType())).ToArray(); for (int i = 0; i < NodesToRemove.Length; i++) PublishedNode.RemoveChild(NodesToRemove[i]);This uses Linq, and we will need to add this as well: using System.Linq; (+ a reference to System.Core)
I have fixed this throughout the code but this would need to be tested.
content.cs 451:452 from foreach (XmlNode child in parentNode.SelectNodes(xpath)) parentNode.RemoveChild(child); to XmlNode[] NodesToRemove = (new List(parentNode.SelectNodes(xpath).OfType())).ToArray(); for (int i = 0; i < NodesToRemove.Length; i++) parentNode.RemoveChild(NodesToRemove[i]); macro.cs 953:954 from foreach (XmlNode n in currentNode.SelectNodes("./node")) currentNode.RemoveChild(n); to XmlNode[] NodesToRemove = (new List(currentNode.SelectNodes("./node").OfType())).ToArray(); for (int i = 0; i < NodesToRemove.Length; i++) currentNode.RemoveChild(NodesToRemove[i]); StandardPackageActions.cs 493:500 from foreach (XmlNode ext in xn.SelectNodes("//ext")) { if (ext.Attributes["alias"] != null && ext.Attributes["alias"].Value == _alias) { xn.RemoveChild(ext); inserted = true; } } to XmlNode[] NodesToRemove = (new List(xn.SelectNodes("//ext").OfType())).ToArray(); for (int j = 0; j < NodesToRemove.Length; j++) { if (NodesToRemove[j].Attributes["alias"] != null && NodesToRemove[j].Attributes["alias"].Value == _alias) { xn.RemoveChild(NodesToRemove[j]); inserted = true; } } StandardPackageActions.cs 588:595 from foreach (XmlNode node in xn.SelectNodes("//allow")) { if (node.Attributes["host"] != null && node.Attributes["host"].Value == hostname) { xn.RemoveChild(node); inserted = true; } } to XmlNode[] NodesToRemove = (new List(xn.SelectNodes("//allow").OfType())).ToArray(); for (int j = 0; j < NodesToRemove.Length; j++) { if (NodesToRemove[j].Attributes["host"] != null && NodesToRemove[j].Attributes["host"].Value == hostname) { xn.RemoveChild(NodesToRemove[j]); inserted = true; } } Document.cs 1446:1447 from foreach (XmlNode xDel in x.SelectNodes("./data")) x.RemoveChild(xDel); to XmlNode[] NodesToRemove = (new List(x.SelectNodes("./data").OfType())).ToArray(); for (int i = 0; i < NodesToRemove.Length; i++) x.RemoveChild(NodesToRemove[i]);Progress on unit testing
A quick update on unit testing is in order: of the 100 unit tests in the 4.7.2 solution, 79 are now passing. 5 Fail. And, a further 16 are throwing errors.
Here is a brief overview of what I have done to get tests passing. In most cases, we are dealing with the absence of the HttpContext or Application Domain values that are normally available during the application run time. I won't repeat what I have said earlier in the post, but I am hoping that there is enough stuff here to point the interested reader in the right direction. It is also important that I have gone for quick fixes, and in some instances these come with caveats, such as subtly altering application behaviour.
//umbraco.Test/SetUpUtilities.cs using System; using System.Collections.Specialized; using System.Xml; using System.Web; using System.Web.Caching; using umbraco.BusinessLogic; namespace umbraco.Test { public class SetUpUtilities { public SetUpUtilities () {} private const string _umbracoDbDSN = "server=127.0.0.1;database=UMBRACO_DB;user id=USER_ID;password=PASSWORD;datalayer=MySql"; private const string _umbracoConfigFile = "<PATH-TO-APPLICATION>/config/umbracoSettings.config"; private const string _dynamicBase = "<PATH-TO-ASSEMBLY-CACHE (e.g., /tmp/USER_ID-temp-aspnet-0)"; public static NameValueCollection GetAppSettings() { NameValueCollection appSettings = new NameValueCollection(); //add application settings appSettings.Add("umbracoDbDSN", _umbracoDbDSN); return appSettings; } public static void AddUmbracoConfigFileToHttpCache() { XmlDocument temp = new XmlDocument(); XmlTextReader settingsReader = new XmlTextReader(_umbracoConfigFile); temp.Load(settingsReader); HttpRuntime.Cache.Insert("umbracoSettingsFile", temp, new CacheDependency(_umbracoConfigFile)); } public static void RemoveUmbracoConfigFileFromHttpCache() { HttpRuntime.Cache.Remove("umbracoSettingsFile"); } public static void InitConfigurationManager() { ConfigurationManagerService.ConfigManager = new ConfigurationManagerTest(SetUpUtilities.GetAppSettings()); } public static void InitAppDomainDynamicBase() { AppDomain.CurrentDomain.SetDynamicBase(_dynamicBase); //(Obsolete but works...) //AppDomain.CurrentDomain.SetupInformation.DynamicBase = _dynamicBase; } } }//sample test file set-up ... private User m_User; [TestFixtureSetUp] public void InitTestFixture() { SetUpUtilities.InitConfigurationManager(); m_User = new User(0); SetUpUtilities.InitAppDomainDynamicBase(); } [SetUp] public void MyTestInitialize() { SetUpUtilities.AddUmbracoConfigFileToHttpCache(); ... } [TearDown] public void MyTestCleanup() { ... SetUpUtilities.RemoveUmbracoConfigFileFromHttpCache(); } ...//couple of hacks... .../umbraco/businesslogic/IO/MultiPlatformHelper.cs, public static string MapUnixPath(string path) { string filePath = path; if (filePath.StartsWith("~")) filePath = IOHelper.ResolveUrl(filePath); filePath = IOHelper.MapPath(filePath, System.Web.HttpContext.Current != null); return filePath; } .../umbraco/businesslogic/IO/IOHelper.cs, private static string getRootDirectorySafe() { if (!String.IsNullOrEmpty(m_rootDir)) { return m_rootDir; } string baseDirectory = System.IO.Path.GetDirectoryName(System.Reflection.Assembly.GetExecutingAssembly().CodeBase.Substring(8)); m_rootDir = baseDirectory.Substring(0, baseDirectory.LastIndexOf("bin") - 1); //changed for tests ck, 9/9/12 if (MultiPlatformHelper.IsUnix && !m_rootDir.StartsWith(IOHelper.DirSepChar.ToString())) m_rootDir = IOHelper.DirSepChar + m_rootDir; return m_rootDir; }Resolving Issues Logging Out
When trying to logout,
MySql.Data.MySqlClient.MySqlException
You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'WHERE CONTEXTID = '...'' at line 1
In .../umbraco/businesslogic/BasePages/BasePage.cs (237), change "DELETE umbracoUserLogins WHERE contextId = @contextId" to "DELETE FROM umbracoUserLogins WHERE contextId = @contextId"Getting tests working
Set-up of Test Database
Tests are highly dependant on a database being set up with some basic values:
One document type with texstring and richtext editor fields.
Otherwise some Document tests will fail with id: xxxx not found error.
Some tests are no longer used in the code base, I have commented these out. Here they are:
There are a few remaining fixes to be applied:
Test: Document_Save_And_Publish_Then_Roll_Back fails DocumentTest.cs line 287:289 change Thread.Sleep(1000); doc.Save(); Assert.IsTrue(doc.HasPendingChanges()); to Thread.Sleep(3000); doc.Save(); Assert.IsTrue(doc.HasPendingChanges()); Thread.Sleep(3000); That is increase the sleep timeout. But the test does still fail on occassion so needs another look.RelationTest.cs failures: You have an error in your SQL syntax: In file .../umbraco/cms/businesslogic/relation/RelationType.cs (131) change, "select id, dual, name, alias from umbracoRelationType" to "select id, [dual], name, alias from umbracoRelationType" Also replace in line 44.Tests using or referencing "GetMemberFromLoginName(string loginName)" fail because the HttpContext is null. A proper fix for this is lengthy. For now, we will do this: In file .../umbraco/cms/businesslogic/member/Member.cs line 286, change else to else if (HttpContext.Current != null)LanguageTest.cs Language_Delete_With_Assigned_Domain Errors with object reference not set to an instance of an object. In DocumentTest.cs, change function signature as follows: From internal static Document CreateNewUnderRoot(DocumentType dt) To internal static Document CreateNewUnderRoot(DocumentType dt, User m_user) and adjust calls accordingly.We now have 96 tests and they are all passing
State of the port update
With the above, the port is essentially complete. I have pretty much gone for the quickest solutions where possible - there are not perfect but they get us sufficiently going.
Concerning testing, the approach I have taken is even simpler. It may even be worthwhile, exploring an approach where essential config settings come from the nunit config file.
Also, I applied some hacks to the code to get some tests working. Ideally, we would have re-written the whole application to allow for HttpContext wrapping - but that was beyond the scope of the present porting exercise - perhaps we will do this for the next port.
Next steps: Look for a release of the completed port project - coming soon. The details will be posted here.
A quick update
I have started deployment testing on Linux kernel 3.0.x / mysql 5 / nginx / fastcgi / mono 2.11. This approach requires that mono 2.11 be downloaded and built from the git repository, and preferably set-up in a parallel environment. There are a couple of errors - which i am investigating at the moment. Hopefully, I ll have them resolved and will be uploading the source in 2 weeks or so. And also post instructions on how to configure the linux server...
@Can: Why on earth aren't you on the core team?
@Can, everything path related can (and should) be delegated to the System.IO.Path class, which has a DirectorySeparatorChar field defined, plus utility methods like Combine() for concatenating paths together with the correct, system-specific separator char. Awesome stuff you're doing! :)
Just got back from a short vacation...
@Niels: That would be great!
@Asbjørn: Thanks for the tip. If I remember correctly, mono's implementation (of System.IO.Path) is slightly different (again if I remember correctly running mono in debug mode mimics windows behaviour, but running in debug mode is not a good choice for production systems), and this had necessitated the additional lines of logic in multiplatform... I ll probably revisit this at some point.
The multiplatform... class at the moment bridges the gap between mono specific and windows specific implementation differences...
Beta-1 Release now available
The beta-1 release of Umbraco 4.7.2 for mono is now available on github. I am dedicating this release to my uncle Erik Laksberg,
with whom I started my computing journey, and who has sadly passed away last night.
The release is fairly stable, and ready for some road testing. If you encounter any issues please submit them on git.
There is one important known issue at this point: Lucene in membership searches do not return anything.
Also, there is a mono bug, that will lead to tabs being inserted progressively into any textarea - this is quite annoying but easily fixed in mono itself. However, I am likely to apply a patch for this before we leave the beta phase.
I will add a wiki page that talks about how to set up a site in the near future.
Binaries are not available at the moment, and the solution will only reliably build in debug mode (requires mono > 2.11)
#meGoesBowingInRespect
Thank you.
Hope to have a a couple of web sites up and running on Linux / Umbraco in 2-3 weeks time. : )
is working on a reply...