With V7 we could fairly easy re-use some core tests in order to set up stubbed content from XML files. We'd override BaseXyzTest.GetXmlContent, reset PublishedContentType.GetPublishedContentTypeCallback and then stub up some content types. It made it real easy to have a common doctype setup for your tests, and then have content for scenarios in files with the tests.
Deserializing SolidPublishedContent from JSON or XML, then configuring content types with the PublishedContentTypeFactory seems like a very nice way to work. I imagine we could even copy that JSON out of the nucache db?
However, the entire set of components is internal. Our only option as far as I can see for now is to rely solely on mocks or our own fakes of everything. I can't see why this whole shebang can't just be public, so we can enjoy the same effectiveness of testing as the core does.
I do realize we still have the option of doing it all through the "UnitTesting.Adapter" thing, but it'll take hours to figure out how to make a public API for everything in the adapter. Fleshing out a nice test-API in the core could be done by flipping a few access modifiers and then maturing it.
I just realized I'm ignoring DB-integration, but I'm after a nice middle-way where we can stub out everything behind the cache.
After all, Umbraco is a content management system and one of the hardest things still is actually testing the published content. ><
If you look at our tests you'll see that most of them still use the Xml cache for tests because we haven't yet ported them to NuCache - and then, using a specific cache implementation for testing smells.
Then, you've mentioned SolidPublishedContent but there are other classes too that kinda, partially, mock or implement IPublishedContent. In fact, way too many of them. I would love to streamline this.
There are a couple of tests that run on top of a mocked cache--will try to find them and point you to them. The "vision" is to align all tests on that pattern. Which remains to be ... fully defined. Should the mocked "thing" be loaded from Json, or defined via a fluent interface, or?
While we're at it: IPublishedContent has a PublishedContentType which is not an interface, and that has been identified by others as a problem when mocking. Currently considering introducing IPublishedContentType for 8.1.
Cool. Happy with me opening a PR for continued discussion?
Not sure about the scope of work around this yet, and have to decide quickly wrt. CG. :)
Reg. code: I don't really mind whether it's SolidPublishedContent or something else, but preferably there's be a nice public *serializable class, and preferably it'd be named just PublishedContent. It would not serialize as any typed model, and the adapter thing we do in v7 is still in place. (?)
Unless the type(s) that are used to stuff things into the btree database can be made public and serializable. The one(s) Warren uses in the cache viewer? Would be perfect to copy stuff from there and pasting into resources in the test project.
Regarding PublishedContentType and PublishedPropertyType I did something in the examples / support class I use. It resides in the adapter assembly so it can at least use internals. But preferably, there'd be public serializable versions of these too.
However, the latter I'm happy to leave up to fluent configuration instead of serialization for tests. You'll have the same document structure, but most likely different content in your different tests. Hence it makes sense to keep doctypes in one config class.
The flip side of making a fluent API for content- and property types is that it could be brought into the core and used as a "baseline" for new sites, migrations, comparisons etc.
But that's out of scope of this discussion, beyond minding the possibility. :)
PS. Reg. internal etc. Note all the references to public serializable with regards to stubbing test-data. :P You can choose files, json, xml, code, database, whatever for your stub data. Things that actually do IO and/or have IO dependencies can be internal / private as long as they have nice public interfaces exchanging public serializable objects.
(That does not go for IContent which is in a more privateish bounded context, possibly benefiting from IO integration tests)
Opening a task and a corresponding PR may be the easiest way to discuss the changes we can make, yes. IPublishedContentType and IPublishedPropertyTypewill be part of 8.1.0.
Then, there are 2 phases: one that impacts the Core or Web DLL and therefore is "sensitive", especially if backward compatibility is at stake -- and one that impacts the Tests DLL and ... there we can do what we want.
Publicize and/or make API for stubbing of published content
With V7 we could fairly easy re-use some core tests in order to set up stubbed content from XML files. We'd override
BaseXyzTest.GetXmlContent, resetPublishedContentType.GetPublishedContentTypeCallbackand then stub up some content types. It made it real easy to have a common doctype setup for your tests, and then have content for scenarios in files with the tests.I have some examples of doing that here:
https://github.com/lars-erik/umbraco-unit-testing-samples/blob/master/Umb.Testing.Tests/SampleUmbracoTests/ContentCacheFromXml.cs
It looks to me that these tests are very similar and could go a long way to facilitate such a mechanism:
https://github.com/umbraco/Umbraco-CMS/tree/v8/dev/src/Umbraco.Tests/PublishedContent
Deserializing
SolidPublishedContentfrom JSON or XML, then configuring content types with thePublishedContentTypeFactoryseems like a very nice way to work. I imagine we could even copy that JSON out of the nucache db?However, the entire set of components is internal. Our only option as far as I can see for now is to rely solely on mocks or our own fakes of everything. I can't see why this whole shebang can't just be public, so we can enjoy the same effectiveness of testing as the core does. I do realize we still have the option of doing it all through the "UnitTesting.Adapter" thing, but it'll take hours to figure out how to make a public API for everything in the adapter. Fleshing out a nice test-API in the core could be done by flipping a few access modifiers and then maturing it.
I just realized I'm ignoring DB-integration, but I'm after a nice middle-way where we can stub out everything behind the cache.
After all, Umbraco is a content management system and one of the hardest things still is actually testing the published content. ><
Thoughts?
Short version: totally agree, now, time.
If you look at our tests you'll see that most of them still use the Xml cache for tests because we haven't yet ported them to NuCache - and then, using a specific cache implementation for testing smells.
Then, you've mentioned
SolidPublishedContentbut there are other classes too that kinda, partially, mock or implementIPublishedContent. In fact, way too many of them. I would love to streamline this.There are a couple of tests that run on top of a mocked cache--will try to find them and point you to them. The "vision" is to align all tests on that pattern. Which remains to be ... fully defined. Should the mocked "thing" be loaded from Json, or defined via a fluent interface, or?
While we're at it:
IPublishedContenthas aPublishedContentTypewhich is not an interface, and that has been identified by others as a problem when mocking. Currently considering introducingIPublishedContentTypefor 8.1.Cool. Happy with me opening a PR for continued discussion?
Not sure about the scope of work around this yet, and have to decide quickly wrt. CG. :)
Reg. code: I don't really mind whether it's
SolidPublishedContentor something else, but preferably there's be a nice public *serializable class, and preferably it'd be named justPublishedContent. It would not serialize as any typed model, and the adapter thing we do in v7 is still in place. (?) Unless the type(s) that are used to stuff things into the btree database can be made public and serializable. The one(s) Warren uses in the cache viewer? Would be perfect to copy stuff from there and pasting into resources in the test project.Regarding
PublishedContentTypeandPublishedPropertyTypeI did something in the examples / support class I use. It resides in the adapter assembly so it can at least use internals. But preferably, there'd be public serializable versions of these too.However, the latter I'm happy to leave up to fluent configuration instead of serialization for tests. You'll have the same document structure, but most likely different content in your different tests. Hence it makes sense to keep doctypes in one config class.
The flip side of making a fluent API for content- and property types is that it could be brought into the core and used as a "baseline" for new sites, migrations, comparisons etc.
But that's out of scope of this discussion, beyond minding the possibility. :)
PS. Reg.
internaletc. Note all the references to public serializable with regards to stubbing test-data. :P You can choose files, json, xml, code, database, whatever for your stub data. Things that actually do IO and/or have IO dependencies can be internal / private as long as they have nice public interfaces exchanging public serializable objects.(That does not go for
IContentwhich is in a more privateish bounded context, possibly benefiting from IO integration tests)Opening a task and a corresponding PR may be the easiest way to discuss the changes we can make, yes.
IPublishedContentTypeandIPublishedPropertyTypewill be part of 8.1.0.Then, there are 2 phases: one that impacts the Core or Web DLL and therefore is "sensitive", especially if backward compatibility is at stake -- and one that impacts the Tests DLL and ... there we can do what we want.
Maybe worth focussing on the first phase first?
PR created. Got stuck on
UmbracoTestAttribute.https://github.com/umbraco/Umbraco-CMS/pull/5325
Should I create a task as well? (Rephrased: What is this github artifact I know nothing about?)
The story continues: https://github.com/umbraco/Umbraco-CMS/issues/10106
is working on a reply...