One of the things we're looking at improving in the core is easier querying of content for people who're not .net developers. So this isn't a discussion on code first, mappers or anything like that.
We're aware that while making a simple query or outputting a simple value, might be easy - it can become tricky when you want to go a little further.
Before we (hq/core) spend too much time on improving querying experience, I'd love to gather some sets of scenarios that we can use as test cases and this is where I ask for your input (if you're in the target group mentioned above). So please leave one or more comments answering:
"When have you tried to query for data, what did you want to output
and where did you got lost or stuck?".
The more detail the better. It could be a screenshot of your tree and the data you wanted to display (and from what page), it could be which lorem ipsum you wanted to replace in a HTML comp, etc.
The more, the better. In return I promise we'll do what we can to make it simpler in the future.
This isn't an example where I got lost or stuck, but it might be useful to add anyway even though I'm a .net developer.
In this scenario I need to fetch children from a node and the children could be different document types, but they all have the same document type parent. So I need to check all document types in the query and also check their parents. With the current way of querying that could be a bit hard. I once did it on an Umbraco 4.5 website with Linq to Umbraco. This was before we knew about the performance problems and I even had Linq to Umbraco in my Level 2 Masterclass. Here is a slightly modified query example:
//Use LinqToUmbraco to get all the BaseVisuals subitems per chamber.
//BaseVisuals is the base class/documenttype for many different classes/documenttypes used in Umbraco, because of this each subitem in the chamber has the required properties.
//Example of some classes/documenttypes which inherit from BaseVisuals: Employee, Project, Content.
from baseVisualNode in chamberNode.Children.OfType<BaseVisuals>()
orderby baseVisualNode.SortOrder
select new Overview()
{
Id = baseVisualNode.Id,
Title = baseVisualNode.Title,
Orientation = baseVisualNode.Orientation,
Image = baseVisualNode.Image, Class = baseVisualNode.Active ? "active" : null
}
With Linq to Umbraco all generated document type classes inherited from it's document type parent so .OfType<BaseVisuals>() just worked. This is something that could be improved for the current querying. I already talked with Stephane about this and he said it could be solved with the "new cache" he's working on, but like you said this isn't about mappers or anything. I hope this example is a bit useful.
One of the simplest and most common things to do is to get a list of nodes from a specific part of the content tree (a list of News Items, for example).
If you've structured your content well often times you can use CurrentPage.Children and that seems straight forward for most in my experience.
What isn't so easy is when you have to get that same list of nodes News Item nodes but you aren't on the News Area (the parent document type of all News Items). This is when people resort to Umbraco.Content(id) and then use .Children. Using the ID is convenient and performant but not resilient/flexible.
A better way would be to go to a known location and use the document type structure to lead you to where you want to end up. This works very well no matter where you want to use the code, but there are problems for non-programmers (and programmers too sometimes). First, concept of walking up the tree to a known location and then guiding step-by-step to the correct area isn't what people naturally start with. Second, the query ends up being quite long. While a programmer might call it "expressive" a non-programmer would call it confusing:
Questions such as, "What does the (1) mean?" and "How is that different than .First()?" and "When do I or don't I need () at the end?" abound. Until you've learned these things it is confusing. Not that it is difficult to teach the "how and why" of querying but it's a learning curve and a place where people get frustrated.
For sites that don't follow a best practice arrangement of document types and structure settings and content tree organization the situation is worse. (think about your own early sites and how poorly they were architected... that only makes the challenge of querying even more difficult) In this case you can't let the content structure work in your favor and end up having to use levels and .Descendants()too often. Again, this is fragile/inflexible and also bad for performance as sites grow larger and larger. Again, the use of Umbraco.Content(id) is common with content trees and doctypes that aren't structured as well as they might be.
In short, the number one challenge for querying is to reliably and quickly select all the pages that exist below some other page (such as all the News Items below the News Area page).
A related challenge for querying is how to get to a 'Config' section of the content tree. Imagine this content arrangement:
CONTENT
- Home
- - Products
- - News
- - Support
- - About
- Config
- - Promos
- - Widgets
Selecting various Promos is a challenge. Even if you learn the approach of going up hill to a known location and striking out from there, that won't work in this case because you can't do .Ancestor(0) but have to know about Umbraco.ContentAtRoot().
While writing my previous post, I thought about how big the disconnect is for those who don't know/understand the underlying document types of a site to add a bit of logic to their markup (either in-line or via a Partial View).
For those people who aren't masters of their sites' implementation details, a query might best be handled by using the names of the pages in the content tree. Using the content tree above as an example, to get all the news items, perhaps something along the lines of:
CONTENT.Home.News.Children
Not sure how to denote that we're using page names rather than doctype aliases but you get the idea.
True, this kind of things is fairly easily broken... change the name of the 'Home' page to something else and the query breaks. But then again, usually the top level pages that define the main parts of the sitemap are defined early in a site's life and don't change over time. The easy of querying by name might be appropriate in many cases?
Just a wild 'outside the box' bit of thinking to consider.
I have been thinking about this a bit today from the point of view is it the language? or is it exposure to how to even begin with it?
So maybe its not that the query language is hard, or that its broken, but it maybe that it is not easy to discover or get started with it unless you are taught.
An idea I have been playing with in my mind is the idea of a query builder, to allow users to form queries an almost spoken way so the query reads like a sentence so it is easily read but then when you insert it into the template it outputs the correctly formed razor query (either Dynamic or Strong Typed).
I quite like how Campaign Monitor allow you to create segmentation of mailing lists. Its a simple series of options, that allow you to perform what under the covers would be some fairly complex data-mining. As seen below.
So the idea would be to provide a similar editor in Umbraco to help users create queries without needing to know the language of the query. The user will form the query using common language so the query would read like a sentence. Once the query is inserted into the template, or partial the query would be as usual but the user would have an entry point to learn the querying language.
Maybe add some extra objects (and introduce some more conventions) that provide a clearer context
Breadcrumb
Navigation
In these make some choices on what is used most for these kinds of things like a navigation could be all visible children of the homepage, breadcrumb all the ancestors...
TopNavigation
Could be one that fetches the items from a multi node picker (adding a convention for the alias umbracoTopNav)
FooterNavigation
Same for this one (so fetches items from multi node picker that uses a convention for the alias umbracoFooterNav)
So all you would need to know then is the name of the object and how to loop the collection
@foreach(var page in Navigation) {
<li><a href="@page.Url">@page.Name</a></li>
}
So more conventions... that makes it easier to start and has all the basics covered
I like the idea by Douglas Robar to use the path of the node in queries -> CONTENT.Home.News.Children
Of course the implications of changing the name or location of a node could be horrendous. Suddenly the site no longer works because some node is moved or renamed.
From the top of my head there are a couple of strategies to help the
user
Warn: you are about to change something which is used in query x
Forbid: you cannot change this node because you have no rights
Adapt: query is changed automatically with rename or move. Deleting of nodes is not possible
Codefirst: by making the query the node is created.
Exotic feature indeed ;-)
Still I think this would not be outside the realm of possibility. And it touches on a bigger issue namely the true marriage between content and code. Is a documenttype for instance code or content? Customers want it to be content (I like to add a subsubtitle to doctype x now this very instance) and .Net devs want it to be code (don't touch it because it will not correspond with the code I have in SVN )
I really like the idea proposed by Peter. having a query builder built into the back office or somewhere else easily accessible & obvious would be very helpful. Things like the razor cheatsheet are great but are very hit and miss for novice users like myself if you don't have a basic grounding in .net
I guess in some ways this concept is relatively similar to what Chriztian Steinmeier did for xslt and xpath with his axes visualizer. http://pimpmyxslt.com/axesviz.aspx that helped me no end when xslt and xpath seemed like a big unknown to me.
It just makes life so much easier for beginners or front end devs who are often more visually orientated to be able to genertate a query from a simple tree diagram. at the very least it lets you tinker with shit to see how it works.
I think it's fine and probably even a good thing that there are multiple ways to acheive things in Umbraco, but coming to the mvc/razor way fresh, it can be confusing when some queries look visually very different to others (esp the lambada syntax really confuses me, whenever I see a x => x.Whatever I know its time to tune out... thats probably just me though) Or the fact that this changed: @Model.Content.Stuff in v6+ as opposed to @Model.Stuff (v4) why?
If some things could be sitting under the hood it would be less confusing for newbies and I think having a visual query-builder helps acheive that as you can make sure it sets a standard way of doing things as a base.
one more thing I've always wondered why you cant just do this: @Model.Content.Children("ThisParticularDocType") rather than @Model.Content.Children.Where(x => x.DocumentTypeAlias == "ThisParticularDocType") or @Model.Content.Children.Where("DocumentTypeAlias == \"ThisParticularDocType\"")
I'm very much in favour of change if it means making the template language easier for those of us with less technical chops
I fully agree with you Tim regarding you last points around the query language. Obviously we are now quite far down the track and if anything changes in this regard both old and updated ways need to work (for a time).
Further to my thoughts the ability to test a query right in the back office (See the old XLST visualizer)would give the user much more confidence in building queries and testing queries as they will instantly see if it returns what they need. This is kinda like the problem most devs have with RegEx, one of the most random and spaz lingos I have ever had to deal with but as soon as I found regExBuddy I was confident I could get it to do what I needed it to do, as I got instant feedback on whether or not I was hitting my matches.
Imagine a query builder that instantly updated and gave you the result in the visualizer as you built it. You learn as it constructs based on rules you give it.
Feedback needed for improving query experience
One of the things we're looking at improving in the core is easier querying of content for people who're not .net developers. So this isn't a discussion on code first, mappers or anything like that.
We're aware that while making a simple query or outputting a simple value, might be easy - it can become tricky when you want to go a little further.
Before we (hq/core) spend too much time on improving querying experience, I'd love to gather some sets of scenarios that we can use as test cases and this is where I ask for your input (if you're in the target group mentioned above). So please leave one or more comments answering:
The more detail the better. It could be a screenshot of your tree and the data you wanted to display (and from what page), it could be which lorem ipsum you wanted to replace in a HTML comp, etc.
The more, the better. In return I promise we'll do what we can to make it simpler in the future.
This isn't an example where I got lost or stuck, but it might be useful to add anyway even though I'm a .net developer.
In this scenario I need to fetch children from a node and the children could be different document types, but they all have the same document type parent. So I need to check all document types in the query and also check their parents. With the current way of querying that could be a bit hard. I once did it on an Umbraco 4.5 website with Linq to Umbraco. This was before we knew about the performance problems and I even had Linq to Umbraco in my Level 2 Masterclass. Here is a slightly modified query example:
//Use LinqToUmbraco to get all the BaseVisuals subitems per chamber. //BaseVisuals is the base class/documenttype for many different classes/documenttypes used in Umbraco, because of this each subitem in the chamber has the required properties. //Example of some classes/documenttypes which inherit from BaseVisuals: Employee, Project, Content. from baseVisualNode in chamberNode.Children.OfType<BaseVisuals>() orderby baseVisualNode.SortOrder select new Overview() { Id = baseVisualNode.Id, Title = baseVisualNode.Title, Orientation = baseVisualNode.Orientation, Image = baseVisualNode.Image,Class = baseVisualNode.Active ? "active" : null }
With Linq to Umbraco all generated document type classes inherited from it's document type parent so .OfType<BaseVisuals>() just worked. This is something that could be improved for the current querying. I already talked with Stephane about this and he said it could be solved with the "new cache" he's working on, but like you said this isn't about mappers or anything. I hope this example is a bit useful.
Jeroen
One of the simplest and most common things to do is to get a list of nodes from a specific part of the content tree (a list of News Items, for example).
If you've structured your content well often times you can use
CurrentPage.Childrenand that seems straight forward for most in my experience.What isn't so easy is when you have to get that same list of nodes News Item nodes but you aren't on the News Area (the parent document type of all News Items). This is when people resort to
Umbraco.Content(id)and then use.Children. Using the ID is convenient and performant but not resilient/flexible.A better way would be to go to a known location and use the document type structure to lead you to where you want to end up. This works very well no matter where you want to use the code, but there are problems for non-programmers (and programmers too sometimes). First, concept of walking up the tree to a known location and then guiding step-by-step to the correct area isn't what people naturally start with. Second, the query ends up being quite long. While a programmer might call it "expressive" a non-programmer would call it confusing:
CurrentPage.AncestorOrSelf(1).NewsAreas.First().ChildrenQuestions such as, "What does the
(1)mean?" and "How is that different than.First()?" and "When do I or don't I need()at the end?" abound. Until you've learned these things it is confusing. Not that it is difficult to teach the "how and why" of querying but it's a learning curve and a place where people get frustrated.For sites that don't follow a best practice arrangement of document types and structure settings and content tree organization the situation is worse. (think about your own early sites and how poorly they were architected... that only makes the challenge of querying even more difficult) In this case you can't let the content structure work in your favor and end up having to use levels and
.Descendants()too often. Again, this is fragile/inflexible and also bad for performance as sites grow larger and larger. Again, the use ofUmbraco.Content(id)is common with content trees and doctypes that aren't structured as well as they might be.In short, the number one challenge for querying is to reliably and quickly select all the pages that exist below some other page (such as all the News Items below the News Area page).
cheers,
doug
A related challenge for querying is how to get to a 'Config' section of the content tree. Imagine this content arrangement:
CONTENT- Home
- - Products
- - News
- - Support
- - About
- Config
- - Promos
- - Widgets
Selecting various Promos is a challenge. Even if you learn the approach of going up hill to a known location and striking out from there, that won't work in this case because you can't do
.Ancestor(0)but have to know aboutUmbraco.ContentAtRoot().cheers,
doug.
While writing my previous post, I thought about how big the disconnect is for those who don't know/understand the underlying document types of a site to add a bit of logic to their markup (either in-line or via a Partial View).
For those people who aren't masters of their sites' implementation details, a query might best be handled by using the names of the pages in the content tree. Using the content tree above as an example, to get all the news items, perhaps something along the lines of:
CONTENT.Home.News.ChildrenNot sure how to denote that we're using page names rather than doctype aliases but you get the idea.
True, this kind of things is fairly easily broken... change the name of the 'Home' page to something else and the query breaks. But then again, usually the top level pages that define the main parts of the sitemap are defined early in a site's life and don't change over time. The easy of querying by name might be appropriate in many cases?
Just a wild 'outside the box' bit of thinking to consider.
cheers,
doug.
I have been thinking about this a bit today from the point of view is it the language? or is it exposure to how to even begin with it?
So maybe its not that the query language is hard, or that its broken, but it maybe that it is not easy to discover or get started with it unless you are taught.
An idea I have been playing with in my mind is the idea of a query builder, to allow users to form queries an almost spoken way so the query reads like a sentence so it is easily read but then when you insert it into the template it outputs the correctly formed razor query (either Dynamic or Strong Typed).
I quite like how Campaign Monitor allow you to create segmentation of mailing lists. Its a simple series of options, that allow you to perform what under the covers would be some fairly complex data-mining. As seen below.
So the idea would be to provide a similar editor in Umbraco to help users create queries without needing to know the language of the query. The user will form the query using common language so the query would read like a sentence. Once the query is inserted into the template, or partial the query would be as usual but the user would have an entry point to learn the querying language.
Comment author was deleted
Maybe add some extra objects (and introduce some more conventions) that provide a clearer context
Breadcrumb
Navigation
In these make some choices on what is used most for these kinds of things like a navigation could be all visible children of the homepage, breadcrumb all the ancestors...
TopNavigation
Could be one that fetches the items from a multi node picker (adding a convention for the alias umbracoTopNav)
FooterNavigation
Same for this one (so fetches items from multi node picker that uses a convention for the alias umbracoFooterNav)
So all you would need to know then is the name of the object and how to loop the collection
@foreach(var page in Navigation)
{
<li><a href="@page.Url">@page.Name</a></li>
}
So more conventions... that makes it easier to start and has all the basics covered
I like the idea by Douglas Robar to use the path of the node in queries -> CONTENT.Home.News.Children
Of course the implications of changing the name or location of a node could be horrendous. Suddenly the site no longer works because some node is moved or renamed.
Exotic feature indeed ;-)
Still I think this would not be outside the realm of possibility. And it touches on a bigger issue namely the true marriage between content and code. Is a documenttype for instance code or content? Customers want it to be content (I like to add a subsubtitle to doctype x now this very instance) and .Net devs want it to be code (don't touch it because it will not correspond with the code I have in SVN )
I really like the idea proposed by Peter.
having a query builder built into the back office or somewhere else easily accessible & obvious would be very helpful. Things like the
razor cheatsheet are great but are very hit and miss for novice users like myself if you don't have a basic grounding in .net
I guess in some ways this concept is relatively similar to what Chriztian Steinmeier did for xslt and xpath with his axes visualizer.
http://pimpmyxslt.com/axesviz.aspx
that helped me no end when xslt and xpath seemed like a big unknown to me.
It just makes life so much easier for beginners or front end devs who are often more visually orientated to be able to genertate a query from a simple tree diagram. at the very least it lets you tinker with shit to see how it works.
I think it's fine and probably even a good thing that there are multiple ways to acheive things in Umbraco, but coming to the mvc/razor way fresh, it can be confusing when some queries look visually very different to others (esp the lambada syntax really confuses me, whenever I see a x => x.Whatever I know its time to tune out... thats probably just me though)
Or the fact that this changed:
@Model.Content.Stuff in v6+ as opposed to @Model.Stuff (v4) why?
If some things could be sitting under the hood it would be less confusing for newbies and I think having a visual query-builder helps
acheive that as you can make sure it sets a standard way of doing things as a base.
one more thing I've always wondered why you cant just do this:
@Model.Content.Children("ThisParticularDocType")
rather than
@Model.Content.Children.Where(x => x.DocumentTypeAlias == "ThisParticularDocType")
or
@Model.Content.Children.Where("DocumentTypeAlias == \"ThisParticularDocType\"")
I'm very much in favour of change if it means making the template language easier for those of us with less technical chops
cheers,
- Tim
I fully agree with you Tim regarding you last points around the query language. Obviously we are now quite far down the track and if anything changes in this regard both old and updated ways need to work (for a time).
Further to my thoughts the ability to test a query right in the back office (See the old XLST visualizer)would give the user much more confidence in building queries and testing queries as they will instantly see if it returns what they need. This is kinda like the problem most devs have with RegEx, one of the most random and spaz lingos I have ever had to deal with but as soon as I found regExBuddy I was confident I could get it to do what I needed it to do, as I got instant feedback on whether or not I was hitting my matches.
Imagine a query builder that instantly updated and gave you the result in the visualizer as you built it. You learn as it constructs based on rules you give it.
@Peter: The Query Builder with visual live updates is exactly what we're working on :)
is working on a reply...