Hopes / Dreams / Feedback for Umbraco Core Pull Requests
I was told that Twitter is not an appropriate avenue to give feedback for Umbraco. So here is how I see pull requests, with some suggestions sprinkled in there for good measure:
200 Open PR's The number of open pull requests has hovered around 200 for years (plus or minus).
Prioritization I was told new pull requests have been prioritized over existing ones. If the list doesn't shrink, this implies existing pull requests will never be integrated into the Umbraco core.
Abandon Lost Causes Maybe it's time to start accepting that, and actively closing older pull requests that you anticipate will never make it into the core.
Timeline If you do expect to get through all 200 pull requests (plus the new ones as they arrive), I would hope to see a timeline of progress so I know it's really happening.
Dedication If there is a timeline, will the core team stick to it, even if it means changing approach (e.g., dedicating more core team members to the PR team)?
I'm sorry if this seems pushy. I want what's best for Umbraco, and all these old PR's lingering doesn't seem like what's best for Umbraco.
Also, this isn't by any means a list of demands. This is a list of observations, thoughts, and suggestions. Maybe others would like to contribute their own observations, thoughts, and suggestions. I would welcome that.
However, I see that as only a first step. I see that as talk about improvement, with very little concrete numbers mentioned. That is, it's a qualitative improvement, but I would hope for quantitative improvement. Numbers are easier to trust. In my ideal world, this is what I'd like to see:
Transparent & Concrete Goals The Umbraco core team announces that they will reduce the number of open PR's by 15/week, with the ultimate goal of getting to fewer than 20 open PR's by the end of October 2018.
Transparent Strategy The Umbraco core team announces that they plan to split prioritization 50/50 between the oldest PR's and the newest PR's. And the include in this announcement the other aspects of their strategy that will ensure progress is made.
Transparent Progress The Umbraco core team releases weekly or monthly numbers, along with a blog post saying "all is well" or "we haven't met our goals and here's how we're going to adapt going forward".
Real Results Pull requests are in fact dropped down to fewer than 20 by the end of October 2018.
Celebration A huge celebration for the Umbraco core team once they've accomplished this monumental task.
Are these ideas too extreme? Should I simply trust that things will change? You tell me; I'm interested to hear what others think, especially the core team.
I don't want to comment on every point you made as you do make some good points, some interesting ones, and some inaccurate ones. In general though, I think we are all on the same page and have similar goals so to that end I applaud you making this post public.
That said, I do have a few comments:
1 - You don't mention the 2000 or so merged pull requests I don't know if this just an oversight but if you look at how many PRs do get reviewed, then merged (or closed and not merged in some cases) I think it's obvious that there is very good progress from the core team in terms of reviewing PRs and taking the appropriate action.
2 - Not all PRs are the same, which is stating the obvious I guess. But the point being that reviewing some PRs will take longer (maybe significantly longer) than others - i.e., a refactor of a legacy feature vs. some language updates. If you look at the scope of some of the open PRs its not hard to see that reviewing and then merging will take some time.
3 - I have a personal/business interest in one specific PR, and I expect you do as well, but I also understand that just because we create a PR that we believe is useful, does not obligate the core team to include the PR. I do agree, however, that I'd love to see the new CPR team close PRs that will not be merged a bit more proactively.
My final comment - if you look at the numbers in relation to PRs and at the number of community PRs included in each Umbraco CMS release - I think it is accurate to say we already see real results and transparent progress from the team. I think they deserve a massive #H5YR from us already - and more encouragement can't hurt :-) .
2,000 pull requests does sound impressive. Here is some additional context to that number:
5-Years That's the total number of pull requests merged over a 5 year period. That's a long time span, and it means about 400 have been closed per year.
Community-Derived? Some of those may include PR's from the Umbraco core team. Unless pull requests are exclusively something initiated from the community. Meaning, this may represent much of the active development from the core rather than community pull requests. Or maybe not; I don't know (though I suspect core contributions are also done via pull requests). Point is, it would be nice to see more focus on the community PR's.
Example Since you gave an example, I'll give on too. One I care about (but is not mine) is older than 2 years: https://github.com/umbraco/Umbraco-CMS/pull/1267 It's quite trivial as well, but would be really helpful. This hits us on every Umbraco install now (since SVG has become the standard we use for logos).
What would be too long to wait for a PR to be reviewed and included or rejected? I'm thinking 6 months would be a good upper limit (with most happening within a month). But that's based entirely on gut feel. Maybe there are specific reasons that number should be different.
And yes, the Umbraco team deserves all the kudus in the world for making Umbraco itself happen. It's wonderful. Right now, though, the topic of this discussion is how it can be improved with specific regard to pull requests.
I'm just going to throw my 2 sense in, and I'm starting with a small disclaimer.
I've not looked into any of this in detail, only read what people have posted and am posting a relatively emotion based response so please don't take offence if I do upset anyone I'm sorry.
I agree and disagree with everything I've seen about PR's over the last 2 - 3 years. There is a constant grip that things don't get acknowledged, let alone merged and I do genuine believe that the introduction of the CPR team is going to change that. They were announced this year at CG18 and, in real terms, I believe have only fully started their work around this in the past 2 weeks due to holidays and the like. I very much look forward to seeing how they operate and how they can start pulling these numbers of outstanding PR's down. This is a good step in the right direction.
With regards to the historic PR's, (by historic, I'm referring to anything older than a year).
I suspect these are going to require a different type of review as so much has changed in Umbraco in that time the PR may or may not still be relevant. But it would be nice, if even though more recent PR's are being prioritised if these were thrown into the mix. Say 1 historic for every 3 - 5 new. This would start making a dent in them.
The size on the PR also clearly makes a difference. If a it's a tiny PR that is updating, for example, some translations, in reality these can filter through rapidly. I'm assuming the CPR team are going through the open ones and doing quick analysis to see what the existing PR's are for and doing this but I could be wrong :-)
I think once the CPR team is fully established, and a mechanism for replacing members if (as is likely) they can no longer be part of the team for what ever reason is also defined, we will see and hear about their activities.
It would be cool, and this is just an idea kinda off the back of Nicholas's, if the CPR did similar updates to Sophie's Doc updates where we get a monthly run down of how well they are doing. Or they have a page telling us how we can make their lives easier. If there are things that we can do with existing PR's to change their format or provide additional testing it could help their progress.
I also don't think that V8 is going to kill V7, at least not to start with but at some point, yes that might be right. But I'd like to think that the CPR and HQ will, by that point in time, come up with a "maintenance" style schedule for handling PR's to the final V7 version. With a full EoL style plan that helps us as dev's maintain and patch when we can. I appreciate this is a massive undertaking though so who knows what will happen.
As I said before this is just a bit of a brain dump, sorry if it doesn't make sense or I'm rambling. It's late and my brain doesn't want to put things in quite the right order but overall I think we are at the point where the PR "problem" is starting down the right path. It will just take a bit longer before we start seeing that more obviously. I know how frustrating it is when your PR sits for ages, but if you have one, I would say update it to the latest Umbraco version if it's not already and throw a comment on the related issue to help draw attention to it :-)
In an Open Source project how PRs are dealt with is an important topic and I'm glad that Nicholas has raised this. A large backlog of PRs that appear to be in limbo isn't a good situation, either for the project or for contributors. However, the good news is that we know HQ is aware of this and are actively trying to change the situation. I think most of Nick's observations are pertinent, but perhaps the timing is a little off given the new CPR (Community Pull Request) team haven't really had chance to make an impact yet.
For me personally one of the worst aspects of having open PRs is the effect that this will have on people wishing to contribute in future. I know personally people who will no longer bother to make PRs because they don't believe they'll ever be looked at. For a successful community based project this perception has to change.
It takes a lot of effort to make a good PR - not just the coding time, but the "thinking time", the research, the time to build and clone the source, create unit tests and the time to write up an associated issue. To do all that (in your own spare time) and then have your PR sit there and gather dust is frustrating and, even worse, disheartening. It's also equally frustrating to see an issue you are currently having with a site has been fixed in a PR over a year ago which is also gathering dust.
So I welcome the CPR team - and let's give them a chance - but I agree with Nicholas that we have to actively see progress, and this includes activity on historical PRs. Feedback and perception is crucial to this endeavour. But I think if progress is not seen to be made then we should be able to call on HQ to give this higher priority - perhaps refocus some more core members into this. It's maybe not fair to leave this mainly to community volunteers to sort out, however awesome those peeps are!
I'd actually like to see Umbraco having more PRs, as it's clear that there are a lot of awesome devs out there who are passionate about making Umbraco better - it really could be pivotal in the progress of Umbraco. But that requires the capacity to deal with the increased volume of PRs.
PS. I've seen some criticism of Nicholas on Twitter for even bringing this up. I understand people are defensive about Umbraco, a project they love, but that shouldn't mean we create an atmosphere were criticism isn't tolerated. The sincerest critics are often the ones who love something the most. We should be inclusive to all points of view.
Thanks, Dave. I have many thoughts on that, but I'll withhold my comments for another month or so (so the situation has a chance to evolve). In the meantime, I'll keep posting stats:
2018-06-13: 203 Open Pull Requests
2018-06-18: 200 Open Pull Requests
2018-06-26: 190 Open Pull Requests
At the current rate, the anticipated completion of all outstanding pull requests is January 2, 2019.
I just took a look at some of the oldest pull requests, as they are a couple of years old. I think the majority of them could really do with their original authors re-basing and re-test them on the latest code base, there is a good chance that they are no longer relevant or currently broken.
As an example, the oldest item is currently 6189 commits behind the umbraco:dev-v7
It would also be good if other members of the community offered to help clear this list, if we want the list to reduce and 20+ of us helped in that process, we could get it finished by the end of the summer, I don't think we should assume the new CPR team have the bandwidth to do this all on their own, I am sure they all have day jobs too :)
I've just added a comment on the above mentioned ticket and offered to help, over to everyone else who is reading this thread, pick a ticket and lets help get them either merged in or closed :)
Thank you for your passion and interest, we are just as passionate at HQ about including community PRs whenever time and resources allow it. We continue to work actively on both version 7 and 8 related PRs (old and new).
At this point it would be premature/counter productive for people to update their older PRs. Please leave PR and issue tracker management to HQ for optimal productivity.
yes, but if there is 15 new PRs each week, the anticipated completion will always be the same period from now
Not sure I follow. The anticipated completion depends both on rate of new PRs and rate of closed PRs, which when combined amount to the total number of open PRs.
I am tracking the total number of open PRs, which will result in the estimated completion date of all open PRs.
Of course, we can expect some fluctuations in each rate (rate of closing PRs and rate of opening PRs), which is why I'm posting each week (i.e., so we can see those fluctuations).
FWIW, in my digging through issues looking for quick wins/useful stuff, there's quite a lot of duplicate issues - probably then also holds true that there are duplicated PRs sitting in that pending pool.
I'd also expect that the older the PR, the more likely it is to no longer be an issue. Not saying that's always the case, but just that to take a raw number of outstanding PRs is not necessarily the best indicator of progress (or lack there of).
To really get ahead of the backlog probably takes more resourcing, but as we all know all too well, resources are limited.
Spare a thought for the poor team looking after Joomla - they're sitting on a backlog of 375 pending PRs...
At the current rate, the anticipated completion of all outstanding pull requests is March 13, 2019.
Final Update
This is the last one of these updates I will be posting. There was basically one good month, and the other months have either been very slow or negative progress. To put a dent in these, faster progress will have to be made, and it's clear to me that this will not happen with the current strategy.
I would encourage the Umbraco HQ to try a new approach, as the approaches they have tried thus far don't seem to be working.
Hopes / Dreams / Feedback for Umbraco Core Pull Requests
I was told that Twitter is not an appropriate avenue to give feedback for Umbraco. So here is how I see pull requests, with some suggestions sprinkled in there for good measure:
I'm sorry if this seems pushy. I want what's best for Umbraco, and all these old PR's lingering doesn't seem like what's best for Umbraco.
Also, this isn't by any means a list of demands. This is a list of observations, thoughts, and suggestions. Maybe others would like to contribute their own observations, thoughts, and suggestions. I would welcome that.
BTW, I'm glad to see the CPR team: https://umbraco.com/blog/introducing-the-new-cpr-team/
However, I see that as only a first step. I see that as talk about improvement, with very little concrete numbers mentioned. That is, it's a qualitative improvement, but I would hope for quantitative improvement. Numbers are easier to trust. In my ideal world, this is what I'd like to see:
Are these ideas too extreme? Should I simply trust that things will change? You tell me; I'm interested to hear what others think, especially the core team.
Hi Nicholas -
I don't want to comment on every point you made as you do make some good points, some interesting ones, and some inaccurate ones. In general though, I think we are all on the same page and have similar goals so to that end I applaud you making this post public.
That said, I do have a few comments:
1 - You don't mention the 2000 or so merged pull requests I don't know if this just an oversight but if you look at how many PRs do get reviewed, then merged (or closed and not merged in some cases) I think it's obvious that there is very good progress from the core team in terms of reviewing PRs and taking the appropriate action.
2 - Not all PRs are the same, which is stating the obvious I guess. But the point being that reviewing some PRs will take longer (maybe significantly longer) than others - i.e., a refactor of a legacy feature vs. some language updates. If you look at the scope of some of the open PRs its not hard to see that reviewing and then merging will take some time.
3 - I have a personal/business interest in one specific PR, and I expect you do as well, but I also understand that just because we create a PR that we believe is useful, does not obligate the core team to include the PR. I do agree, however, that I'd love to see the new CPR team close PRs that will not be merged a bit more proactively.
My final comment - if you look at the numbers in relation to PRs and at the number of community PRs included in each Umbraco CMS release - I think it is accurate to say we already see real results and transparent progress from the team. I think they deserve a massive #H5YR from us already - and more encouragement can't hurt :-) .
-Paul
I appreciate the feedback and numbers, Paul.
2,000 pull requests does sound impressive. Here is some additional context to that number:
What would be too long to wait for a PR to be reviewed and included or rejected? I'm thinking 6 months would be a good upper limit (with most happening within a month). But that's based entirely on gut feel. Maybe there are specific reasons that number should be different.
And yes, the Umbraco team deserves all the kudus in the world for making Umbraco itself happen. It's wonderful. Right now, though, the topic of this discussion is how it can be improved with specific regard to pull requests.
I'm just going to throw my 2 sense in, and I'm starting with a small disclaimer.
I've not looked into any of this in detail, only read what people have posted and am posting a relatively emotion based response so please don't take offence if I do upset anyone I'm sorry.
I agree and disagree with everything I've seen about PR's over the last 2 - 3 years. There is a constant grip that things don't get acknowledged, let alone merged and I do genuine believe that the introduction of the CPR team is going to change that. They were announced this year at CG18 and, in real terms, I believe have only fully started their work around this in the past 2 weeks due to holidays and the like. I very much look forward to seeing how they operate and how they can start pulling these numbers of outstanding PR's down. This is a good step in the right direction.
With regards to the historic PR's, (by historic, I'm referring to anything older than a year).
I suspect these are going to require a different type of review as so much has changed in Umbraco in that time the PR may or may not still be relevant. But it would be nice, if even though more recent PR's are being prioritised if these were thrown into the mix. Say 1 historic for every 3 - 5 new. This would start making a dent in them.
The size on the PR also clearly makes a difference. If a it's a tiny PR that is updating, for example, some translations, in reality these can filter through rapidly. I'm assuming the CPR team are going through the open ones and doing quick analysis to see what the existing PR's are for and doing this but I could be wrong :-)
I think once the CPR team is fully established, and a mechanism for replacing members if (as is likely) they can no longer be part of the team for what ever reason is also defined, we will see and hear about their activities.
It would be cool, and this is just an idea kinda off the back of Nicholas's, if the CPR did similar updates to Sophie's Doc updates where we get a monthly run down of how well they are doing. Or they have a page telling us how we can make their lives easier. If there are things that we can do with existing PR's to change their format or provide additional testing it could help their progress.
I also don't think that V8 is going to kill V7, at least not to start with but at some point, yes that might be right. But I'd like to think that the CPR and HQ will, by that point in time, come up with a "maintenance" style schedule for handling PR's to the final V7 version. With a full EoL style plan that helps us as dev's maintain and patch when we can. I appreciate this is a massive undertaking though so who knows what will happen.
As I said before this is just a bit of a brain dump, sorry if it doesn't make sense or I'm rambling. It's late and my brain doesn't want to put things in quite the right order but overall I think we are at the point where the PR "problem" is starting down the right path. It will just take a bit longer before we start seeing that more obviously. I know how frustrating it is when your PR sits for ages, but if you have one, I would say update it to the latest Umbraco version if it's not already and throw a comment on the related issue to help draw attention to it :-)
Hope my random thoughts help the discussion :-)
In an Open Source project how PRs are dealt with is an important topic and I'm glad that Nicholas has raised this. A large backlog of PRs that appear to be in limbo isn't a good situation, either for the project or for contributors. However, the good news is that we know HQ is aware of this and are actively trying to change the situation. I think most of Nick's observations are pertinent, but perhaps the timing is a little off given the new CPR (Community Pull Request) team haven't really had chance to make an impact yet.
For me personally one of the worst aspects of having open PRs is the effect that this will have on people wishing to contribute in future. I know personally people who will no longer bother to make PRs because they don't believe they'll ever be looked at. For a successful community based project this perception has to change.
It takes a lot of effort to make a good PR - not just the coding time, but the "thinking time", the research, the time to build and clone the source, create unit tests and the time to write up an associated issue. To do all that (in your own spare time) and then have your PR sit there and gather dust is frustrating and, even worse, disheartening. It's also equally frustrating to see an issue you are currently having with a site has been fixed in a PR over a year ago which is also gathering dust.
So I welcome the CPR team - and let's give them a chance - but I agree with Nicholas that we have to actively see progress, and this includes activity on historical PRs. Feedback and perception is crucial to this endeavour. But I think if progress is not seen to be made then we should be able to call on HQ to give this higher priority - perhaps refocus some more core members into this. It's maybe not fair to leave this mainly to community volunteers to sort out, however awesome those peeps are!
I'd actually like to see Umbraco having more PRs, as it's clear that there are a lot of awesome devs out there who are passionate about making Umbraco better - it really could be pivotal in the progress of Umbraco. But that requires the capacity to deal with the increased volume of PRs.
PS. I've seen some criticism of Nicholas on Twitter for even bringing this up. I understand people are defensive about Umbraco, a project they love, but that shouldn't mean we create an atmosphere were criticism isn't tolerated. The sincerest critics are often the ones who love something the most. We should be inclusive to all points of view.
Since I haven't heard any official word, I suppose I'll just keep a weekly tally going to track progress. Here is as good a place as any to do that:
At the current rate, the anticipated completion of all outstanding pull requests is May 17, 2019.
ICYMI Sebastiaan released a blog today about the PR progress : https://umbraco.com/blog/the-pr-team-chronicles-month-one/
Dave
Thanks, Dave. I have many thoughts on that, but I'll withhold my comments for another month or so (so the situation has a chance to evolve). In the meantime, I'll keep posting stats:
At the current rate, the anticipated completion of all outstanding pull requests is January 2, 2019.
I just took a look at some of the oldest pull requests, as they are a couple of years old. I think the majority of them could really do with their original authors re-basing and re-test them on the latest code base, there is a good chance that they are no longer relevant or currently broken.
As an example, the oldest item is currently 6189 commits behind the umbraco:dev-v7
Navigational map for the Grid
It would also be good if other members of the community offered to help clear this list, if we want the list to reduce and 20+ of us helped in that process, we could get it finished by the end of the summer, I don't think we should assume the new CPR team have the bandwidth to do this all on their own, I am sure they all have day jobs too :)
I've just added a comment on the above mentioned ticket and offered to help, over to everyone else who is reading this thread, pick a ticket and lets help get them either merged in or closed :)
Chris
Thank you for your passion and interest, we are just as passionate at HQ about including community PRs whenever time and resources allow it. We continue to work actively on both version 7 and 8 related PRs (old and new).
At this point it would be premature/counter productive for people to update their older PRs. Please leave PR and issue tracker management to HQ for optimal productivity.
Here are this week's stats:
At the current rate, the anticipated completion of all outstanding pull requests is January 2, 2019.
Hi Nicholas,
Are you also counting PRs opened within the last week?
I see 15 new within the last 7 days.
Not sure what you mean. I'm counting total open, which I get here: https://github.com/umbraco/Umbraco-CMS/pulls
yes, but if there is 15 new PRs each week, the anticipated completion will allways be the same period from now :)
https://github.com/umbraco/Umbraco-CMS/pulls?utf8=%E2%9C%93&q=is%3Apr++created%3A%3E2018-06-01
PRs created since 1st of june: 28 open, 63 closed :)
Not sure I follow. The anticipated completion depends both on rate of new PRs and rate of closed PRs, which when combined amount to the total number of open PRs.
I am tracking the total number of open PRs, which will result in the estimated completion date of all open PRs.
Of course, we can expect some fluctuations in each rate (rate of closing PRs and rate of opening PRs), which is why I'm posting each week (i.e., so we can see those fluctuations).
It was down to 174 I think on thursday....but then some people decided to have a V8 hackaton on friday :-)
The number of open PR's on V7 is actually 168. You can get that using this filter
is:pr is:open base:dev-v7Since 2018-06-26 the number of closed PR's is 29 for V7. I used this filter :
is:pr is:open base:dev-v7 closed:>=2018-06-26Since 2018-06-26 the number of created PR's is 12 for V7, but 7 are already closed. I used this filter :
is:pr is:open base:dev-v7 closed:>=2018-06-26If we look at those numbers of V8 in the same time period (by changing base param in filters to temp8)
The number of open PR's for V8 is 15.
Dave
Feel free to track v7 or v8 or whatever subset you like. My focus will continue to be on the total number of open PRs.
FWIW, in my digging through issues looking for quick wins/useful stuff, there's quite a lot of duplicate issues - probably then also holds true that there are duplicated PRs sitting in that pending pool.
I'd also expect that the older the PR, the more likely it is to no longer be an issue. Not saying that's always the case, but just that to take a raw number of outstanding PRs is not necessarily the best indicator of progress (or lack there of).
To really get ahead of the backlog probably takes more resourcing, but as we all know all too well, resources are limited.
Spare a thought for the poor team looking after Joomla - they're sitting on a backlog of 375 pending PRs...
Here are this week's stats:
At the current rate, the anticipated completion of all outstanding pull requests is January 26, 2019.
Here are this week's stats:
At the current rate, the anticipated completion of all outstanding pull requests is January 8, 2019.
Here are this week's stats:
At the current rate, the anticipated completion of all outstanding pull requests is January 24, 2019.
Here are this week's stats:
At the current rate, the anticipated completion of all outstanding pull requests is November 4, 2018.
Here are this week's stats:
At the current rate, the anticipated completion of all outstanding pull requests is November 27, 2018.
I'll have to remind myself not to fix any core bug on a Wednesday.
(I submitted 2 PRs today) ;-)
Here are this week's stats:
At the current rate, the anticipated completion of all outstanding pull requests is December 23, 2018.
Aww Nicholas, you're early this week :-)
Maybe the calculation could be in real-time on a website? WhenWillAllTheUmbracoPullRequestsBeDone.com
Early this week, or late last week. All a matter of perspective :-)
I just added two small pull requests this morning, so hopefully they will be quick to merge in before next week's stats update :)
Here are this week's stats:
At the current rate, the anticipated completion of all outstanding pull requests is February 6, 2019.
Here are this week's stats:
At the current rate, the anticipated completion of all outstanding pull requests is December 18, 2018.
Here are this week's stats:
At the current rate, the anticipated completion of all outstanding pull requests is January 12, 2019.
Here are this week's stats:
At the current rate, the anticipated completion of all outstanding pull requests is February 18, 2019.
Here are this week's stats:
At the current rate, the anticipated completion of all outstanding pull requests is March 13, 2019.
Final Update
This is the last one of these updates I will be posting. There was basically one good month, and the other months have either been very slow or negative progress. To put a dent in these, faster progress will have to be made, and it's clear to me that this will not happen with the current strategy.
I would encourage the Umbraco HQ to try a new approach, as the approaches they have tried thus far don't seem to be working.
is working on a reply...