Display outstanding backported fixes for each release?

classic Classic list List threaded Threaded
33 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Display outstanding backported fixes for each release?

Julian Foad-6
It would be good for visibility if we could show on the web site what changes are waiting to be released in maintenance branches. Both merged and nominated changes.

How could we implement that to get an acceptable display on our web site?

(While we're there, we might as well include a list of the new changes on trunk too, mightn't we?)

--
- Julian
Reply | Threaded
Open this post in threaded view
|

Re: Display outstanding backported fixes for each release?

Daniel Shahaf-2
Julian Foad wrote on Mon, Dec 10, 2018 at 18:29:31 +0000:
> It would be good for visibility if we could show on the web site what changes
> are waiting to be released in maintenance branches. Both merged and nominated
> changes.
>

Sure.

> How could we implement that to get an acceptable display on our web site?
>

One option:

Start using the CHANGES-info-in-log-message convention more pervasively,
and then write a script that:

1. Queries the repository for trunk revnums that have been merged to the
stable branch since the last tag;

2. Parses STATUS for trunk revnums that are candidates for merging (this
is easy to implement using backport.py's libraries);

3. Synthesizes a CHANGES-like listing of all those revisions, distinguishing
the ones that have been merged and the ones that haven't yet been; and
say that even the ones that haven't yet been merged probably will be
before the next release (this is not a promise, but simply a reflection
of our practice of approving most or all pending nominations before
a release).

> (While we're there, we might as well include a list of the new changes on
> trunk too, mightn't we?)

I think this, too, boils to writing the CHANGES info ahead of time,
otherwise the work would amount to a custom user interface around «svn
mergeinfo --show-revs=eligible ^/subversion/trunk ^/subversion/tags/1.latestx.latesty» —
which is a job better suited for third parties such as viewvc or whimsy.
(For one, implementing the functionality in viewvc or whimsy would make
it available to all svn-using ASF projects, not just us.)

Cheers,

Daniel
Reply | Threaded
Open this post in threaded view
|

Re: Display outstanding backported fixes for each release?

Julian Foad-5
Daniel Shahaf wrote:
> [...] write a script that:
> 1. Queries the repository for trunk revnums that have been merged to the
stable branch since the last tag;

See attached "svn-revs-backported.sh".

> 3. Synthesizes a CHANGES-like listing [...]

That's one reasonable option.

I do also wonder if we couldn't implement something that more directly queries 'svn' and displays the result. That is, start designing and implementing some more powerful querying, starting with "changes in 1.10.x but not in 1.10.3".

--
- Julian

svn-revs-backported.sh (600 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Display outstanding backported fixes for each release?

Daniel Shahaf-2
Julian Foad wrote on Fri, Dec 14, 2018 at 22:30:01 +0000:
> Daniel Shahaf wrote:
> > [...] write a script that:
> > 1. Queries the repository for trunk revnums that have been merged to the
> stable branch since the last tag;
>
> See attached "svn-revs-backported.sh".

I think the script might break when we reach r10⁷ (r10000000) because
"svn mergeinfo"'s output is sorted numerically, but comm(1) expects its
inputs to be sorted according to strcoll(3).

Currently comm(1)'s precondition does hold, but only because all the
revision numbers in "svn mergeinfo"'s output have the same number of
digits.  However, as soon as HEAD passes a power of ten, we'll run into
the problem that "r9" is printed before "r10" (because 9 < 10) but
«!(strcoll("r9", "r10") < 0)».  (This is the same problem we warned
about in the 1.9 and 1.10 release notes.)
 
> > 3. Synthesizes a CHANGES-like listing [...]
>
> That's one reasonable option.
>
> I do also wonder if we couldn't implement something that more directly
> queries 'svn' and displays the result. That is, start designing and
> implementing some more powerful querying, starting with "changes in
> 1.10.x but not in 1.10.3".

Would that need new RA APIs?
Reply | Threaded
Open this post in threaded view
|

Re: Display outstanding backported fixes for each release?

Branko Čibej
On 15.12.2018 11:10, Daniel Shahaf wrote:

> Julian Foad wrote on Fri, Dec 14, 2018 at 22:30:01 +0000:
>> Daniel Shahaf wrote:
>>> [...] write a script that:
>>> 1. Queries the repository for trunk revnums that have been merged to the
>> stable branch since the last tag;
>>
>> See attached "svn-revs-backported.sh".
> I think the script might break when we reach r10⁷ (r10000000) because
> "svn mergeinfo"'s output is sorted numerically, but comm(1) expects its
> inputs to be sorted according to strcoll(3).
>
> Currently comm(1)'s precondition does hold, but only because all the
> revision numbers in "svn mergeinfo"'s output have the same number of
> digits.  However, as soon as HEAD passes a power of ten, we'll run into
> the problem that "r9" is printed before "r10" (because 9 < 10) but
> «!(strcoll("r9", "r10") < 0)».  (This is the same problem we warned
> about in the 1.9 and 1.10 release notes.)
>  
>>> 3. Synthesizes a CHANGES-like listing [...]
>> That's one reasonable option.
>>
>> I do also wonder if we couldn't implement something that more directly
>> queries 'svn' and displays the result. That is, start designing and
>> implementing some more powerful querying, starting with "changes in
>> 1.10.x but not in 1.10.3".
> Would that need new RA APIs?


For that particular query, no; it's "just" an intersection of log and
mergeinfo (with history).

But before we start on something I'd like to see some proposals on query
language, so that we don't implement ourselves into a corner. I'd
propose this as a starting point:

https://www.mercurial-scm.org/repo/hg/help/revsets


-- Brane

Reply | Threaded
Open this post in threaded view
|

Re: Display outstanding backported fixes for each release?

Julian Foad-5
Branko Čibej wrote:
> But before we start on something I'd like to see some proposals on query
> language, so that we don't implement ourselves into a corner. I'd
> propose this as a starting point:
>
> https://www.mercurial-scm.org/repo/hg/help/revsets

Yes. To be clear, I was rather intending this suggestion might be taken as a nudge towards developing some structured queries. Maybe I'll have a read through that and see how it could be adapted to Subversion, some time.

--
- Julian
Reply | Threaded
Open this post in threaded view
|

Re: Display outstanding backported fixes for each release?

Mark Phippard-3
> On Dec 15, 2018, at 2:47 PM, Julian Foad <[hidden email]> wrote:
>
> Branko Čibej wrote:
>> But before we start on something I'd like to see some proposals on query
>> language, so that we don't implement ourselves into a corner. I'd
>> propose this as a starting point:
>>
>> https://www.mercurial-scm.org/repo/hg/help/revsets
>
> Yes. To be clear, I was rather intending this suggestion might be taken as a nudge towards developing some structured queries. Maybe I'll have a read through that and see how it could be adapted to Subversion, some time.

Just a meta comment ...

I am not 100% clear what you would like to see, so I am probably not alone.  Maybe it would help if you put something manual in place first.  Then that would give people something to look at and think about how similar information could be produced in a more automated manner.

I personally think when you start talking about putting information on a website that I think the abstraction of using something like the Jira issues would be more useful than commits.  This project does not currently use Jira consistently enough that it would be an option, but aside from the occasional bug that is fixed with a single commit, I just do not think anything that is commit based will be useful.  You might as well just use svn log output at that point if that is all you want.  Think of a "feature" like shelving. That is basically one issue to show in a list, but you might have done 30-40 commits to implement it.  Seeing the commits is too low level to be useful on the website ... IMO.

Anyway, that is why I would suggest you start doing this manually.  If you start with svn log output and find yourself omitting a lot of commits that is going to give you a lot of hints about what would be needed in order to ever automate this.

Mark
Reply | Threaded
Open this post in threaded view
|

Re: Display outstanding backported fixes for each release?

Julian Foad-5
Mark Phippard wrote:

> > On Dec 15, 2018, at 2:47 PM, Julian Foad <[hidden email]> wrote:
> > Branko Čibej wrote:
> >> But before we start on something I'd like to see some proposals on query
> >> language, so that we don't implement ourselves into a corner. I'd
> >> propose this as a starting point:
> >>
> >> https://www.mercurial-scm.org/repo/hg/help/revsets
> >
> > Yes. To be clear, I was rather intending this suggestion might be taken
> as a nudge towards developing some structured queries. [...]
>
> Just a meta comment ...
>
> I am not 100% clear what you would like to see, [...]

Well, two things:
(1) an up-to-date public listing of what's pending on the release branches;
(2) development of structured queries, of any and all kinds, in Subversion.

These are very much different goals. The first is what I started this thread for -- to add visibility of what we are doing and what users can expect. This info should be simply generated, rather raw, not highly curated.

The second is a tangent, that's interesting and loosely related, with limited relationship between them.

> I personally think when you start talking about putting information on a
> website that I think the abstraction of using something like the Jira
> issues would be more useful than commits.  [...]

Agreed. And I think we should more often ensure there is an issue tracking each change we backport.

However, nearer to what we currently have available, we could list the sections from "STATUS" files where a group of revisions is proposed as a backport. The backport merges themselves generally use that same text in their log msg, so listing those would be reasonable.

> Anyway, that is why I would suggest you start doing this manually.
> [...] that is going to give you a lot of hints about what would be
> needed in order to ever automate this.

Well, simply-scripted, I would say. I have no intention of adding another manual step to the backporting process. We already put manual effort into writing the backport nominations, and sometimes issues, and eventually changelog entries. Extracting any of those is what I would consider a reasonable short-term solution. (Even listing raw commits would be better than no visibility at all, though I completely agree it's not great.)

--
- Julian
Reply | Threaded
Open this post in threaded view
|

Re: Display outstanding backported fixes for each release?

Mark Phippard-3
> On Dec 15, 2018, at 3:47 PM, Julian Foad <[hidden email]> wrote:
>
> Mark Phippard wrote:
>>>> On Dec 15, 2018, at 2:47 PM, Julian Foad <[hidden email]> wrote:
>>>> Branko Čibej wrote:
>>>> But before we start on something I'd like to see some proposals on query
>>>> language, so that we don't implement ourselves into a corner. I'd
>>>> propose this as a starting point:
>>>>
>>>> https://www.mercurial-scm.org/repo/hg/help/revsets
>>>
>>> Yes. To be clear, I was rather intending this suggestion might be taken
>> as a nudge towards developing some structured queries. [...]
>>
>> Just a meta comment ...
>>
>> I am not 100% clear what you would like to see, [...]
>
> Well, two things:
> (1) an up-to-date public listing of what's pending on the release branches;
> (2) development of structured queries, of any and all kinds, in Subversion.
>
> These are very much different goals. The first is what I started this thread for -- to add visibility of what we are doing and what users can expect. This info should be simply generated, rather raw, not highly curated.

I am just saying these are both still pretty vague.

1. You are talking about the website right?  So I am saying throw together the HTML that shows what you want to see.  Maybe that will stimulate some ideas.  I am just saying for me, as more of just a user than you are, I would not care about seeing commits.  I would want something higher level (like a Jira issue) for it to be of value to me.

2. Again vague. Isn't svn log a query?  I am sure what you are thinking is not vague to you.  Maybe if we could see what you were thinking for the previous point then the sort of thing you are looking for here would be more clear.

I am also not clear whose itch you are scratching here.  If you are thinking about SVN users wanting to follow the project, I think higher level info would be needed.  If you are just trying to build something for the SVN devs then that would be different but that is where the website part of this would be throwing me off as I cannot envision many of the SVN devs using it.

Mark
Reply | Threaded
Open this post in threaded view
|

Re: Display outstanding backported fixes for each release?

Daniel Shahaf-2
In reply to this post by Julian Foad-5
Julian Foad wrote on Sat, 15 Dec 2018 20:47 +0000:
> Well, simply-scripted, I would say. I have no intention of adding
> another manual step to the backporting process. We already put manual
> effort into writing the backport nominations,

We do?  I just run

    ./n r42 "Hello world"

and it nominates r42 with "Hello world" for its justification.  That
uses 'ln -s ../trunk-wc/tools/dist/nominate.pl n'.  (It needs so few
manual inputs because it uses the first line of the log message as the
first line of the nomination.)

> and sometimes issues, and eventually changelog entries.

Of these four --- log messages, STATUS entries, jira entries, and
CHANGES --- the latter is the only one that's explicitly written with
end users in mind; the others may use terms that users aren't familiar
with (e.g., "Fix a segfault in the reporter").  That's why I suggested
upthread to design this around CHANGES.  (Not to mention the benefits of
reducing the RM's workload of writing CHANGES at release time...)

I assume this could be previewed using 'release.py write-changelog', but
don't have time to look up the exact incantation, sorry.
Reply | Threaded
Open this post in threaded view
|

Re: Display outstanding backported fixes for each release?

Branko Čibej
In reply to this post by Mark Phippard-3
On 15.12.2018 22:22, Mark Phippard wrote:

>> On Dec 15, 2018, at 3:47 PM, Julian Foad <[hidden email]> wrote:
>>
>> Mark Phippard wrote:
>>>>> On Dec 15, 2018, at 2:47 PM, Julian Foad <[hidden email]> wrote:
>>>>> Branko Čibej wrote:
>>>>> But before we start on something I'd like to see some proposals on query
>>>>> language, so that we don't implement ourselves into a corner. I'd
>>>>> propose this as a starting point:
>>>>>
>>>>> https://www.mercurial-scm.org/repo/hg/help/revsets
>>>> Yes. To be clear, I was rather intending this suggestion might be taken
>>> as a nudge towards developing some structured queries. [...]
>>>
>>> Just a meta comment ...
>>>
>>> I am not 100% clear what you would like to see, [...]
>> Well, two things:
>> (1) an up-to-date public listing of what's pending on the release branches;
>> (2) development of structured queries, of any and all kinds, in Subversion.
>>
>> These are very much different goals. The first is what I started this thread for -- to add visibility of what we are doing and what users can expect. This info should be simply generated, rather raw, not highly curated.
> I am just saying these are both still pretty vague.
>
> 1. You are talking about the website right?  So I am saying throw together the HTML that shows what you want to see.  Maybe that will stimulate some ideas.  I am just saying for me, as more of just a user than you are, I would not care about seeing commits.  I would want something higher level (like a Jira issue) for it to be of value to me.

We're actually talking about populating CHANGES for patch releases (and
for .0 releases too, but that's tangential). We have a rather strict
process for backports, so "what's new in the patch release" is fairly
easy to find, but "how does that relate to CHANGES entries on trunk" is not.

Jira will not help unless we force ourselves to use it to prepare
CHANGES entries. I don't see that happening, unless we ditch CHANGES and
move that tracking to Jira. But IMO that would make the project far too
dependent on one particular issue tracker ...


> 2. Again vague. Isn't svn log a query?  I am sure what you are thinking is not vague to you.  Maybe if we could see what you were thinking for the previous point then the sort of thing you are looking for here would be more clear.

This talk of queries comes from Julian's comment about "designing and
implementing some more powerful querying, starting with "changes in
1.10.x but not in 1.10.3" in a previous message and my thinking aloud
about not jumping without looking first. It's not a declaration about a
new feature in 1.13.


> I am also not clear whose itch you are scratching here.

The release manager's. Managing CHANGES on release branches is quite
painful.

-- Brane

Reply | Threaded
Open this post in threaded view
|

Re: Display outstanding backported fixes for each release?

Branko Čibej
On 15.12.2018 22:55, Branko Čibej wrote:

> Jira will not help


Just to make sure we all understand what I mean: Subversion does not
have product/project management where each development cycle starts with
a set of requirements[1], which are converted to work items (i.e.,
issues in Jira) and then implemented. Normal development is not and
never was driven by an issue tracker. And can't be as long as Subversion
is driven by volunteers.


-- Brane

[1] I'm talking about the ideal workflow, of course, not what actually
happens in real life ...

Reply | Threaded
Open this post in threaded view
|

Re: Display outstanding backported fixes for each release?

Mark Phippard-3

> On Dec 15, 2018, at 5:37 PM, Branko Čibej <[hidden email]> wrote:
>
>> On 15.12.2018 22:55, Branko Čibej wrote:
>>
>> Jira will not help
>
>
> Just to make sure we all understand what I mean: Subversion does not
> have product/project management where each development cycle starts with
> a set of requirements[1], which are converted to work items (i.e.,
> issues in Jira) and then implemented. Normal development is not and
> never was driven by an issue tracker. And can't be as long as Subversion
> is driven by volunteers.

Agree 100%

For some reason I interpreted Julian's original message as wanting some kind of fancy list of changes to appear on our website.  As opposed to a better way to manage CHANGES.

Mark
Reply | Threaded
Open this post in threaded view
|

Re: Display outstanding backported fixes for each release?

Julian Foad-5
The itch I'm scratching is this: I want the people who are interested in our patch releases, and who read our CHANGES file, to be able to see what changes are coming or likely to be coming in the next patch release.

As an example, a few weeks ago a downstream packager/distributor asked me, "are there any server-side fixes coming in the next 1.10.x patch release?" I was disappointed to realize that there wasn't a quick self-serve answer to that.

There are several possible solutions. All that matters to me is that end-users can easily find the information. It would be ideal in the form that we use for CHANGES file entries, but it doesn't have to be as "digested" as that.

Mark Phippard commented:
> Seeing the commits is too low level to be useful on the website...

That is true for trunk changes: there, every meaningful change is built from a series of commits, many of which are trivial or meaningless in isolation, and rarely is there a single commit that describes the whole change well. However, a backport branch is very different: there, every backport commit should be a complete, meaningful, tested, documented, change. So showing commits (even a raw log output) is meaningful.

Given that we don't yet keep CHANGES up to date, I am thinking a reasonable short-term solution could look like this (example for the 1.9.x branch):

$ svn log -r1:HEAD --limit=1 --stop-on-copy ^/subversion/tags/1.9.9 -vq
------------------------------------------------------------------------
r1835933 | julianfoad | 2018-07-14 21:43:40 +0100 (Sat, 14 Jul 2018)
Changed paths:
   A /subversion/tags/1.9.9 (from /subversion/branches/1.9.x:1835931)
   M /subversion/tags/1.9.9/subversion/include/svn_version.h
------------------------------------------------------------------------

# note the parent branch copy-from revision -- 1835931 -- and add one

$ svn log -r1835932:HEAD ^/subversion/branches/1.9.x > commits-on-1.9.x

$ vim commits-on-1.9.x
# filter out all the log entries that are not merges (manually, this time)

Result:
[[[
------------------------------------------------------------------------
r1837042 | julianfoad | 2018-07-30 11:35:08 +0100 (Mon, 30 Jul 2018) | 10 lines

On the '1.9.x' branch: Fix german translation for 'svn help merge'.

(Merge r1837037 from trunk.)

Patch by: Christoph Vogtländer <Christoph.Vogtlaender{_AT_}sigma-surface-science.com>

* subversion/subversion/po/de.po
  Fix translation error and a typo in help text regarding the reintegration
  of a feature branch back to trunk.

------------------------------------------------------------------------
r1845529 | svn-role | 2018-11-02 04:00:05 +0000 (Fri, 02 Nov 2018) | 9 lines

Merge r1844882 from trunk:

 * r1844882
   Fix propagation of mod_dav_svn's SVNUseUTF8 configuration setting.
   Justification:
     The option has no effect in some setups. User submitted the patch.
   Votes:
     +1: stsp, brane, rhuijben

------------------------------------------------------------------------
[...]
------------------------------------------------------------------------
r1849263 | svn-role | 2018-12-19 04:00:33 +0000 (Wed, 19 Dec 2018) | 13 lines

Merge the 1.9.x-issue4791 branch:

 * r1847572, r1847596
   fsfs: Fix SVN-4791, an issue with the DAG open_path() that was causing
   unexpected SVN_ERR_FS_NOT_DIRECTORY errors when attempting to open a path
   with `open_path_node_only | open_path_allow_null` flags.
   Justification:
     Some valid FSFS read operations errored out. This could break some
     end-user operations like 'update'.
   Branch: 1.9.x-issue4791
   Votes:
     +1: julianfoad, brane, stefan2

------------------------------------------------------------------------
]]]

This output contains roughly the same level of information as if we had filled in the CHANGES file. Of course it's not the same, but for these purposes it's enough to start with.

To that list of merged backports I would want to add the lists of proposed backports. We can do this by literally appending a copy of the STATUS file. Again, if we had prepared CHANGES entries in advance, that would surely be nicer, but it's enough to start with.

Where and how to display the information? Our roadmap page would be an obvious place to look for it. So, I envision a section there called something catchy like "Upcoming changes in the next patch release in each supported release line". Under that heading, either inline or links to the verbose text samples above.

I will reiterate: I would not propose to do this at all if it has to be manually updated, only if we can auto-generate it.

- Julian
Reply | Threaded
Open this post in threaded view
|

Re: Display outstanding backported fixes for each release?

Mark Phippard-3
On Thu, Jan 3, 2019 at 10:58 AM Julian Foad <[hidden email]> wrote:
The itch I'm scratching is this: I want the people who are interested in our patch releases, and who read our CHANGES file, to be able to see what changes are coming or likely to be coming in the next patch release.

As an example, a few weeks ago a downstream packager/distributor asked me, "are there any server-side fixes coming in the next 1.10.x patch release?" I was disappointed to realize that there wasn't a quick self-serve answer to that.

There are several possible solutions. All that matters to me is that end-users can easily find the information. It would be ideal in the form that we use for CHANGES file entries, but it doesn't have to be as "digested" as that.

Mark Phippard commented:
> Seeing the commits is too low level to be useful on the website...

That is true for trunk changes: there, every meaningful change is built from a series of commits, many of which are trivial or meaningless in isolation, and rarely is there a single commit that describes the whole change well. However, a backport branch is very different: there, every backport commit should be a complete, meaningful, tested, documented, change. So showing commits (even a raw log output) is meaningful.

Given that we don't yet keep CHANGES up to date, I am thinking a reasonable short-term solution could look like this (example for the 1.9.x branch):

$ svn log -r1:HEAD --limit=1 --stop-on-copy ^/subversion/tags/1.9.9 -vq
------------------------------------------------------------------------
r1835933 | julianfoad | 2018-07-14 21:43:40 +0100 (Sat, 14 Jul 2018)
Changed paths:
   A /subversion/tags/1.9.9 (from /subversion/branches/1.9.x:1835931)
   M /subversion/tags/1.9.9/subversion/include/svn_version.h
------------------------------------------------------------------------

# note the parent branch copy-from revision -- 1835931 -- and add one

$ svn log -r1835932:HEAD ^/subversion/branches/1.9.x > commits-on-1.9.x

$ vim commits-on-1.9.x
# filter out all the log entries that are not merges (manually, this time)

Result:
[[[
------------------------------------------------------------------------
r1837042 | julianfoad | 2018-07-30 11:35:08 +0100 (Mon, 30 Jul 2018) | 10 lines

On the '1.9.x' branch: Fix german translation for 'svn help merge'.

(Merge r1837037 from trunk.)

Patch by: Christoph Vogtländer <Christoph.Vogtlaender{_AT_}sigma-surface-science.com>

* subversion/subversion/po/de.po
  Fix translation error and a typo in help text regarding the reintegration
  of a feature branch back to trunk.

------------------------------------------------------------------------
r1845529 | svn-role | 2018-11-02 04:00:05 +0000 (Fri, 02 Nov 2018) | 9 lines

Merge r1844882 from trunk:

 * r1844882
   Fix propagation of mod_dav_svn's SVNUseUTF8 configuration setting.
   Justification:
     The option has no effect in some setups. User submitted the patch.
   Votes:
     +1: stsp, brane, rhuijben

------------------------------------------------------------------------
[...]
------------------------------------------------------------------------
r1849263 | svn-role | 2018-12-19 04:00:33 +0000 (Wed, 19 Dec 2018) | 13 lines

Merge the 1.9.x-issue4791 branch:

 * r1847572, r1847596
   fsfs: Fix SVN-4791, an issue with the DAG open_path() that was causing
   unexpected SVN_ERR_FS_NOT_DIRECTORY errors when attempting to open a path
   with `open_path_node_only | open_path_allow_null` flags.
   Justification:
     Some valid FSFS read operations errored out. This could break some
     end-user operations like 'update'.
   Branch: 1.9.x-issue4791
   Votes:
     +1: julianfoad, brane, stefan2

------------------------------------------------------------------------
]]]

This output contains roughly the same level of information as if we had filled in the CHANGES file. Of course it's not the same, but for these purposes it's enough to start with.

To that list of merged backports I would want to add the lists of proposed backports. We can do this by literally appending a copy of the STATUS file. Again, if we had prepared CHANGES entries in advance, that would surely be nicer, but it's enough to start with.

Where and how to display the information? Our roadmap page would be an obvious place to look for it. So, I envision a section there called something catchy like "Upcoming changes in the next patch release in each supported release line". Under that heading, either inline or links to the verbose text samples above.


If we did not keep the STATUS file in the branch, then wouldn't it be fairly easy for anyone to see the info you want just by using svn log?  It seems like the only barrier to that today is all of the "noise" generated in the branch via the STATUS file activity.


I will reiterate: I would not propose to do this at all if it has to be manually updated, only if we can auto-generate it.

Of course.  I was not suggesting the process be manual.  I was saying I had no idea what you were proposing so maybe it makes sense to manually generate some examples before discussing how to script it.  It is kind of hard to have an automation discussion when it is not at all clear what you want to automate.

Even with your svn log examples, there is still the issue of how you would want to display that info on the ROADMAP page.  If the output from log is not suitable for display then we would still need to see what a suitable display would look like so that we could discuss how a script might massage the log output into the desired format.

--
Reply | Threaded
Open this post in threaded view
|

Re: Display outstanding backported fixes for each release?

Julian Foad-5
Mark Phippard wrote:
> If we did not keep the STATUS file in the branch, then wouldn't it be fairly easy
> for anyone to see the info you want just by using svn log?

Look at the manual process I described: it starts with how the user has to remember the magic incantation for making 'svn log' find the branch-point of a tag, then copy a revision number from that output into another 'svn log' command. And after doing that, then there is still the 'STATUS' file to append, easy of course for anyone who knows how we run our backporting process, obscure to everyone else.

> Even with your svn log examples, there is still the issue of how you would want
> to display that info on the ROADMAP page.  If the output from log is not suitable

But I said the output of log *is* suitable. Specifically, I wrote, "Under that heading, either inline or links to the verbose text samples above." And I meant it. By "inline," I'm imagining a scrollable sub-window or something; obviously I wouldn't want it to ridiculously overwhelm the whole page. More distilled and polished output would be lovely, of course, but I really don't care about the presentation style; I just care about getting the information visible.

- Julian
Reply | Threaded
Open this post in threaded view
|

Re: Display outstanding backported fixes for each release?

Julian Foad-5
Julian Foad wrote:
> [...] either inline or links to the verbose text samples above. [...]

More precisely: I propose the output of "svn log" as illustrated in those samples, concatenated with the full content of the relevant 'STATUS' file.

--
- Julian
Reply | Threaded
Open this post in threaded view
|

Re: Display outstanding backported fixes for each release?

Daniel Shahaf-2
Julian Foad wrote on Thu, Jan 03, 2019 at 17:15:16 +0000:
> Julian Foad wrote:
> > [...] either inline or links to the verbose text samples above. [...]
>
> More precisely: I propose the output of "svn log" as illustrated in those
> samples,

Here you go.

[[[
% ./upcoming.py | tail -46
------------------------------------------------------------------------
r1847610 | svn-role | 2018-11-28 04:00:29 +0000 (Wed, 28 Nov 2018)

Merge the r1847181 group from trunk:

 * r1847181, r1847182, r1847188, r1847264
   Fix issue SVN-4792: Foreign repo copy of file adding mergeinfo.
   Justification:
     We don't want bogus mergeinfo.
   Votes:
     +1: julianfoad, brane, rhuijben
------------------------------------------------------------------------
r1849266 | svn-role | 2018-12-19 04:00:55 +0000 (Wed, 19 Dec 2018)

Merge the r1847572 group from trunk:

 * r1847572, r1847596
   fsfs: Fix SVN-4791, an issue with the DAG open_path() that was causing
   unexpected SVN_ERR_FS_NOT_DIRECTORY errors when attempting to open a path
   with `open_path_node_only | open_path_allow_null` flags.
   Justification:
     Some valid FSFS read operations errored out. This could break some
     end-user operations like 'update'.
   Votes:
     +1: julianfoad, brane, stefan2
------------------------------------------------------------------------
]]]

> concatenated with the full content of the relevant 'STATUS' file.

Minus the "Veto-blocked" changes, though.

Cheers,

Daniel

upcoming.py (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Display outstanding backported fixes for each release?

Daniel Shahaf-2
Daniel Shahaf wrote on Thu, Jan 03, 2019 at 17:38:04 +0000:

> % ./upcoming.py | tail -46
> ------------------------------------------------------------------------
> r1847610 | svn-role | 2018-11-28 04:00:29 +0000 (Wed, 28 Nov 2018)
>
> Merge the r1847181 group from trunk:
>
>  * r1847181, r1847182, r1847188, r1847264
>    Fix issue SVN-4792: Foreign repo copy of file adding mergeinfo.
>    Justification:
>      We don't want bogus mergeinfo.
>    Votes:
>      +1: julianfoad, brane, rhuijben
> ------------------------------------------------------------------------
> r1849266 | svn-role | 2018-12-19 04:00:55 +0000 (Wed, 19 Dec 2018)
>
> Merge the r1847572 group from trunk:
>
>  * r1847572, r1847596
>    fsfs: Fix SVN-4791, an issue with the DAG open_path() that was causing
>    unexpected SVN_ERR_FS_NOT_DIRECTORY errors when attempting to open a path
>    with `open_path_node_only | open_path_allow_null` flags.
>    Justification:
>      Some valid FSFS read operations errored out. This could break some
>      end-user operations like 'update'.
>    Votes:
>      +1: julianfoad, brane, stefan2
> ------------------------------------------------------------------------

And to go along with that:

[[[
% cat escape.py
#!/usr/bin/env python3

"""HTML-escape and linkify"""

import fileinput
import html
import re

revision_numbers = re.compile(r'\br(\d+)')
issue_references = re.compile(r'((?:SVN-|[Ii]ssue [#]?)(\d+))')

for line in fileinput.input():
    line = html.escape(line)
    line = revision_numbers.sub(r'<a href="https://svn.apache.org/r\1">r\1</a>', line)
    line = issue_references.sub(r'<a href="/issue-\2">\1</a>', line)
    print(line, end='')
]]]

That's a stdin-to-stdout filter that converts the text output to HTML, suitable
for inclusion in the Web site.

Cheers,

Daniel
Reply | Threaded
Open this post in threaded view
|

Re: Display outstanding backported fixes for each release?

Daniel Shahaf-2
Daniel Shahaf wrote on Thu, 03 Jan 2019 18:50 +0000:
> That's a stdin-to-stdout filter that converts the text output to HTML, suitable
> for inclusion in the Web site.

I've committed the scripts, added a cron job, and wired the output
into the site:

https://subversion-staging.apache.org/docs/release-notes/#upcoming-patch-release

I make no claim that the current state is visually pleasing; feel free
to take it from here.  There are some other todo's besides the
aesthetics (see the link for details).

Cheers,

Daniel

P.S. For reference, the cron job is:

[[[
# TODO: add this to puppet once it's stable
# Also relies on ${SVN} being set
15  4 * * * cd ~/src/svn/1.11.x && ~/src/svn/site/tools/upcoming.py | { echo "<pre>"; ~/src/svn/site/tools/escape.py; echo "</pre>"; } > ~/src/svn/site/staging/upcoming.part.html && chronic ${SVN:-svn} commit ~/src/svn/site/staging/upcoming.part.html -m "* upcoming.part.html: Automatically regenerated"
]]]

(chronic(1) on the VM doesn't have the -v option.)

P.P.S. The w3c validator complains about the 'iframe' tag, but I don't
       know how to fix that (or if we care; it already complains about
       the 'placeholder' (sic) attribute in our search box).
12