Workflow for editing the subversion website

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

Workflow for editing the subversion website

Johan Corveleyn-3
The recent work on our quick-start.html page by Pavel Lyalyakin
prompted some thinking about how to better organize our site editing.
Pavel asked about lightweight branching and Daniel suggested to copy
site/publish to site/staging and having it served as
http://subversion.staging.apache.org/ to facilitate previewing [1].

I think that's a great idea (I've sometimes wanted something like this
myself, for instance when working on a difficult FAQ entry). So, if
we'd have such a staging site, how should we use it?

How about a very simple workflow like this?

1) Commit to staging. Other devs get the commit mail via the
notifications@ list.

2) Wait for others to review (the commit mail is the notification that
something needs to be reviewed). In case of large or sensitive
changes, preferably send a mail to dev@ to draw extra attention.

3) If any other committer says +1, go ahead and "promote" (merge) to
the live site.

4) If no response after 1 week? 3 days? ...? go ahead and promote to
live site (lazy consensus).

Small changes and corrections can go directly to the live site. Maybe
we'll need some exceptions for things like news, release notes and
security pages, which are usually updated as part of releases and get
a lot of eyes already.

Thoughts?

[1] https://svn.haxx.se/dev/archive-2017-10/0004.shtml

--
Johan
Reply | Threaded
Open this post in threaded view
|

Re: Workflow for editing the subversion website

Branko Čibej
On 03.10.2017 23:32, Johan Corveleyn wrote:

> The recent work on our quick-start.html page by Pavel Lyalyakin
> prompted some thinking about how to better organize our site editing.
> Pavel asked about lightweight branching and Daniel suggested to copy
> site/publish to site/staging and having it served as
> http://subversion.staging.apache.org/ to facilitate previewing [1].
>
> I think that's a great idea (I've sometimes wanted something like this
> myself, for instance when working on a difficult FAQ entry). So, if
> we'd have such a staging site, how should we use it?
>
> How about a very simple workflow like this?
>
> 1) Commit to staging. Other devs get the commit mail via the
> notifications@ list.
>
> 2) Wait for others to review (the commit mail is the notification that
> something needs to be reviewed). In case of large or sensitive
> changes, preferably send a mail to dev@ to draw extra attention.
>
> 3) If any other committer says +1, go ahead and "promote" (merge) to
> the live site.
>
> 4) If no response after 1 week? 3 days? ...? go ahead and promote to
> live site (lazy consensus).
>
> Small changes and corrections can go directly to the live site. Maybe
> we'll need some exceptions for things like news, release notes and
> security pages, which are usually updated as part of releases and get
> a lot of eyes already.
>
> Thoughts?


Something that has most (all?) merges going in one direction and no
cherry-picks.

Except for security announcements, everything else can be tested on
staging first.

-- Brane
Reply | Threaded
Open this post in threaded view
|

Re: Workflow for editing the subversion website

Daniel Shahaf-2
In reply to this post by Johan Corveleyn-3
Johan Corveleyn wrote on Tue, 03 Oct 2017 23:32 +0200:
> The recent work on our quick-start.html page by Pavel Lyalyakin
> prompted some thinking about how to better organize our site editing.
> Pavel asked about lightweight branching and Daniel suggested to copy
> site/publish to site/staging and having it served as
> http://subversion.staging.apache.org/ to facilitate previewing [1].
>

Small technical note: *.staging.apache.org is what the CMS uses, also it
will cause SSL errors since the *.apache.org certificate won't match
that hostname.  (Compare svn-eu.apache.org/svn.eu.apache.org: asterisk
in the cert doesn't match dot in the URL's hostname)  We can get around
both problems by having the staging preview site live on, say,
https://subversion-staging.apache.org/ (or even on svn-qavm).

> Small changes and corrections can go directly to the live site. Maybe
> we'll need some exceptions for things like news, release notes and
> security pages, which are usually updated as part of releases and get
> a lot of eyes already.
>
> Thoughts?

I'd like to understand the topology / flow of changes: what ensures that
changes made directly to publish are not reverted by a subsequent
promotion of staging?

FWIW, in the Apache CMS, a "publish" operation uses 'svnmucc rm publish
cp N staging publish', so it's an O(1) operation, but it literally overwrites any
changes that may have been made directly to publish/.  (I'm glossing over
a detail but that's the gist)

Cheers,

Daniel
Reply | Threaded
Open this post in threaded view
|

RE: Workflow for editing the subversion website

Bert Huijben-5


> -----Original Message-----
> From: Daniel Shahaf [mailto:[hidden email]]
> Sent: woensdag 4 oktober 2017 02:05
> To: [hidden email]
> Subject: Re: Workflow for editing the subversion website
>
> Johan Corveleyn wrote on Tue, 03 Oct 2017 23:32 +0200:
> > The recent work on our quick-start.html page by Pavel Lyalyakin
> > prompted some thinking about how to better organize our site editing.
> > Pavel asked about lightweight branching and Daniel suggested to copy
> > site/publish to site/staging and having it served as
> > http://subversion.staging.apache.org/ to facilitate previewing [1].
> >
>
> Small technical note: *.staging.apache.org is what the CMS uses, also it will
> cause SSL errors since the *.apache.org certificate won't match that
> hostname.  (Compare svn-eu.apache.org/svn.eu.apache.org: asterisk in the
> cert doesn't match dot in the URL's hostname)  We can get around both
> problems by having the staging preview site live on, say, https://subversion-
> staging.apache.org/ (or even on svn-qavm).
>
> > Small changes and corrections can go directly to the live site. Maybe
> > we'll need some exceptions for things like news, release notes and
> > security pages, which are usually updated as part of releases and get
> > a lot of eyes already.
> >
> > Thoughts?
>
> I'd like to understand the topology / flow of changes: what ensures that
> changes made directly to publish are not reverted by a subsequent
> promotion of staging?
>
> FWIW, in the Apache CMS, a "publish" operation uses 'svnmucc rm publish
> cp N staging publish', so it's an O(1) operation, but it literally overwrites any
> changes that may have been made directly to publish/.  (I'm glossing over a
> detail but that's the gist)

I think we should just use svn merge, to avoid these problems? No CMS here.

        Bert
>
> Cheers,
>
> Daniel

Reply | Threaded
Open this post in threaded view
|

Re: Workflow for editing the subversion website

Daniel Shahaf-2
Bert Huijben wrote on Wed, 04 Oct 2017 11:07 +0200:

> > -----Original Message-----
> > From: Daniel Shahaf [mailto:[hidden email]]
> >
> > I'd like to understand the topology / flow of changes: what ensures that
> > changes made directly to publish are not reverted by a subsequent
> > promotion of staging?
> >
> > FWIW, in the Apache CMS, a "publish" operation uses 'svnmucc rm publish
> > cp N staging publish', so it's an O(1) operation, but it literally overwrites any
> > changes that may have been made directly to publish/.  (I'm glossing over a
> > detail but that's the gist)
>
> I think we should just use svn merge, to avoid these problems? No CMS here.

For clarity, I'm not proposing a move to the CMS; I'm simply pointing out that
the physical way by which commits port from staging to publish, or from
publish to staging, may be either 'svn merge + svn commit' or that 'svnmucc'
replace operation.

Reply | Threaded
Open this post in threaded view
|

Re: Workflow for editing the subversion website

Johan Corveleyn-3
On Wed, Oct 4, 2017 at 1:21 PM, Daniel Shahaf <[hidden email]> wrote:

> Bert Huijben wrote on Wed, 04 Oct 2017 11:07 +0200:
>> > -----Original Message-----
>> > From: Daniel Shahaf [mailto:[hidden email]]
>> >
>> > I'd like to understand the topology / flow of changes: what ensures that
>> > changes made directly to publish are not reverted by a subsequent
>> > promotion of staging?
>> >
>> > FWIW, in the Apache CMS, a "publish" operation uses 'svnmucc rm publish
>> > cp N staging publish', so it's an O(1) operation, but it literally overwrites any
>> > changes that may have been made directly to publish/.  (I'm glossing over a
>> > detail but that's the gist)
>>
>> I think we should just use svn merge, to avoid these problems? No CMS here.
>
> For clarity, I'm not proposing a move to the CMS; I'm simply pointing out that
> the physical way by which commits port from staging to publish, or from
> publish to staging, may be either 'svn merge + svn commit' or that 'svnmucc'
> replace operation.

Yes, I think we should go for 'svn merge + svn commit'. Committing new
stuff to 'staging', and merging (with complete merges, not
cherry-picks) to 'publish'. It would be good for us, as a project, to
use this sort of workflow in practice, as I think it's a useful
workflow that's used by some Subversion users. So in the interest of
eating our own dogfood ...

I'm just not sure if this will always work out perfectly.

- Maybe we'll have some change on staging that we don't want to merge
to publish (I mean, something like showing a different header or
"staging" watermark, to indicate to users that it's not the production
site). That should be easy: we just 'merge --record-only' that commit
to publish.

- Maybe we'll do some changes directly on 'publish' (intentionally, to
be quick and efficient; or accidentally). Can we just merge those to
'staging', and expect this not to be a problem when merging staging
back to publish later? Where does svn stand currently with holding two
branches in sync, while merges can go in either direction?

--
Johan
Reply | Threaded
Open this post in threaded view
|

Re: Workflow for editing the subversion website

Branko Čibej
On 05.10.2017 01:13, Johan Corveleyn wrote:

> On Wed, Oct 4, 2017 at 1:21 PM, Daniel Shahaf <[hidden email]> wrote:
>> Bert Huijben wrote on Wed, 04 Oct 2017 11:07 +0200:
>>>> -----Original Message-----
>>>> From: Daniel Shahaf [mailto:[hidden email]]
>>>>
>>>> I'd like to understand the topology / flow of changes: what ensures that
>>>> changes made directly to publish are not reverted by a subsequent
>>>> promotion of staging?
>>>>
>>>> FWIW, in the Apache CMS, a "publish" operation uses 'svnmucc rm publish
>>>> cp N staging publish', so it's an O(1) operation, but it literally overwrites any
>>>> changes that may have been made directly to publish/.  (I'm glossing over a
>>>> detail but that's the gist)
>>> I think we should just use svn merge, to avoid these problems? No CMS here.
>> For clarity, I'm not proposing a move to the CMS; I'm simply pointing out that
>> the physical way by which commits port from staging to publish, or from
>> publish to staging, may be either 'svn merge + svn commit' or that 'svnmucc'
>> replace operation.
> Yes, I think we should go for 'svn merge + svn commit'. Committing new
> stuff to 'staging', and merging (with complete merges, not
> cherry-picks) to 'publish'. It would be good for us, as a project, to
> use this sort of workflow in practice, as I think it's a useful
> workflow that's used by some Subversion users. So in the interest of
> eating our own dogfood ...
>
> I'm just not sure if this will always work out perfectly.
>
> - Maybe we'll have some change on staging that we don't want to merge
> to publish (I mean, something like showing a different header or
> "staging" watermark, to indicate to users that it's not the production
> site). That should be easy: we just 'merge --record-only' that commit
> to publish.

Or merge and revert. We do that all the time with BRANCH-README files.

> - Maybe we'll do some changes directly on 'publish' (intentionally, to
> be quick and efficient; or accidentally). Can we just merge those to
> 'staging', and expect this not to be a problem when merging staging
> back to publish later? Where does svn stand currently with holding two
> branches in sync, while merges can go in either direction?

We do that all the time with regular development branches, too; sync
from trunk, then merge to trunk.

-- Brane
Reply | Threaded
Open this post in threaded view
|

Re: Workflow for editing the subversion website

Daniel Shahaf-2
In reply to this post by Johan Corveleyn-3
Devil's advocate hat on, and in light of Brane's sibling reply, let me
describe how an svnmucc workflow might work.

Johan Corveleyn wrote on Thu, 05 Oct 2017 01:13 +0200:
> - Maybe we'll have some change on staging that we don't want to merge
> to publish (I mean, something like showing a different header or
> "staging" watermark, to indicate to users that it's not the production
> site). That should be easy: we just 'merge --record-only' that commit
> to publish.

With svnmucc, we'll have two options: either revert the change on
staging —

   % cd site/staging/
   % svn merge -c -42
   % svn commit

— or re-create staging from current publish:

   % svnmucc rm staging cp HEAD publish staging

> - Maybe we'll do some changes directly on 'publish' (intentionally, to
> be quick and efficient; or accidentally). Can we just merge those to
> 'staging', and expect this not to be a problem when merging staging
> back to publish later? Where does svn stand currently with holding two
> branches in sync, while merges can go in either direction?

To backport from publish to staging, we could either a run cherry-pick
merge, or run the above 'svnmucc rm cp' command.  The former would be
preferred if we have other pending changes on staging/ at the time of
the direct commit to publish/.  The latter would be more efficient in
terms of developer time.

>

Cheers,

Daniel

P.S. You may have noticed that the use of HEAD in the svnmucc command
introduces a race condition: what if Alice makes a commit to staging
(creating r42) exactly at the time Bob replaces staging by publish@HEAD
(creating r43, which replaces staging/ by a copy of publish@42), meaning
Alice's commit was immediately reverted by Bob's.  If that happens, then
commit notifications will betray it to us, and we'll run 'svn merge -c
42 staging@42', thereby dog-fooding the peg-revision codepaths of 'svn
merge' ☺.

P.P.S. Alternatively, Bob could pass "-r 41" to svnmucc, which would
cause his commit command to fail with a conflict.
Reply | Threaded
Open this post in threaded view
|

Re: Workflow for editing the subversion website

Johan Corveleyn-3
On Thu, Oct 5, 2017 at 12:30 PM, Daniel Shahaf <[hidden email]> wrote:
> Devil's advocate hat on, and in light of Brane's sibling reply, let me
> describe how an svnmucc workflow might work.

Thanks, but I prefer the merge workflow. It seems more natural to me,
and I think it's more likely to be used by other svn users out there,
in case they have such a workflow. So it seems like the more
interesting dog food to me :-).

I'm not very good at writing down an accurate procedure, but I still
think it should be something like I wrote in my first mail in this
thread:

> 1) Commit to staging. Other devs get the commit mail via the
> notifications@ list.
>
> 2) Wait for others to review (the commit mail is the notification that
> something needs to be reviewed). In case of large or sensitive
> changes, preferably send a mail to dev@ to draw extra attention.
>
> 3) If any other committer says +1, go ahead and "promote" (merge) to
> the live site.
>
> 4) If no response after 1 week? 3 days? ...? go ahead and promote to
> live site (lazy consensus).

As Brane suggested, let's do everything in this direction (test on
staging first, then merge to publish), except for security
announcements.

And as Daniel suggested, let's serve the staging site via
https://subversion-staging.apache.org/ (I'd say we ask infra to set
this up for us).

--
Johan
Reply | Threaded
Open this post in threaded view
|

Re: Workflow for editing the subversion website

Branko Čibej
On 05.10.2017 22:36, Johan Corveleyn wrote:

> On Thu, Oct 5, 2017 at 12:30 PM, Daniel Shahaf <[hidden email]> wrote:
>> Devil's advocate hat on, and in light of Brane's sibling reply, let me
>> describe how an svnmucc workflow might work.
> Thanks, but I prefer the merge workflow. It seems more natural to me,
> and I think it's more likely to be used by other svn users out there,
> in case they have such a workflow. So it seems like the more
> interesting dog food to me :-).
>
> I'm not very good at writing down an accurate procedure, but I still
> think it should be something like I wrote in my first mail in this
> thread:
>
>> 1) Commit to staging. Other devs get the commit mail via the
>> notifications@ list.
>>
>> 2) Wait for others to review (the commit mail is the notification that
>> something needs to be reviewed). In case of large or sensitive
>> changes, preferably send a mail to dev@ to draw extra attention.
>>
>> 3) If any other committer says +1, go ahead and "promote" (merge) to
>> the live site.
>>
>> 4) If no response after 1 week? 3 days? ...? go ahead and promote to
>> live site (lazy consensus).
> As Brane suggested, let's do everything in this direction (test on
> staging first, then merge to publish), except for security
> announcements.
>
> And as Daniel suggested, let's serve the staging site via
> https://subversion-staging.apache.org/ (I'd say we ask infra to set
> this up for us).

Sounds like a plan.

-- Brane
Reply | Threaded
Open this post in threaded view
|

Re: Workflow for editing the subversion website

Johan Corveleyn-3
On Thu, Oct 5, 2017 at 11:42 PM, Branko Čibej <[hidden email]> wrote:

> On 05.10.2017 22:36, Johan Corveleyn wrote:
>> On Thu, Oct 5, 2017 at 12:30 PM, Daniel Shahaf <[hidden email]> wrote:
>>> Devil's advocate hat on, and in light of Brane's sibling reply, let me
>>> describe how an svnmucc workflow might work.
>> Thanks, but I prefer the merge workflow. It seems more natural to me,
>> and I think it's more likely to be used by other svn users out there,
>> in case they have such a workflow. So it seems like the more
>> interesting dog food to me :-).
>>
>> I'm not very good at writing down an accurate procedure, but I still
>> think it should be something like I wrote in my first mail in this
>> thread:
>>
>>> 1) Commit to staging. Other devs get the commit mail via the
>>> notifications@ list.
>>>
>>> 2) Wait for others to review (the commit mail is the notification that
>>> something needs to be reviewed). In case of large or sensitive
>>> changes, preferably send a mail to dev@ to draw extra attention.
>>>
>>> 3) If any other committer says +1, go ahead and "promote" (merge) to
>>> the live site.
>>>
>>> 4) If no response after 1 week? 3 days? ...? go ahead and promote to
>>> live site (lazy consensus).
>> As Brane suggested, let's do everything in this direction (test on
>> staging first, then merge to publish), except for security
>> announcements.
>>
>> And as Daniel suggested, let's serve the staging site via
>> https://subversion-staging.apache.org/ (I'd say we ask infra to set
>> this up for us).
>
> Sounds like a plan.
>
> -- Brane

Sorry, I dropped this on the floor trying to catch some other things.

I'll try to pick up the pieces now :-) ...

    svn cp $URL/site/publish $URL/site/staging
    open infra ticket for serving /site/staging on subversion-staging.a.o
    put a short description of our desired workflow in /site/README

--
Johan
Reply | Threaded
Open this post in threaded view
|

Re: Workflow for editing the subversion website

Johan Corveleyn-3
On Thu, Oct 19, 2017 at 10:04 PM, Johan Corveleyn <[hidden email]> wrote:

> On Thu, Oct 5, 2017 at 11:42 PM, Branko Čibej <[hidden email]> wrote:
>> On 05.10.2017 22:36, Johan Corveleyn wrote:
>>> On Thu, Oct 5, 2017 at 12:30 PM, Daniel Shahaf <[hidden email]> wrote:
>>>> Devil's advocate hat on, and in light of Brane's sibling reply, let me
>>>> describe how an svnmucc workflow might work.
>>> Thanks, but I prefer the merge workflow. It seems more natural to me,
>>> and I think it's more likely to be used by other svn users out there,
>>> in case they have such a workflow. So it seems like the more
>>> interesting dog food to me :-).
>>>
>>> I'm not very good at writing down an accurate procedure, but I still
>>> think it should be something like I wrote in my first mail in this
>>> thread:
>>>
>>>> 1) Commit to staging. Other devs get the commit mail via the
>>>> notifications@ list.
>>>>
>>>> 2) Wait for others to review (the commit mail is the notification that
>>>> something needs to be reviewed). In case of large or sensitive
>>>> changes, preferably send a mail to dev@ to draw extra attention.
>>>>
>>>> 3) If any other committer says +1, go ahead and "promote" (merge) to
>>>> the live site.
>>>>
>>>> 4) If no response after 1 week? 3 days? ...? go ahead and promote to
>>>> live site (lazy consensus).
>>> As Brane suggested, let's do everything in this direction (test on
>>> staging first, then merge to publish), except for security
>>> announcements.
>>>
>>> And as Daniel suggested, let's serve the staging site via
>>> https://subversion-staging.apache.org/ (I'd say we ask infra to set
>>> this up for us).
>>
>> Sounds like a plan.
>>
>> -- Brane
>
> Sorry, I dropped this on the floor trying to catch some other things.
>
> I'll try to pick up the pieces now :-) ...
>
>     svn cp $URL/site/publish $URL/site/staging
>     open infra ticket for serving /site/staging on subversion-staging.a.o
>     put a short description of our desired workflow in /site/README

Done. The infra ticket is here:
https://issues.apache.org/jira/browse/INFRA-15323

The description of the workflow in site/README is more or less
copy-pasted from this mail thread. Can probably be improved ...
(anyone: feel free). While writing it down, I realized that this
"promote from staging to publish with complete merges" isn't
parallellizable (if two committers want to make two distinct changes,
and not merge all of it in one go to publish). Oh well, it's a start.

--
Johan
Reply | Threaded
Open this post in threaded view
|

Re: Workflow for editing the subversion website

Johan Corveleyn-3
On Thu, Oct 19, 2017 at 11:58 PM, Johan Corveleyn <[hidden email]> wrote:

> On Thu, Oct 19, 2017 at 10:04 PM, Johan Corveleyn <[hidden email]> wrote:
>> On Thu, Oct 5, 2017 at 11:42 PM, Branko Čibej <[hidden email]> wrote:
>>> On 05.10.2017 22:36, Johan Corveleyn wrote:
>>>> On Thu, Oct 5, 2017 at 12:30 PM, Daniel Shahaf <[hidden email]> wrote:
>>>>> Devil's advocate hat on, and in light of Brane's sibling reply, let me
>>>>> describe how an svnmucc workflow might work.
>>>> Thanks, but I prefer the merge workflow. It seems more natural to me,
>>>> and I think it's more likely to be used by other svn users out there,
>>>> in case they have such a workflow. So it seems like the more
>>>> interesting dog food to me :-).
>>>>
>>>> I'm not very good at writing down an accurate procedure, but I still
>>>> think it should be something like I wrote in my first mail in this
>>>> thread:
>>>>
>>>>> 1) Commit to staging. Other devs get the commit mail via the
>>>>> notifications@ list.
>>>>>
>>>>> 2) Wait for others to review (the commit mail is the notification that
>>>>> something needs to be reviewed). In case of large or sensitive
>>>>> changes, preferably send a mail to dev@ to draw extra attention.
>>>>>
>>>>> 3) If any other committer says +1, go ahead and "promote" (merge) to
>>>>> the live site.
>>>>>
>>>>> 4) If no response after 1 week? 3 days? ...? go ahead and promote to
>>>>> live site (lazy consensus).
>>>> As Brane suggested, let's do everything in this direction (test on
>>>> staging first, then merge to publish), except for security
>>>> announcements.
>>>>
>>>> And as Daniel suggested, let's serve the staging site via
>>>> https://subversion-staging.apache.org/ (I'd say we ask infra to set
>>>> this up for us).
>>>
>>> Sounds like a plan.
>>>
>>> -- Brane
>>
>> Sorry, I dropped this on the floor trying to catch some other things.
>>
>> I'll try to pick up the pieces now :-) ...
>>
>>     svn cp $URL/site/publish $URL/site/staging
>>     open infra ticket for serving /site/staging on subversion-staging.a.o
>>     put a short description of our desired workflow in /site/README
>
> Done. The infra ticket is here:
> https://issues.apache.org/jira/browse/INFRA-15323
>
> The description of the workflow in site/README is more or less
> copy-pasted from this mail thread. Can probably be improved ...
> (anyone: feel free). While writing it down, I realized that this
> "promote from staging to publish with complete merges" isn't
> parallellizable (if two committers want to make two distinct changes,
> and not merge all of it in one go to publish). Oh well, it's a start.

It took a while for subversion-staging to be set up correctly (with
server-side includes etc), but now it's there:
http://subversion-staging.apache.org/

So we can now fully utilize the "edit website via staging" workflow as
described in http://svn.apache.org/repos/asf/subversion/site/README

--
Johan