Checkpointing v1 design

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
22 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Checkpointing v1 design

Julian Foad-5
After playing with two pre-prototypes and discussing a wide variety of
ideas on this list, I have given detailed thought to a v1 checkpointing
design with the following properties:

   * local (not server)
   * built on shelving v1 (patches)
   * main commands are 'checkpoint' to save a new version and 'rollback'
to revert to version N
   * not intended for making a series of commits
   * designed to integrate with 'changelists' later

This aims to streamline the manual method, which I use myself, of using
'svn diff' and 'svn patch' to create and apply patches named
'feature-v1.patch', 'feature-v2.patch', etc.

I plan to go ahead and implement this, in order to have something
useful. At the same time I will keep an eye on further developments and
the wider picture.


== Shelving and Checkpointing Commands ==

unshelve X [N]
   assert(not X.applied)
   if (any patch-get-paths(X) is modified):
     warn?
   patch-apply X [N]
   X.applied := true

shelve X [PATH...]
   # X may or may not be previously 'applied'
   assert(any PATH... is modified)
   patch-save(X, PATH...)
   patch-unapply X
   X.applied := false

checkpoint X [PATH...]
   # X may or may not be previously 'applied'
   assert(any PATH... is modified)
   patch-save(X, PATH...)
   X.applied := true

rollback X [N]
   assert(X.applied)
   revert(union( patch-get-files(X, X.current),
                 patch-get-files(X, N) ))
   patch-apply(X, N)
   patch-set-current(X, N)

drop/remove/delete X
   # current syntax "svn shelve --delete X"
   assert(not X.applied)
   patch-delete-all X


== Interaction with existing commands ==

Existing ways to remove a change from the WC are 'revert' and 'commit'.

commit X
   # possible syntax "svn commit --changelist X"
   assert(X.applied)
   commit(patch-get-files(X))
   X.applied := false

revert X
   # possible syntax "svn revert --changelist X"
   assert(X.applied)
   revert(patch-get-files(X))
   X.applied := false

revert PATH...
   revert(PATH...)
   if PATH... includes patch-get-files(X):
     X.active := false


== Low-level Interface ==

patch-save(X, PATH...)
   save diff(PATH...) as 'X-<latest+1>.patch'

patch-apply(X, N=latest)
   apply 'X-N.patch' to the WC

patch-unapply(X, N=latest)
   reverse-apply 'X-N.patch' to the WC

patch-delete-all(X)
   delete all versions of X

patch-get-current(X)
   return N = the maximum '*' in 'X-*.patch'

path-set-current(X, N)
   mark the current applied version of X as being N
   (options: prune all versions greater than N, or set X.current=N,
   or even perhaps 'symlink' 'X-<latest+1>.patch' to 'X-N.patch')

patch-get-files(X, N=latest)
   return the list of (file) paths in X version N


=== Remembering the Applied/Shelved state ===

For each change-set name X, the WC will remember whether X is considered
to be currently 'applied' to the WC, and if so, at which version N. This
is needed so that 'rollback' knows (roughly) what to revert, and to a
lesser degree so that commands like 'shelve' and 'unshelve' can say
'hold on, you already have [or do not have] that change-set applied'.

The Shelving v1 implementation by default deleted a patch (or rather
renamed it to '.bak') when unshelving, and when shelving it objected if
the given name was already shelved. Now in this design we are going to
keep all versions of X (and 'shelve' will always add a new version) so
we will store a separate 'applied' flag instead.

Multiple independent change-sets can be active in the WC at the same
time, like with 'changelists'. Clashes (overlaps) in the list of paths
affected must be managed manually. (Potential enhancement: add some
warnings, e.g. if 'rollback' is going to revert a file that is part of
both the specified change-set and another one.)


=== Rollback and Revert ===

Perhaps one of the trickiest parts is which files will 'rollback'
revert. Suggestion is union(files in current applied version, files in
requested version).

May want to check and warn if those files overlap with any other
currently applied change-set.

Integration with changelists should clarify this.


=== Path spec [PATH...] ===

* is applicable to 'save' commands (shelve, checkpoint)
* is not applicable to other commands
* default is "." like in most svn commands
* is restrictive (restricts operation to PATH...)
* is not tracked or managed
* clashes (same path in more than one applied patch) are not managed

Future considerations:

* when checkpointing, warn if PATH... excludes any paths that were in
the previous version?
* consider allowing restricting paths on 'apply' commands.


=== Changelists and Terminology ===

Already in this design we see a need to specify a 'change set' which is
not necessarily in a 'shelved' state. The existing 'changelist' fits
this concept quite closely. It makes sense to extend it to do so. I even
suggest 'svn revert|commit --changelist=X' as a possible syntax although
that won't quite fit the v1 implementation if we haven't integrated with
changelists yet.

One implication is we need a consistent name to use for the change-sets
that we are dealing with. 'change-set' is pretty good; 'changelist' has
the advantage of already existing although I don't like it so much.
'Shelved change' and 'patch' no longer work when talking about
modifications in the WC.

I was already finding terminology very awkward (and I have been
inconsistent in using it) because of the difference between 'shelf'
(noun) and 'shelve' (verb) and whether a single change-set contains 'a
change' or 'some changes'. I am glad to have a reason to change it.


=== Roll Forward ===

As a starting point, rollback be destructive, deleting versions newer
than the target version. As an enhancement, it could be made to keep the
newer versions and allow rolling forward to them. It could operate like
the 'undo stack' model commonly found in editing applications, where
roll-forward (often named 'redo') is possible up until a different
change is saved to the stack, at which time the possibility is lost.

Auto checkpoint before rollback would then be possible too.


=== Commit Log Messages ===

Each new version should keep any previously specified log message by
default, and allow it to be replaced or edited.

In 'rollback': keep the latest version of the log message; maybe do not
even store older versions of it.

This mental model -- unversioned log message, versioned files -- is
simple and fits well with existing svn concepts.


- Julian
Reply | Threaded
Open this post in threaded view
|

Re: Checkpointing v1 design

Julian Foad-5
Listing shelved and applied change-sets could look something like this:

[[[
$ svn shelves
applied:
   X (at version 1)
   thing (at version 10)
shelved:
   foo (1 version)
   bar (4 versions)
   old-thing (1 version)
   new-changes (18 versions)
]]]

or integrated with changelists:

[[[
$ svn changelist --list
--- Applied Changelists:
   X (checkpoint 1, 1 file)
   thing (checkpoint 10, 2 files)
--- Shelved Changelists:
   foo (checkpoint 1, 14 files)
   bar (checkpoint 4, 1 file)
   old-thing (checkpoint 1, 2 files)
   new-changes (checkpoint 18, 3 files)
]]]

Listing checkpoints:

[[[
$ svn checkpoint list [--cl=]thing
thing-1     1230 mins old        874 bytes
  (no log message)
  hello.txt |    1 +
  1 file changed, 1 insertion(+)
...
thing-10      10 mins old      12345 bytes
  log message: Blah blah blah...
  hello.txt |    1 +
  bar.txt   |   12 ++++++++++++
  2 files changed, 13 insertions(+)
]]]

- Julian
Reply | Threaded
Open this post in threaded view
|

Re: Checkpointing v1 design

Branko Čibej
In reply to this post by Julian Foad-5
On 03.11.2017 15:54, Julian Foad wrote:
> After playing with two pre-prototypes and discussing a wide variety of
> ideas on this list, I have given detailed thought to a v1
> checkpointing design with the following properties:
>
>   * local (not server)
>   * built on shelving v1 (patches)
>   * main commands are 'checkpoint' to save a new version and
> 'rollback' to revert to version N


The main issue I have with this is that "checkpoint" is not a verb. (I
know, neither are 'changelist' or 'auth').

How about 'save' (to create a savepoint) and 'restore' instead?

As to why I'm nitpicking at this point: whatever we choose will be
pretty much set in stone, in the UI and the API, for the conceivable future.

-- Brane
Reply | Threaded
Open this post in threaded view
|

Re: Checkpointing v1 design -- terminology

Julian Foad-5
Branko Čibej wrote:

>>    * main commands are 'checkpoint' to save a new version and
>> 'rollback' to revert to version N
>
> The main issue I have with this is that "checkpoint" is not a verb. (I
> know, neither are 'changelist' or 'auth').
>
> How about 'save' (to create a savepoint) and 'restore' instead?
>
> As to why I'm nitpicking at this point: whatever we choose will be
> pretty much set in stone, in the UI and the API, for the conceivable future.

Thanks for helping with the terminology. I agree it's important.

The prototype CLI I'm working on (in the 'shelve-checkpoint' branch)
actually has 'svn checkpoint save' and 'svn checkpoint rollback'. Do you
think mean to suggest plain 'svn save' and 'svn restore'? I'm not sure
about that. They could deserve to be first-class 'svn' commands if we
consider them first-class companions to 'commit', 'revert' and 'diff'.

The analogy to 'save' a file or the work in almost any application is
apt. (For the converse we should then consider 'open' or 'load', but
neither of these sound good.)

Saving the work-in-progress is also like a deferred not-quite commit, so
some variant of the 'commit' command such as 'svn commit [--]save' might
be appropriate.

A reason for choosing a noun (phrase) such as 'checkpoint', 'savepoint',
'changelist' or 'changeset' followed by a verb is to allow us to add
verbs without cluttering the main command name space. I am thinking for
example that we might want to 'send' this change to a review system or
'commit' it to the svn server (ok we'd use 'svn commit <something>' for
that) or export it to a patch or email (git format-patch). The main
actions on a Perforce changelist are done by first-class commands 'p4
submit', 'p4 shelve', etc. while the 'p4 change[list]' command has
(option-letter) verbs for 'input' and 'output' actions.

Any suggestions for the 'shelve' terminology problem? Change to 'stash'
and incur false expectations based on git stash? Is there something else
that, like 'stash', can be used as a noun and a verb and sounds good?

- Julian
Reply | Threaded
Open this post in threaded view
|

Re: Checkpointing v1 design -- terminology

Daniel Shahaf-2
Julian Foad wrote on Fri, Nov 03, 2017 at 21:51:37 +0000:

> Branko Čibej wrote:
> > >    * main commands are 'checkpoint' to save a new version and
> > > 'rollback' to revert to version N
> >
> > The main issue I have with this is that "checkpoint" is not a verb. (I
> > know, neither are 'changelist' or 'auth').
> >
> > How about 'save' (to create a savepoint) and 'restore' instead?
> >
> > As to why I'm nitpicking at this point: whatever we choose will be
> > pretty much set in stone, in the UI and the API, for the conceivable future.
>
> Thanks for helping with the terminology. I agree it's important.
>
> The prototype CLI I'm working on (in the 'shelve-checkpoint' branch)
> actually has 'svn checkpoint save' and 'svn checkpoint rollback'. Do you
> think mean to suggest plain 'svn save' and 'svn restore'? I'm not sure about
> that. They could deserve to be first-class 'svn' commands if we consider
> them first-class companions to 'commit', 'revert' and 'diff'.
>
> The analogy to 'save' a file or the work in almost any application is apt.
> (For the converse we should then consider 'open' or 'load', but neither of
> these sound good.)

"edit"?  Inspired by Vim's :edit command; goes well with the "prototype
commit" mental model.

> Any suggestions for the 'shelve' terminology problem? Change to 'stash' and
> incur false expectations based on git stash? Is there something else that,
> like 'stash', can be used as a noun and a verb and sounds good?

pile, pocket, fold, store, house, dock, box, rack, hold, stow.

But I don't understand what's the problem that we're trying to solve
here.  Why is it a problem that "shelf" is a noun and "shelve" a verb?
And for that matter, why couldn't we use "checkpoint" as a verb?

Cheers,

Daniel
Reply | Threaded
Open this post in threaded view
|

Re: Checkpointing v1 design

Vincent Lefevre-11
In reply to this post by Branko Čibej
On 2017-11-03 19:39:51 +0100, Branko Čibej wrote:

> On 03.11.2017 15:54, Julian Foad wrote:
> > After playing with two pre-prototypes and discussing a wide variety of
> > ideas on this list, I have given detailed thought to a v1
> > checkpointing design with the following properties:
> >
> >   * local (not server)
> >   * built on shelving v1 (patches)
> >   * main commands are 'checkpoint' to save a new version and
> > 'rollback' to revert to version N
>
> The main issue I have with this is that "checkpoint" is not a verb. (I
> know, neither are 'changelist' or 'auth').
>
> How about 'save' (to create a savepoint) and 'restore' instead?

Wouldn't it be better if these related commands have a common prefix?
A bit like property-related commands start with "prop".

--
Vincent Lefèvre <[hidden email]> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)
Reply | Threaded
Open this post in threaded view
|

Re: Checkpointing v1 design -- terminology

Branko Čibej
In reply to this post by Julian Foad-5
On 03.11.2017 22:51, Julian Foad wrote:

> Branko Čibej wrote:
>>>    * main commands are 'checkpoint' to save a new version and
>>> 'rollback' to revert to version N
>>
>> The main issue I have with this is that "checkpoint" is not a verb. (I
>> know, neither are 'changelist' or 'auth').
>>
>> How about 'save' (to create a savepoint) and 'restore' instead?
>>
>> As to why I'm nitpicking at this point: whatever we choose will be
>> pretty much set in stone, in the UI and the API, for the conceivable
>> future.
>
> Thanks for helping with the terminology. I agree it's important.
>
> The prototype CLI I'm working on (in the 'shelve-checkpoint' branch)
> actually has 'svn checkpoint save' and 'svn checkpoint rollback'. Do
> you think mean to suggest plain 'svn save' and 'svn restore'?

No, the two-keyword approach is actually better. I've often wished we'd
made our argument parser slightly more flexible in that respect, so that
instead of

    svn command [options args]

we could write

    svn [topic] command [options args]

which would be useful for topics such as "(save|check)point",
"changelist", "auth", "branch" and so on. I'm absolutely in favour of
the second form for complex cases.

[...]

> A reason for choosing a noun (phrase) such as 'checkpoint',
> 'savepoint', 'changelist' or 'changeset' followed by a verb is to
> allow us to add verbs without cluttering the main command name space.

Precisely. +1

FWIW, the reason I'm slightly in favour of "savepoint" vs. "checkpoint"
has been discussed before: the abbreviation "sp" is not taken yet, but
"cp" is; I suspect it would save "power users" of the command line a ton
of grief if they could write "svn sp s[quash] 0:-3" instead of "svn cp
squash --from 0 --to -3" and then get a syntax error. :)


> I am thinking for example that we might want to 'send' this change to
> a review system or 'commit' it to the svn server (ok we'd use 'svn
> commit <something>' for that) or export it to a patch or email (git
> format-patch). The main actions on a Perforce changelist are done by
> first-class commands 'p4 submit', 'p4 shelve', etc. while the 'p4
> change[list]' command has (option-letter) verbs for 'input' and
> 'output' actions.
>
> Any suggestions for the 'shelve' terminology problem? Change to
> 'stash' and incur false expectations based on git stash? Is there
> something else that, like 'stash', can be used as a noun and a verb
> and sounds good?

IIUC the shelving prototype is a lot closer to Perforce's than Git's,
right? Then thinking about shelves is better than about stashes. Why not
do the same as with (save|check)points: the [topic] should be a noun,
the [command] a verb, so:

    svn shelf save ...
    svn shelf restore ...

I'm not sure about the abbreviation here ... "sh" sounds tempting but is
also a well-known abbreviation for "shell".

-- Brane
Reply | Threaded
Open this post in threaded view
|

Re: Checkpointing v1 design -- terminology

Branko Čibej
In reply to this post by Daniel Shahaf-2
On 04.11.2017 00:21, Daniel Shahaf wrote:
> But I don't understand what's the problem that we're trying to solve
> here.  Why is it a problem that "shelf" is a noun and "shelve" a verb?
> And for that matter, why couldn't we use "checkpoint" as a verb?

Traditionally we've striven to appear marginally literate in the text we
produce, that includes command structure. There's no harm in trying to
adhere to that tradition, especially as it does not cost us more than
some chattering on the developers' list.

"Checkpoint" is not a verb. Written English has deteriorated to the
toddler level in the last several decades, but I see no reason for us to
follow suit.

-- Brane

Reply | Threaded
Open this post in threaded view
|

Re: Checkpointing v1 design -- terminology

Daniel Shahaf-2
Branko Čibej wrote on Sat, 04 Nov 2017 13:51 +0100:
> Traditionally we've striven to appear marginally literate in the text we
> produce, that includes command structure. There's no harm in trying to
> adhere to that tradition, especially as it does not cost us more than
> some chattering on the developers' list.
>
> "Checkpoint" is not a verb. Written English has deteriorated to the
> toddler level in the last several decades, but I see no reason for us to
> follow suit.

I agree that we should use correct English but I don't see how using
"checkpoint" as a verb would be a problem in any way.

Let's just agree to disagree.
Reply | Threaded
Open this post in threaded view
|

Re: Checkpointing v1 design -- terminology

Julian Foad-5
In reply to this post by Branko Čibej
Branko Čibej wrote:
> [...] I've often wished we'd
> made our argument parser slightly more flexible in that respect, so that
> [...] we could write
>
>      svn [topic] command [options args]

> FWIW, the reason I'm slightly in favour of "savepoint" vs. "checkpoint"
> has been discussed before: the abbreviation "sp" is not taken yet, [...]

Ack.


>> I am thinking for example that we might want to 'send' this change to
>> a review system or 'commit' it to the svn server (ok we'd use 'svn
>> commit <something>' for that) or export it to a patch or email (git
>> format-patch). The main actions on a Perforce changelist are done by
>> first-class commands 'p4 submit', 'p4 shelve', etc. while the 'p4
>> change[list]' command has (option-letter) verbs for 'input' and
>> 'output' actions.
>>
>> Any suggestions for the 'shelve' terminology problem? Change to
>> 'stash' and incur false expectations based on git stash? Is there
>> something else that, like 'stash', can be used as a noun and a verb
>> and sounds good?
>
> IIUC the shelving prototype is a lot closer to Perforce's than Git's,
> right? Then thinking about shelves is better than about stashes. Why not
> do the same as with (save|check)points: the [topic] should be a noun,
> the [command] a verb, so:
>
>      svn shelf save ...
>      svn shelf restore ...

Next observation: 'shelves' and 'checkpoints' are not two alternative
kinds of thing; rather savepoints/checkpoints are versions of a
(shelved) change.

So 'save this point' and 'roll back to a saved point' need not be in
grouped in their own 'savepoint' topic namespace but would perhaps be
better described as actions on a (potentially shelved) change.

'Roll back' means retrieving an old version of a shelved change and
could be named like one or a permutation of these:

   svn unshelve --checkpoint=3 foo
   svn shelf restore foo@3
   svn revert --savepoint=3 --cl=foo
   svn changelist restore -v3 foo

(I am deliberately bringing 'changelist' into the mix, as I believe
integration with 'changelist' is where we need to take this.)

I was particularly looking at the case where we have some local
modifications already and we want to 'roll back' to an earlier version
of the files. I had been thinking about how Subversion should decide
which modifications in the WC it should revert before restoring the old
version. E.g. revert(union(files in requested old version, files in
most-recently-restored version)). That seems prone to unintended
consequences as the 'most recently restored version isn't necessarily
known at all, or correctly known in cases where the user restored
something and then manually reverted and applied some other change. Also
the set of files that was previously recorded may not be the same as the
set of files currently undergoing editing, as it is common to start
affecting other files. (To handle this sort of case, we need the
'changelist' mechanism augmented by the notion of assigning new changes
to a 'default changelist'.)

Instead of all that uncertainty, let the user be responsible for
reverting the current modifications currently in progress, and then roll
back using a simple form of 'unshelve' that does not attempt to revert
anything first. For added safety, 'unshelve' should warn (e.g. require
'force' to override) if it attempts to modify any already-modified
files. (This was already a suggested enhancement.)

- Julian
Reply | Threaded
Open this post in threaded view
|

Re: Checkpointing v1 design -- terminology

Julian Foad-5
Julian Foad wrote:

> 'Roll back' means retrieving an old version of a shelved change and
> could be named like one or a permutation of these:
>
>    svn unshelve --checkpoint=3 foo
>    svn shelf restore foo@3
>    svn revert --savepoint=3 --cl=foo
>    svn changelist restore -v3 foo
>
> (I am deliberately bringing 'changelist' into the mix, as I believe
> integration with 'changelist' is where we need to take this.)

The integration with changelists that I am thinking of is basically that
changelist FOO corresponds to the working (unshelved) version of a
shelved change named FOO. Unshelving or restoring FOO will apply the
saved change FOO and assign the affected files to changelist FOO; and
saving or shelving FOO will use the changelist FOO as the set of files
to save or shelve.

With this changelist integration we could have... (I know we can't
compatibly have exactly this syntax; see below for alternatives)...

Shelve = save and revert:
   svn cl shelve FOO

Checkpoint = save and don't revert:
   svn cl save FOO

Unshelve / restore / roll back (all possible names for same thing):
   svn cl unshelve [-v VERSION_NUM] FOO
   svn cl restore [-v VERSION_NUM] FOO
   (etc.)

Revert files in (active) changelist FOO:
   svn revert --cl=FOO  # existing syntax
   svn cl revert FOO

Commit the (possibly shelved) changelist FOO:
   svn commit --cl=FOO  # existing syntax
   svn cl commit FOO


Possible different ways to write the topic noun and the verb:

   svn clsave          FOO [PATH...]
   svn cl save         FOO [PATH...]
   svn cl --save       FOO [PATH...]
   svn changelist save FOO [PATH...]
   svn changelist-save FOO [PATH...]

   svn save       --cl=FOO [PATH...]


The key point I am making here is the "topic" we're talking about in
these commands is the "changelist" rather than the "savepoint" which is
merely a stored version of a changelist.

- Julian
Reply | Threaded
Open this post in threaded view
|

Re: Checkpointing v1 design -- terminology

Paul Hammant-3
I'd be happier with save-cl rather than ci-save.  Or just 'save', as I'm not sure what you're saving if not a change list.  Incidentally CL looks pretty close to CI in lower case and that's already used.

Spaces allowed in FOO name?

$ svn save -v2 "add hair color to person page"

Also..

$ svn save --revert -v2 "add hair color to person page"

.. does the save and THEN drops the CL and it's changes in working copy - back to no changes but not necessarily up to date.

> Revert files in (active) changelist FOO:
>   svn revert --cl=FOO  # existing syntax

I don't know what this one does Julian?
Reply | Threaded
Open this post in threaded view
|

Re: Checkpointing v1 design -- terminology

Branko Čibej
In reply to this post by Julian Foad-5
On 06.11.2017 17:30, Julian Foad wrote:
> The key point I am making here is the "topic" we're talking about in
> these commands is the "changelist" rather than the "savepoint" which
> is merely a stored version of a changelist.

All our current commands that use changelists use them as an alias for a
set of targets. Instead of 'svn commit foo bar baz' you'd say 'svn
commit --changelist fbb'.

Your proposal elevates changelists to a status that they do not
currently have in the client, and makes their usage inconsistent with
the rest of the command line. You might as well propose

    svn changelist commit
    svn changelist diff
    svn changelist switch

and so on. Whether that would be a good idea is not the issue here: the
issue is consistency, and (possibly) backwards-compatibility. And the
rather important detail that changelists cannot be associated with
directories.

Changelists were always a sort of afterthought ... unless you plan to
elevate them to first-class concepts across the whole command-line, I
recommend that the shelve/save/whatnot commands simply accept a
--changelist option and not complicate further. That this doesn't
preclude populating a changelist when a shelved change is restored.

On another note: changelists with checkpoints make no sense IMO; a
checkpoint should save and restore the state of the whole working copy.
Shelved changes are different. Their internal representation may be the
same, but the semantics are not.

-- Brane
Reply | Threaded
Open this post in threaded view
|

Re: Checkpointing v1 design -- terminology

Julian Foad-5
Branko Čibej wrote:

> On 06.11.2017 17:30, Julian Foad wrote:
>> The key point I am making here is the "topic" we're talking about in
>> these commands is the "changelist" rather than the "savepoint" which
>> is merely a stored version of a changelist.
>
> All our current commands that use changelists use them as an alias for a
> set of targets. [...]
>
> Your proposal elevates changelists to a status that they do not
> currently have in the client, and makes their usage inconsistent with
> the rest of the command line. You might as well propose
>
>      svn changelist commit
>      svn changelist diff
>      svn changelist switch
>
> and so on. Whether that would be a good idea is not the issue here: the
> issue is consistency, and (possibly) backwards-compatibility.

I absolutely am considering elevating the status of changelists to that
level, yes! That makes a very good model in my opinion.

> And the
> rather important detail that changelists cannot be associated with
> directories.

Yes, that's noted as a needed enhancement.

> Changelists were always a sort of afterthought ...

Indeed, that made me sad from their beginning. And what better
opportunity to at last make a plan to improve them?

> unless you plan to
> elevate them to first-class concepts across the whole command-line, I
> recommend that the shelve/save/whatnot commands simply accept a
> --changelist option and not complicate further. That this doesn't
> preclude populating a changelist when a shelved change is restored.
>
> On another note: changelists with checkpoints make no sense IMO; a
> checkpoint should save and restore the state of the whole working copy.
> Shelved changes are different. Their internal representation may be the
> same, but the semantics are not.

Why do you think 'checkpointing' your work should be across the whole WC
whereas 'shelving' it should not? I can see how 'checkpoint your whole
WC' has a utility, sure, but no more so than 'shelve your whole WC'. I
think it makes a great deal of sense to checkpoint a unit of work rather
than the whole WC, and to work towards elevating 'changelist' to fulfil
that role.

- Julian
Reply | Threaded
Open this post in threaded view
|

Re: Checkpointing v1 design -- terminology

Branko Čibej
On 07.11.2017 16:15, Julian Foad wrote:

> Branko Čibej wrote:
>> On 06.11.2017 17:30, Julian Foad wrote:
>>> The key point I am making here is the "topic" we're talking about in
>>> these commands is the "changelist" rather than the "savepoint" which
>>> is merely a stored version of a changelist.
>>
>> All our current commands that use changelists use them as an alias for a
>> set of targets. [...]
>>
>> Your proposal elevates changelists to a status that they do not
>> currently have in the client, and makes their usage inconsistent with
>> the rest of the command line. You might as well propose
>>
>>      svn changelist commit
>>      svn changelist diff
>>      svn changelist switch
>>
>> and so on. Whether that would be a good idea is not the issue here: the
>> issue is consistency, and (possibly) backwards-compatibility.
>
> I absolutely am considering elevating the status of changelists to
> that level, yes! That makes a very good model in my opinion.
>
>> And the
>> rather important detail that changelists cannot be associated with
>> directories.
>
> Yes, that's noted as a needed enhancement.
>
>> Changelists were always a sort of afterthought ...
>
> Indeed, that made me sad from their beginning. And what better
> opportunity to at last make a plan to improve them?

Noted. +1. Enjoy the corner cases. :)


>> On another note: changelists with checkpoints make no sense IMO; a
>> checkpoint should save and restore the state of the whole working copy.
>> Shelved changes are different. Their internal representation may be the
>> same, but the semantics are not.
>
> Why do you think 'checkpointing' your work should be across the whole
> WC whereas 'shelving' it should not? I can see how 'checkpoint your
> whole WC' has a utility, sure, but no more so than 'shelve your whole
> WC'. I think it makes a great deal of sense to checkpoint a unit of
> work rather than the whole WC, and to work towards elevating
> 'changelist' to fulfil that role.

I shudder to think of the interactions and side effects of partial
checkpoints. Consider our standard Greek tree and the following sequence:

 1. <do stuff>
 2. svn checkpoint save "#1" A/B A/D/G
 3. <do more stuff>
 4. svn checkpoint save "#2" A/B/F A/D
 5. <do even more extremely tricky stuff>
 6. svn checkpoint rollback "#1"
 7. svn checkpoint rollback "#2"


Um ... is that a rebase, a reorder, or a pretzel? Or does rolling back
to the state before #1 imply rolling back #2 as well? If the latter,
then checkpoints form a stack and represent the whole WC state.

-- Brane

Reply | Threaded
Open this post in threaded view
|

Re: Checkpointing v1 design -- terminology

Julian Foad-5
Branko Čibej wrote:
> I shudder to think of the interactions and side effects of partial
> checkpoints. Consider our standard Greek tree and the following sequence:

First think about saving checkpoints the manual way, to set our initial
expectations fairly low:

   svn diff foo-paths > foo-4.patch
   svn diff bar-paths > bar-17.patch

Then let's look at how we can improve on that.

>   1. <do stuff>
>   2. svn checkpoint save "#1" A/B A/D/G
>   3. <do more stuff>
>   4. svn checkpoint save "#2" A/B/F A/D

I assume (1) these named changelists '#1' and '#2' were not defined (so
not checkpointed) before step 1; and (2) you intend these 'save'
commands to act with depth 'infinity' on the specified subtrees.

So you are concerned about how to handle overlapping subtrees -- in
particular the case where a directory is associated with one changelist,
and a subdirectory within it is (already or later) associated with another.

>   5. <do even more extremely tricky stuff>

I'll interpret that as 'do normal Subversion operations' rather than
'execute arbitrary SQL statements on wc.db' :-)

>   6. svn checkpoint rollback "#1"
>   7. svn checkpoint rollback "#2"

OK, now is a good time to think about possible semantics. Here is a
starting point.

   * A subtree is associated with a changelist either explicitly by
attaching the changelist name to its root directory, or if not
explicitly then implicitly by inheritance from its parent. A subtree
association applies only to parts of the subtree that are not associated
with a different changelist.

(Therefore a directory can be associated with only one changelist, and
an explicit assignment overrides any association that would otherwise be
inherited from its parents. After step 4 of your example everything in
A/B is associated with '#1' *except* everything in A/B/F which is
associated with '#2'.)

   * 'Saving' a changelist 'NAME' records the explicit attachment of
NAME to particular PATHS.

   * Restoring a changelist 'NAME' restores the explicit attachment of
NAME to particular PATHS. If some paths are already associated with
changelist NAME, behaviour is client-defined, e.g. error, append to
exiting assignments, replace existing assignments, with or without a
warning. Let's say the initial CLI behaviour is error, for simplicity.
(In other words the user has to revert before restoring, as I suggested
a little earlier in this thread.)

   * The syntax '... save NAME PATHS...' first assigns changelist NAME
to PATH..., replacing any previous assignment for PATH..., and then
saves all paths belonging to NAME.

With this, your step 2 would save the whole subtree at A/B (and the one
at A/D/G) in a first checkpoint of '#1', while your step 4 would
unassign A/B/F from changelist '#1' and assign it to changelist '#2'
instead.

You can work out the 'rollbacks' from there (I haven't time right now);
I acknowledge it won't do magic for you -- but I'm sure you wouldn't
want it to.

How does that strike you from a pragmatic point of view?

- Julian
Reply | Threaded
Open this post in threaded view
|

Re: Checkpointing v1 design -- terminology

Julian Foad-5
Julian Foad wrote:
> You can work out the 'rollbacks' from there (I haven't time right now);

(That sounded like a crass cop-out, but I seriously didn't have a moment
more to spare to think about it right then.)

The 'rollback' commands as written in steps 6 and 7 would both error. To
achieve such rollbacks, the user would have to first revert, which
involves deciding what to revert.

And that seems OK to me.

- Julian
Reply | Threaded
Open this post in threaded view
|

Re: Checkpointing v1 design -- terminology

Julian Foad-5
In reply to this post by Paul Hammant-3
Paul Hammant wrote:
> I'd be happier with save-cl rather than ci-save.  Or just 'save', as I'm
> not sure what you're saving if not a change list.  Incidentally CL looks
> pretty close to CI in lower case and that's already used.
>
> Spaces allowed in FOO name?
>
> $ svn save -v2 "add hair color to person page"

Yes. The name obeys existing rules for a changelist name, for forward
compatibility, and that does allow spaces.

> Also..
>
> $ svn save --revert -v2 "add hair color to person page"
>
> .. does the save and THEN drops the CL and it's changes in working copy
> - back to no changes but not necessarily up to date.

That sounds like you're proposing an alternative syntax for what the
'shelve' command does -- except I'm not sure what you want the 'v2'
option to do here; was that accidental?

>> Revert files in (active) changelist FOO:
>>   svn revert --cl=FOO  # existing syntax
>
> I don't know what this one does Julian?

I abbreviated a bit; the actual existing syntax for reverting the files
tagged as changelist 'FOO' is:

   svn revert --cl=FOO -R .

(assuming all the files you need to revert are under '.').

- Julian

Reply | Threaded
Open this post in threaded view
|

Re: Checkpointing v1 design -- terminology

Julian Foad-5
In reply to this post by Julian Foad-5
Julian Foad wrote:
> [...] To achieve such rollbacks, the user would have to first revert, which
> involves deciding what to revert.
>
> And that seems OK to me.

The larger point is that this kind of work flow, and the difficulty of
working with changes in overlapping subtrees, is analogous to some
existing techniques. For example, if one tries to use two different
branches to manage two different change-sets, and work with them both in
one WC by switching part of the WC to one branch and another part to
another branch, and the changes involve overlapping subdirectories,
maybe like this...

  svn switch "^/branches/#1" A/B
  svn switch "^/branches/#2" A/B/F A/D
  svn switch "^/branches/#1"       A/D/G

... it seems to me that more or less the same issues arise.

Looking more specifically at one of the details, we note that 'svn
switch URL PATH' recursively overrides the current URL on PATH and all
its descendents (hence having to write the above example as three
steps), whereas 'svn update' doesn't, and we have yet to define whether
the proposed 'restore' command would do one or the other or something
different. But if it's broadly analogous it seems reasonable.

- Julian
Reply | Threaded
Open this post in threaded view
|

Re: Checkpointing v1 design -- terminology

Branko Čibej
On 10.11.2017 14:20, Julian Foad wrote:

> Julian Foad wrote:
>> [...] To achieve such rollbacks, the user would have to first revert,
>> which involves deciding what to revert.
>>
>> And that seems OK to me.
>
> The larger point is that this kind of work flow, and the difficulty of
> working with changes in overlapping subtrees, is analogous to some
> existing techniques. For example, if one tries to use two different
> branches to manage two different change-sets, and work with them both
> in one WC by switching part of the WC to one branch and another part
> to another branch, and the changes involve overlapping subdirectories,
> maybe like this...
>
>  svn switch "^/branches/#1" A/B
>  svn switch "^/branches/#2" A/B/F A/D
>  svn switch "^/branches/#1"       A/D/G
>
> ... it seems to me that more or less the same issues arise.

To be fair, I've yet to hear of a VCS that can handle this sort of
complexity and make it easy for users. Git cops out by having
all-or-nothing branches, commits, etc. But add submodules and you're in
the same leaky boat again.

-- Brane

12