[PATCH] Add '--include' and '--exclude' options to 'svnadmin dump'

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

[PATCH] Add '--include' and '--exclude' options to 'svnadmin dump'

Sergey Raevskiy
Hi!

I've attached a patch that adds '--include/--exclude' options to 'svnadmin
dump'. These options work similarly to 'svndumpfilter include/exclude'
but provide proper handling of 'copy from' paths.

Consider the following example with svndumpfilter:
[[
$ svnadmin create /repo
$ svn mkdir -m "" file:///repo/A
$ svn copy -m "" file:///repo/A file:///repo/B
$ svnadmin dump /repo | svndumpfilter include /B > dump
...
Revision 0 committed as 0.
Revision 1 committed as 1.
svndumpfilter: E200003: Invalid copy source path '/A'
]]

The 'svnadmin dump' with proposed include/exclude options can handle this use
case properly:
[[
$ svnadmin dump /repo --include /B > dump
* Dumped revision 0.
* Dumped revision 1.
* Dumped revision 2.
]]

The proposed options are implemented using 'svn_repos_authz_func_t' and the
not-included copy sources are hidden by the 'svn_repos' layer.

Some usage examples for the proposed options:
[[
$ svnadmin dump repos --include /calc > calc-dumpfile
...
$ svnadmin dump repos --include /calc --include /calendar > dumpfile
...
$ svnadmin dump repos --exclude /secret > dumpfile
...
$ svnadmin dump repos --include /calc --exclude /calendar > dumpfile
svnadmin: E205000: Try 'svnadmin help' for more info
svnadmin: E205000: '--exclude' and '--include' options cannot be used
simultaneously

$ svnadmin dump repos --include /cal* --pattern > dumpfile
...
]]

Log message:

[[
Add '--include' and '--exclude' options to 'svnadmin dump'.

* include/svn_repos.h
  (svn_repos_dump_filter_func_t): New.
  (svn_repos_dump_fs4): Update function signature and comment.
  (svn_repos_dump_fs3): Update comment.

* libsvn_repos/deprecated.c
  (svn_repos_dump_fs3): Update caller.

* libsvn_repos/dump.c
  (write_revision_record): Call to svn_repos_fs_revision_proplist() (with
   AUTHZ_FUNC) instead of svn_fs_revision_proplist2() to filter revision
   properties as well as revisions. Update comment.
  (dump_filter_baton_t,
   dump_filter_authz_func): New.
  (svn_repos_dump_fs4): Initialize and pass AUTHZ_FUNC and AUTHZ_BATON to
   the repos layer API if FILTER_FUNC is specified by caller.

* subversion/svnadmin/svnadmin.c
  (svnadmin__cmdline_options_t): Add enum values for new options.
  (options_table): Add new options.
  (cmd_table): Add new options to 'dump' subcommand.
  (svnadmin_opt_state): Add new fields to represent new options.
  (ary_prefix_match): New. Copied from svndumpfilter.
  (dump_filter_baton_t,
   dump_filter_func): New.
  (subcommand_dump): Initialize FILTER_BATON.  Pass DUMP_FILTER_FUNC and a
   pointer to FILTER_BATON to svn_repos_dump_fs() if any filtering prefixes
   specified.
  (sub_main): Handle new options.

* subversion/tests/cmdline/svnadmin_tests.py
  (dump_exclude,
   dump_exclude_copysource,
   dump_include,
   dump_not_include_copysource,
   dump_exclude_by_pattern,
   dump_include_by_pattern,
   dump_exclude_all_rev_changes,
   dump_invalid_filtering_option): New.
  (test_list): Add new tests to table.

* subversion/tests/libsvn_repos/dump-load-test.c
  (test_dump_bad_props): Update caller.

Patch by: sergey.raevskiy{_AT_}visualsvn.com
]]

dump-exclude-include.patch.txt (40K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

RE: [PATCH] Add '--include' and '--exclude' options to 'svnadmin dump'

Bert Huijben-5


> -----Original Message-----
> From: Sergey Raevskiy [mailto:[hidden email]]
> Sent: woensdag 12 oktober 2016 12:44
> To: Subversion Development <[hidden email]>
> Subject: [PATCH] Add '--include' and '--exclude' options to 'svnadmin dump'
>
> Hi!
>
> I've attached a patch that adds '--include/--exclude' options to 'svnadmin
> dump'. These options work similarly to 'svndumpfilter include/exclude'
> but provide proper handling of 'copy from' paths.
>
> Consider the following example with svndumpfilter:
> [[
> $ svnadmin create /repo
> $ svn mkdir -m "" file:///repo/A
> $ svn copy -m "" file:///repo/A file:///repo/B
> $ svnadmin dump /repo | svndumpfilter include /B > dump

I don't see a --renumber-revs on your invocation of svndumpfilter. Did you try these options to see if these resolve your problem (they should)

$ svndumpfilter help include
include: Filter out nodes without given prefixes from dumpstream.
usage: svndumpfilter include PATH_PREFIX...

Valid options:
  --drop-empty-revs        : Remove revisions emptied by filtering.
  --drop-all-empty-revs    : Remove all empty revisions found in dumpstream
                             except revision 0.
  --renumber-revs          : Renumber revisions left after filtering.
  --skip-missing-merge-sources : Skip missing merge sources.
  --targets ARG            : Read additional prefixes, one per line, from
                             file ARG.
  --preserve-revprops      : Don't filter revision properties.
  --quiet                  : Do not display filtering statistics.
  --pattern                : Treat the path prefixes as file glob patterns.


        Bert

> ...
> Revision 0 committed as 0.
> Revision 1 committed as 1.
> svndumpfilter: E200003: Invalid copy source path '/A'
> ]]
>
> The 'svnadmin dump' with proposed include/exclude options can handle this
> use
> case properly:
> [[
> $ svnadmin dump /repo --include /B > dump
> * Dumped revision 0.
> * Dumped revision 1.
> * Dumped revision 2.
> ]]
>
> The proposed options are implemented using 'svn_repos_authz_func_t' and the
> not-included copy sources are hidden by the 'svn_repos' layer.
>
> Some usage examples for the proposed options:
> [[
> $ svnadmin dump repos --include /calc > calc-dumpfile
> ...
> $ svnadmin dump repos --include /calc --include /calendar > dumpfile
> ...
> $ svnadmin dump repos --exclude /secret > dumpfile
> ...
> $ svnadmin dump repos --include /calc --exclude /calendar > dumpfile
> svnadmin: E205000: Try 'svnadmin help' for more info
> svnadmin: E205000: '--exclude' and '--include' options cannot be used
> simultaneously
>
> $ svnadmin dump repos --include /cal* --pattern > dumpfile
> ...
> ]]
>
> Log message:
>
> [[
> Add '--include' and '--exclude' options to 'svnadmin dump'.
>
> * include/svn_repos.h
>   (svn_repos_dump_filter_func_t): New.
>   (svn_repos_dump_fs4): Update function signature and comment.
>   (svn_repos_dump_fs3): Update comment.
>
> * libsvn_repos/deprecated.c
>   (svn_repos_dump_fs3): Update caller.
>
> * libsvn_repos/dump.c
>   (write_revision_record): Call to svn_repos_fs_revision_proplist() (with
>    AUTHZ_FUNC) instead of svn_fs_revision_proplist2() to filter revision
>    properties as well as revisions. Update comment.
>   (dump_filter_baton_t,
>    dump_filter_authz_func): New.
>   (svn_repos_dump_fs4): Initialize and pass AUTHZ_FUNC and AUTHZ_BATON
> to
>    the repos layer API if FILTER_FUNC is specified by caller.
>
> * subversion/svnadmin/svnadmin.c
>   (svnadmin__cmdline_options_t): Add enum values for new options.
>   (options_table): Add new options.
>   (cmd_table): Add new options to 'dump' subcommand.
>   (svnadmin_opt_state): Add new fields to represent new options.
>   (ary_prefix_match): New. Copied from svndumpfilter.
>   (dump_filter_baton_t,
>    dump_filter_func): New.
>   (subcommand_dump): Initialize FILTER_BATON.  Pass DUMP_FILTER_FUNC
> and a
>    pointer to FILTER_BATON to svn_repos_dump_fs() if any filtering prefixes
>    specified.
>   (sub_main): Handle new options.
>
> * subversion/tests/cmdline/svnadmin_tests.py
>   (dump_exclude,
>    dump_exclude_copysource,
>    dump_include,
>    dump_not_include_copysource,
>    dump_exclude_by_pattern,
>    dump_include_by_pattern,
>    dump_exclude_all_rev_changes,
>    dump_invalid_filtering_option): New.
>   (test_list): Add new tests to table.
>
> * subversion/tests/libsvn_repos/dump-load-test.c
>   (test_dump_bad_props): Update caller.
>
> Patch by: sergey.raevskiy{_AT_}visualsvn.com
> ]]

Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] Add '--include' and '--exclude' options to 'svnadmin dump'

Sergey Raevskiy
Hi Bert, thanks for your reply.

The problem that I've described is not about revision numbers, it's about copied
nodes.

Suppose we have an repository 'repo' containing nodes '/A' and '/B',
where '/B' (or some path under '/B') was create by copying of '/A' (or some
path under '/A').

Now, if we run 'svndumpfilter include /B' against dump of 'repo', we will get
an error:

[[
...
svndumpfilter: E200003: Invalid copy source path '/A'
]]

This happens because 'svndumpfilter' tries to add copy of '/A', but node '/A'
is not included in destination dump.

On Wed, Oct 12, 2016 at 2:10 PM, Bert Huijben <[hidden email]> wrote:

>
>
>> -----Original Message-----
>> From: Sergey Raevskiy [mailto:[hidden email]]
>> Sent: woensdag 12 oktober 2016 12:44
>> To: Subversion Development <[hidden email]>
>> Subject: [PATCH] Add '--include' and '--exclude' options to 'svnadmin dump'
>>
>> Hi!
>>
>> I've attached a patch that adds '--include/--exclude' options to 'svnadmin
>> dump'. These options work similarly to 'svndumpfilter include/exclude'
>> but provide proper handling of 'copy from' paths.
>>
>> Consider the following example with svndumpfilter:
>> [[
>> $ svnadmin create /repo
>> $ svn mkdir -m "" file:///repo/A
>> $ svn copy -m "" file:///repo/A file:///repo/B
>> $ svnadmin dump /repo | svndumpfilter include /B > dump
>
> I don't see a --renumber-revs on your invocation of svndumpfilter. Did you try these options to see if these resolve your problem (they should)
>
> $ svndumpfilter help include
> include: Filter out nodes without given prefixes from dumpstream.
> usage: svndumpfilter include PATH_PREFIX...
>
> Valid options:
>   --drop-empty-revs        : Remove revisions emptied by filtering.
>   --drop-all-empty-revs    : Remove all empty revisions found in dumpstream
>                              except revision 0.
>   --renumber-revs          : Renumber revisions left after filtering.
>   --skip-missing-merge-sources : Skip missing merge sources.
>   --targets ARG            : Read additional prefixes, one per line, from
>                              file ARG.
>   --preserve-revprops      : Don't filter revision properties.
>   --quiet                  : Do not display filtering statistics.
>   --pattern                : Treat the path prefixes as file glob patterns.
>
>
>         Bert
>
>> ...
>> Revision 0 committed as 0.
>> Revision 1 committed as 1.
>> svndumpfilter: E200003: Invalid copy source path '/A'
>> ]]
>>
>> The 'svnadmin dump' with proposed include/exclude options can handle this
>> use
>> case properly:
>> [[
>> $ svnadmin dump /repo --include /B > dump
>> * Dumped revision 0.
>> * Dumped revision 1.
>> * Dumped revision 2.
>> ]]
>>
>> The proposed options are implemented using 'svn_repos_authz_func_t' and the
>> not-included copy sources are hidden by the 'svn_repos' layer.
>>
>> Some usage examples for the proposed options:
>> [[
>> $ svnadmin dump repos --include /calc > calc-dumpfile
>> ...
>> $ svnadmin dump repos --include /calc --include /calendar > dumpfile
>> ...
>> $ svnadmin dump repos --exclude /secret > dumpfile
>> ...
>> $ svnadmin dump repos --include /calc --exclude /calendar > dumpfile
>> svnadmin: E205000: Try 'svnadmin help' for more info
>> svnadmin: E205000: '--exclude' and '--include' options cannot be used
>> simultaneously
>>
>> $ svnadmin dump repos --include /cal* --pattern > dumpfile
>> ...
>> ]]
>>
>> Log message:
>>
>> [[
>> Add '--include' and '--exclude' options to 'svnadmin dump'.
>>
>> * include/svn_repos.h
>>   (svn_repos_dump_filter_func_t): New.
>>   (svn_repos_dump_fs4): Update function signature and comment.
>>   (svn_repos_dump_fs3): Update comment.
>>
>> * libsvn_repos/deprecated.c
>>   (svn_repos_dump_fs3): Update caller.
>>
>> * libsvn_repos/dump.c
>>   (write_revision_record): Call to svn_repos_fs_revision_proplist() (with
>>    AUTHZ_FUNC) instead of svn_fs_revision_proplist2() to filter revision
>>    properties as well as revisions. Update comment.
>>   (dump_filter_baton_t,
>>    dump_filter_authz_func): New.
>>   (svn_repos_dump_fs4): Initialize and pass AUTHZ_FUNC and AUTHZ_BATON
>> to
>>    the repos layer API if FILTER_FUNC is specified by caller.
>>
>> * subversion/svnadmin/svnadmin.c
>>   (svnadmin__cmdline_options_t): Add enum values for new options.
>>   (options_table): Add new options.
>>   (cmd_table): Add new options to 'dump' subcommand.
>>   (svnadmin_opt_state): Add new fields to represent new options.
>>   (ary_prefix_match): New. Copied from svndumpfilter.
>>   (dump_filter_baton_t,
>>    dump_filter_func): New.
>>   (subcommand_dump): Initialize FILTER_BATON.  Pass DUMP_FILTER_FUNC
>> and a
>>    pointer to FILTER_BATON to svn_repos_dump_fs() if any filtering prefixes
>>    specified.
>>   (sub_main): Handle new options.
>>
>> * subversion/tests/cmdline/svnadmin_tests.py
>>   (dump_exclude,
>>    dump_exclude_copysource,
>>    dump_include,
>>    dump_not_include_copysource,
>>    dump_exclude_by_pattern,
>>    dump_include_by_pattern,
>>    dump_exclude_all_rev_changes,
>>    dump_invalid_filtering_option): New.
>>   (test_list): Add new tests to table.
>>
>> * subversion/tests/libsvn_repos/dump-load-test.c
>>   (test_dump_bad_props): Update caller.
>>
>> Patch by: sergey.raevskiy{_AT_}visualsvn.com
>> ]]
>
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] Add '--include' and '--exclude' options to 'svnadmin dump'

Julian Foad-6
In reply to this post by Sergey Raevskiy
On 12/10/16, Sergey Raevskiy wrote:
> I've attached a patch that adds '--include/--exclude' options to 'svnadmin
> dump'. These options work similarly to 'svndumpfilter include/exclude'
> but provide proper handling of 'copy from' paths.

Thank you! As I'm sure you're aware, this feature has been wanted almost
forever. For example, over ten years ago

Ben Collins-Sussman wrote "History simply cannot be filtered by
filtering a stream; the filtering process sometimes needs random-access
to all of the history. [...] 'svndumpfilter' needs to die, and the
functionality needs to be pushed into 'svnadmin dump' directly." [1]

However, I don't have the capacity to review it. I hope someone can.

- Julian


[1] On users@, 2005-08-17, "Re: svndumpfilter: Invalid copy source
path", https://svn.haxx.se/users/archive-2005-08/0744.shtml


> Consider the following example with svndumpfilter:
> [[
> $ svnadmin create /repo
> $ svn mkdir -m "" file:///repo/A
> $ svn copy -m "" file:///repo/A file:///repo/B
> $ svnadmin dump /repo | svndumpfilter include /B > dump
> ...
> Revision 0 committed as 0.
> Revision 1 committed as 1.
> svndumpfilter: E200003: Invalid copy source path '/A'
> ]]
>
> The 'svnadmin dump' with proposed include/exclude options can handle this use
> case properly:
> [[
> $ svnadmin dump /repo --include /B > dump
> * Dumped revision 0.
> * Dumped revision 1.
> * Dumped revision 2.
> ]]
>
> The proposed options are implemented using 'svn_repos_authz_func_t' and the
> not-included copy sources are hidden by the 'svn_repos' layer.

[...]

> Log message:
>
> [[
> Add '--include' and '--exclude' options to 'svnadmin dump'.
>
> * include/svn_repos.h
>   (svn_repos_dump_filter_func_t): New.
>   (svn_repos_dump_fs4): Update function signature and comment.
>   (svn_repos_dump_fs3): Update comment.
>
> * libsvn_repos/deprecated.c
>   (svn_repos_dump_fs3): Update caller.
>
> * libsvn_repos/dump.c
>   (write_revision_record): Call to svn_repos_fs_revision_proplist() (with
>    AUTHZ_FUNC) instead of svn_fs_revision_proplist2() to filter revision
>    properties as well as revisions. Update comment.
>   (dump_filter_baton_t,
>    dump_filter_authz_func): New.
>   (svn_repos_dump_fs4): Initialize and pass AUTHZ_FUNC and AUTHZ_BATON to
>    the repos layer API if FILTER_FUNC is specified by caller.
>
> * subversion/svnadmin/svnadmin.c
>   (svnadmin__cmdline_options_t): Add enum values for new options.
>   (options_table): Add new options.
>   (cmd_table): Add new options to 'dump' subcommand.
>   (svnadmin_opt_state): Add new fields to represent new options.
>   (ary_prefix_match): New. Copied from svndumpfilter.
>   (dump_filter_baton_t,
>    dump_filter_func): New.
>   (subcommand_dump): Initialize FILTER_BATON.  Pass DUMP_FILTER_FUNC and a
>    pointer to FILTER_BATON to svn_repos_dump_fs() if any filtering prefixes
>    specified.
>   (sub_main): Handle new options.
>
> * subversion/tests/cmdline/svnadmin_tests.py
>   (dump_exclude,
>    dump_exclude_copysource,
>    dump_include,
>    dump_not_include_copysource,
>    dump_exclude_by_pattern,
>    dump_include_by_pattern,
>    dump_exclude_all_rev_changes,
>    dump_invalid_filtering_option): New.
>   (test_list): Add new tests to table.
>
> * subversion/tests/libsvn_repos/dump-load-test.c
>   (test_dump_bad_props): Update caller.
>
> Patch by: sergey.raevskiy{_AT_}visualsvn.com
> ]]
>
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] Add '--include' and '--exclude' options to 'svnadmin dump'

maxin
In reply to this post by Sergey Raevskiy
I am wondering if there has been any progress on this. I just ran into the
same problem.



--
Sent from: http://subversion.1072662.n5.nabble.com/Subversion-Dev-f4725.html
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] Add '--include' and '--exclude' options to 'svnadmin dump'

Julian Foad-5
In reply to this post by Sergey Raevskiy
Sergey Raevskiy wrote:
> I've attached a patch that adds '--include/--exclude' options to 'svnadmin
> dump'. These options work similarly to 'svndumpfilter include/exclude'
> but provide proper handling of 'copy from' paths.

This is really important. I'm going to make some time to review it.

- Julian


> Consider the following example with svndumpfilter:
> [[
> $ svnadmin create /repo
> $ svn mkdir -m "" file:///repo/A
> $ svn copy -m "" file:///repo/A file:///repo/B
> $ svnadmin dump /repo | svndumpfilter include /B > dump
> ...
> Revision 0 committed as 0.
> Revision 1 committed as 1.
> svndumpfilter: E200003: Invalid copy source path '/A'
> ]]
>
> The 'svnadmin dump' with proposed include/exclude options can handle this use
> case properly:
> [[
> $ svnadmin dump /repo --include /B > dump
> * Dumped revision 0.
> * Dumped revision 1.
> * Dumped revision 2.
> ]]
>
> The proposed options are implemented using 'svn_repos_authz_func_t' and the
> not-included copy sources are hidden by the 'svn_repos' layer.
>
> Some usage examples for the proposed options:
> [[
> $ svnadmin dump repos --include /calc > calc-dumpfile
> ...
> $ svnadmin dump repos --include /calc --include /calendar > dumpfile
> ...
> $ svnadmin dump repos --exclude /secret > dumpfile
> ...
> $ svnadmin dump repos --include /calc --exclude /calendar > dumpfile
> svnadmin: E205000: Try 'svnadmin help' for more info
> svnadmin: E205000: '--exclude' and '--include' options cannot be used
> simultaneously
>
> $ svnadmin dump repos --include /cal* --pattern > dumpfile
> ...
> ]]
>
> Log message:
>
> [[
> Add '--include' and '--exclude' options to 'svnadmin dump'.
>
> * include/svn_repos.h
>    (svn_repos_dump_filter_func_t): New.
>    (svn_repos_dump_fs4): Update function signature and comment.
>    (svn_repos_dump_fs3): Update comment.
>
> * libsvn_repos/deprecated.c
>    (svn_repos_dump_fs3): Update caller.
>
> * libsvn_repos/dump.c
>    (write_revision_record): Call to svn_repos_fs_revision_proplist() (with
>     AUTHZ_FUNC) instead of svn_fs_revision_proplist2() to filter revision
>     properties as well as revisions. Update comment.
>    (dump_filter_baton_t,
>     dump_filter_authz_func): New.
>    (svn_repos_dump_fs4): Initialize and pass AUTHZ_FUNC and AUTHZ_BATON to
>     the repos layer API if FILTER_FUNC is specified by caller.
>
> * subversion/svnadmin/svnadmin.c
>    (svnadmin__cmdline_options_t): Add enum values for new options.
>    (options_table): Add new options.
>    (cmd_table): Add new options to 'dump' subcommand.
>    (svnadmin_opt_state): Add new fields to represent new options.
>    (ary_prefix_match): New. Copied from svndumpfilter.
>    (dump_filter_baton_t,
>     dump_filter_func): New.
>    (subcommand_dump): Initialize FILTER_BATON.  Pass DUMP_FILTER_FUNC and a
>     pointer to FILTER_BATON to svn_repos_dump_fs() if any filtering prefixes
>     specified.
>    (sub_main): Handle new options.
>
> * subversion/tests/cmdline/svnadmin_tests.py
>    (dump_exclude,
>     dump_exclude_copysource,
>     dump_include,
>     dump_not_include_copysource,
>     dump_exclude_by_pattern,
>     dump_include_by_pattern,
>     dump_exclude_all_rev_changes,
>     dump_invalid_filtering_option): New.
>    (test_list): Add new tests to table.
>
> * subversion/tests/libsvn_repos/dump-load-test.c
>    (test_dump_bad_props): Update caller.
>
> Patch by: sergey.raevskiy{_AT_}visualsvn.com
> ]]
>
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] Add '--include' and '--exclude' options to 'svnadmin dump'

Julian Foad-5
Julian Foad wrote:
> Sergey Raevskiy wrote:
>> I've attached a patch that adds '--include/--exclude' options to
>> 'svnadmin
>> dump'. These options work similarly to 'svndumpfilter include/exclude'
>> but provide proper handling of 'copy from' paths.
>
> This is really important. I'm going to make some time to review it.

The functionality, and even the coding style, all looks perfect.

Committed as revision 1811992.

Thank you, Sergey!

I only noticed one little error: in svnadmin_tests.py test_list one of
the pre-existing tests was duplicated as well as adding the new tests.

Good follow-ups would be:
   * update the help text to say: this functionality is equivalent to
using authz exclusions, and unlike svndumpfilter, using these options
does not break copies
   * update the help text for "--pattern" to match svndumpfilter (I
added an explanation of the glob syntax there in r1783741, which was
after you wrote your patch)
   * update tools/client-side/bash_completion
   * update the 1.10 release notes and changelog to mention this

I will do the first three of these soon.

- Julian



>> Consider the following example with svndumpfilter:
>> [[
>> $ svnadmin create /repo
>> $ svn mkdir -m "" file:///repo/A
>> $ svn copy -m "" file:///repo/A file:///repo/B
>> $ svnadmin dump /repo | svndumpfilter include /B > dump
>> ...
>> Revision 0 committed as 0.
>> Revision 1 committed as 1.
>> svndumpfilter: E200003: Invalid copy source path '/A'
>> ]]
>>
>> The 'svnadmin dump' with proposed include/exclude options can handle
>> this use
>> case properly:
>> [[
>> $ svnadmin dump /repo --include /B > dump
>> * Dumped revision 0.
>> * Dumped revision 1.
>> * Dumped revision 2.
>> ]]
>>
>> The proposed options are implemented using 'svn_repos_authz_func_t'
>> and the
>> not-included copy sources are hidden by the 'svn_repos' layer.
>>
>> Some usage examples for the proposed options:
>> [[
>> $ svnadmin dump repos --include /calc > calc-dumpfile
>> ...
>> $ svnadmin dump repos --include /calc --include /calendar > dumpfile
>> ...
>> $ svnadmin dump repos --exclude /secret > dumpfile
>> ...
>> $ svnadmin dump repos --include /calc --exclude /calendar > dumpfile
>> svnadmin: E205000: Try 'svnadmin help' for more info
>> svnadmin: E205000: '--exclude' and '--include' options cannot be used
>> simultaneously
>>
>> $ svnadmin dump repos --include /cal* --pattern > dumpfile
>> ...
>> ]]
>>
>> Log message:
>>
>> [[
>> Add '--include' and '--exclude' options to 'svnadmin dump'.
>>
>> * include/svn_repos.h
>>    (svn_repos_dump_filter_func_t): New.
>>    (svn_repos_dump_fs4): Update function signature and comment.
>>    (svn_repos_dump_fs3): Update comment.
>>
>> * libsvn_repos/deprecated.c
>>    (svn_repos_dump_fs3): Update caller.
>>
>> * libsvn_repos/dump.c
>>    (write_revision_record): Call to svn_repos_fs_revision_proplist()
>> (with
>>     AUTHZ_FUNC) instead of svn_fs_revision_proplist2() to filter revision
>>     properties as well as revisions. Update comment.
>>    (dump_filter_baton_t,
>>     dump_filter_authz_func): New.
>>    (svn_repos_dump_fs4): Initialize and pass AUTHZ_FUNC and
>> AUTHZ_BATON to
>>     the repos layer API if FILTER_FUNC is specified by caller.
>>
>> * subversion/svnadmin/svnadmin.c
>>    (svnadmin__cmdline_options_t): Add enum values for new options.
>>    (options_table): Add new options.
>>    (cmd_table): Add new options to 'dump' subcommand.
>>    (svnadmin_opt_state): Add new fields to represent new options.
>>    (ary_prefix_match): New. Copied from svndumpfilter.
>>    (dump_filter_baton_t,
>>     dump_filter_func): New.
>>    (subcommand_dump): Initialize FILTER_BATON.  Pass DUMP_FILTER_FUNC
>> and a
>>     pointer to FILTER_BATON to svn_repos_dump_fs() if any filtering
>> prefixes
>>     specified.
>>    (sub_main): Handle new options.
>>
>> * subversion/tests/cmdline/svnadmin_tests.py
>>    (dump_exclude,
>>     dump_exclude_copysource,
>>     dump_include,
>>     dump_not_include_copysource,
>>     dump_exclude_by_pattern,
>>     dump_include_by_pattern,
>>     dump_exclude_all_rev_changes,
>>     dump_invalid_filtering_option): New.
>>    (test_list): Add new tests to table.
>>
>> * subversion/tests/libsvn_repos/dump-load-test.c
>>    (test_dump_bad_props): Update caller.
>>
>> Patch by: sergey.raevskiy{_AT_}visualsvn.com
>> ]]
>>
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] Add '--include' and '--exclude' options to 'svnadmin dump'

Julian Foad-5
Julian Foad wrote:

> Good follow-ups would be:
>    * update the help text to say: this functionality is equivalent to
> using authz exclusions, and unlike svndumpfilter, using these options
> does not break copies
>    * update the help text for "--pattern" to match svndumpfilter (I
> added an explanation of the glob syntax there in r1783741, which was
> after you wrote your patch)
>    * update tools/client-side/bash_completion
>    * update the 1.10 release notes and changelog to mention this
>
> I will do the first three of these soon.

r1812050 does the first three and the CHANGES file.

- Julian
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] Add '--include' and '--exclude' options to 'svnadmin dump'

maxin
Hi Julian,

Thanks for the prompt response! I just compiled r1812050 and tested with my
own repo, however, somehow the svndumpfilter command cannot remove the
'empty' revisions from the filtered dump file....

Maxin



--
Sent from: http://subversion.1072662.n5.nabble.com/Subversion-Dev-f4725.html
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] Add '--include' and '--exclude' options to 'svnadmin dump'

Julian Foad-5
maxin wrote:
> Thanks for the prompt response! I just compiled r1812050 and tested with my
> own repo, however, somehow the svndumpfilter command cannot remove the
> 'empty' revisions from the filtered dump file....

Please report a small test case that demonstrates the problem, with the
exact commands. A script that starts with "svnadmin create repo" and
continues with "svn mkdir ..." and "svnadmin dump ..." and
"svndumpfilter ..." would be ideal.

Without knowing what commands you used, my first guess is you used
something like "svndumpfilter include / --drop-empty-revs".

> $ svndumpfilter include --help
> [...]
>   --drop-empty-revs        : Remove revisions emptied by filtering.
>   --drop-all-empty-revs    : Remove all empty revisions found in dumpstream
>                              except revision 0.

--drop-empty-revs only removes revisions that *svndumpfilter* makes
empty, not revisions that were already empty before svndumpfilter
received them. In that case maybe you want --drop-all-empty-revs
instead. Is that it?

- Julian


Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] Add '--include' and '--exclude' options to 'svnadmin dump'

Julian Foad-5
In reply to this post by Sergey Raevskiy
Should we now add all the other options from svndumpfilter?

>   --drop-empty-revs        : Remove revisions emptied by filtering.
>   --drop-all-empty-revs    : Remove all empty revisions found in dumpstream
>                              except revision 0.
>   --renumber-revs          : Renumber revisions left after filtering.
>   --skip-missing-merge-sources : Skip missing merge sources.
>   --targets ARG            : Read additional prefixes, one per line, from
>                              file ARG.
>   --preserve-revprops      : Don't filter revision properties.
>   --quiet                  : Do not display filtering statistics.

These were all designed to be useful with filtering. Why *wouldn't* we
want to include them directly in 'svnadmin dump --include/exclude'?

- Julian

Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] Add '--include' and '--exclude' options to 'svnadmin dump'

Julian Foad-5
In reply to this post by Sergey Raevskiy
We should update the JavaHL bindings for 'dump' accordingly.

Here is a patch that begins that process. It is not finished: as noted
in the log message, at least I haven't implemented CreateJ:FSPath and
the callback should (probably) be moved into its own file, and I have
presumably made some mistake already because even when I stub out the
unimplemented part all the tests are failing.

Would someone care to take a look at this and point out what I need to
fix or maybe even finish it, as I am running out of steam? (Brane?)

- Julian

svnadmin-dump-include-exclude-javahl-1.patch (9K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] Add '--include' and '--exclude' options to 'svnadmin dump'

Branko Čibej
On 13.10.2017 11:36, Julian Foad wrote:

> We should update the JavaHL bindings for 'dump' accordingly.
>
> Here is a patch that begins that process. It is not finished: as noted
> in the log message, at least I haven't implemented CreateJ:FSPath and
> the callback should (probably) be moved into its own file, and I have
> presumably made some mistake already because even when I stub out the
> unimplemented part all the tests are failing.
>
> Would someone care to take a look at this and point out what I need to
> fix or maybe even finish it, as I am running out of steam? (Brane?)

This is not going to happen any time soon. I have far too many
$dayjob-related things to do right now. Sorry ... JavaHL is an
interesting pile of spaghetti, and I can commiserate with anyone taking
the first steps towards understanding its magic.

-- Brane