Error E140001: Sum of subblock sizes larger than total block content length

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Error E140001: Sum of subblock sizes larger than total block content length

Ronald Taneza
Hi,

I got the error below while running "svnadmin load -M 0" to load a dump file created by "svnrdump dump".

  svnadmin: E140001: Sum of subblock sizes larger than total block content length

This error was reported when "svnadmin load" was loading a big file (around 2 GB) from a revision in the dump file.
I checked the dump file produced by svnrdump, and noticed that the Content-length for the 2GB file is a negative value! (see below)

I finally got it working by first creating a local dump file using "svnadmin dump", then loading this dump file using the same "svnadmin load" command used above.

More details below:
We are migrating our subversion server from Ubuntu Linux to Window Server 2012.
* Current server (Linux): svn, version 1.9.3 (r1718519)
* New server (Windows): svn, version 1.8.19 (r1800620)

We are actually using an older svn version in the new Windows server, because we are using CollabNet Subversion Edge, which still uses the svn 1.8 series.

To migrate data, we have a script running in the new server that does:
* svnrdump dump https://current-server/repo --incremental > dumpfile
* svnadmin load -M 0 local-path\repo < dumpfile

The "svnadmin load" command failed with the error message above, when loading a revision containing a big file (around 2 GB).
Checking the dump file produced by svnrdump (svn version 1.8.19), I noticed that the Content-length for the 2GB file is a negative value!
The expected Content-length value is (prop-content length + text-content-length) = 2238208388.
But the actual value is -2056758908, which is what you get when you try to interpret 2238208388 as a signed 32-bit integer (max 2147483647).

  SVN-fs-dump-format-version: 3

  Node-path: TheBigFile
  Node-kind: file
  Node-action: add
  Prop-delta: true
  Prop-content-length: 59
  Text-delta: true
  Text-content-length: 2238208329
  Text-content-md5: d2f79377abb0db99b37460c8156727fb
  Content-length: -2056758908

When the same revision is dumped using "svnadmin dump" in the Linux server (svn version 1.9.3), the content-length is a positive value, as expected:

  SVN-fs-dump-format-version: 2
  
  Node-path: TheBigFile
  Node-kind: file
  Node-action: add
  Text-content-md5: d2f79377abb0db99b37460c8156727fb
  Text-content-sha1: 5f5c06f734f394697f90cfab26dee5d820216ee5
  Prop-content-length: 59
  Text-content-length: 2237924210
  Content-length: 2237924269

Lastly, I tried "svnrdump dump" in the Linux server (svn version 1.9.3), and the content-length is also correct:

  SVN-fs-dump-format-version: 3
 
  Node-path: TheBigFile
  Node-kind: file
  Node-action: add
  Prop-delta: true
  Text-delta: true
  Text-content-md5: d2f79377abb0db99b37460c8156727fb
  Prop-content-length: 59
  Text-content-length: 2238208329
  Content-length: 2238208388

So this seems to be a problem with svnrdump version 1.8.x?

Of course, we would prefer to use the latest svn 1.9.x version, but we find the CollabNet Subversion Edge package quite useful (apache2 and svn in one installer package; provides web-based management console).

Regards,
Ronald
Reply | Threaded
Open this post in threaded view
|

Re: Error E140001: Sum of subblock sizes larger than total block content length

Daniel Shahaf-2
Ronald Taneza wrote on Tue, 21 Nov 2017 15:43 +0100:
> Checking the dump file produced by svnrdump (svn version 1.8.19), I noticed
> that the Content-length for the 2GB file is a negative value!
> The expected Content-length value is (prop-content length +
> text-content-length) = 2238208388.
> But the actual value is -2056758908, which is what you get when you try to
> interpret 2238208388 as a signed 32-bit integer (max 2147483647).

Subversion stores file sizes as svn_filesize_t which is an alias to
apr_int64_t which ought not to wrap until 2**63-1.  Are apr_int64_t and
APR_INT64_T_FMT correct on your systems?

It's possible that we're casting 64bit->32bit somewhere --- that'd be a
bug --- but let's first rule out the above (heeding Occam's razor).

Cheers,

Daniel
Reply | Threaded
Open this post in threaded view
|

Re: Error E140001: Sum of subblock sizes larger than total block content length

Julian Foad-5
In reply to this post by Ronald Taneza
> *From:* Ronald Taneza [mailto:[hidden email]]

> *Sent:* dinsdag 21 november 2017 15:44
> *To:* [hidden email]
>
> I got the error below while running "svnadmin load -M 0" to load a dump
> file created by "svnrdump dump".
>
>    svnadmin: E140001: Sum of subblock sizes larger than total block
> content length
>
> This error was reported when "svnadmin load" was loading a big file
> (around 2 GB) from a revision in the dump file.
[...]
> Checking the dump file produced by svnrdump (svn version 1.8.19), I
> noticed that the Content-length for the 2GB file is a negative value!
[...]

>    SVN-fs-dump-format-version: 3
>    Node-path: TheBigFile
>    Node-kind: file
>    Node-action: add
>    Prop-delta: true
>    Prop-content-length: 59
>    Text-delta: true
>    Text-content-length: 2238208329
>    Text-content-md5: d2f79377abb0db99b37460c8156727fb
>    Content-length: -2056758908
Thank you for finding this!

I can see this bug existed in svnrdump up to 1.8.19. (For 1.9 I refactored this to use common code shared with 'svnadmin dump' which does not have this bug.)

In 1.8.19, subversion/svnrdump/svnrdump.c:close_file() contains:

  if (fb->dump_text)
  ...
      SVN_ERR(svn_stream_printf(eb->stream, pool,
                                SVN_REPOS_DUMPFILE_TEXT_CONTENT_LENGTH
                                ": %lu\n",
                                (unsigned long)info->size));
  ...
  if (fb->dump_props)
    SVN_ERR(svn_stream_printf(eb->stream, pool,
                              SVN_REPOS_DUMPFILE_CONTENT_LENGTH
                              ": %ld\n\n",
                              (unsigned long)info->size +
                                propstring->len));
  else if (fb->dump_text)
    SVN_ERR(svn_stream_printf(eb->stream, pool,
                              SVN_REPOS_DUMPFILE_CONTENT_LENGTH
                              ": %ld\n\n",
                              (unsigned long)info->size));
  ...


info->size is apr_off_t ... probably 64 bits on most systems.
propstring->len is apr_size_t ... probably 64 bits on most systems.

It uses "%lu" for the text content, which thus work OK up to 4 GB, and "%ld" for the overall content length which thus only works up to 2 GB.

Earlier in this file, the property content length is printed correctly:

  buf = apr_psprintf(pool, SVN_REPOS_DUMPFILE_CONTENT_LENGTH
                     ": %" APR_SIZE_T_FMT "\n", len);

The attached patch should fix it; not yet tested.

- Julian

svnrdump-2GB-content-length-1.patch (2K) Download Attachment