upgrading server

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

upgrading server

Andy So

We have an old subversion version 1.4.3 (r23084) running on Solaris.

We would like to upgrade to use new hardware on Linux based OS (CentOS 6.9), possibly version 1.8.x or 1.9.x

 

Our plan is to installed and configure the latest SVN on CentOS 6.9. Then go through dump and load of the repository as described in various online post and documentations.  The repository is quite large…guessing the size to be in the order of 20-40GB

 

Before we start undertaking such tasks

1.      Does anyone know if there are there any problem/gotcha in migrating the repository?

2.      Does anyone know how long it would take to export the repository of this size?  This will give us an estimate how long to schedule down time and cut off time.

 

Thanks for any insight.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: upgrading server

Mark Phippard-3
On Tue, Jul 25, 2017 at 2:00 PM, Andy So <[hidden email]> wrote:

We have an old subversion version 1.4.3 (r23084) running on Solaris.

We would like to upgrade to use new hardware on Linux based OS (CentOS 6.9), possibly version 1.8.x or 1.9.x

 

Our plan is to installed and configure the latest SVN on CentOS 6.9. Then go through dump and load of the repository as described in various online post and documentations.  The repository is quite large…guessing the size to be in the order of 20-40GB

 

Before we start undertaking such tasks

1.      Does anyone know if there are there any problem/gotcha in migrating the repository?

2.      Does anyone know how long it would take to export the repository of this size?  This will give us an estimate how long to schedule down time and cut off time.

 

Thanks for any insight.



Dump and load is a good idea because it lets the repository be rewritten using the newer code and repository format which will give you a smaller repository that will run a little faster.  That said you do not HAVE to dump/load.  You have other options:

1. Just move the repository folder to the new server.  Perhaps using tar and then moving the archive.

2. Instead of using dump/load, use svnsync.  This gives all the benefits of the dump/load but allows you to shrink your downtime to almost nothing.  Just svnsync the repository to the new server.  This will probably take a long time, but it does not matter since the original server can be running while it happens.  At the time of your choosing, do a final svnsync, and then shutdown the old server and use the new one.

3. Do option 1 now and then do a dump/load or svnsync at some future time that is more convenient for downtime.  It will probably run faster since it is on new and better hardware too.

There are some gotchas no matter what:

1.  Does the new server have a new hostname or do you intend to update DNS to point to new server?  If you are not doing the latter, then all of your existing working copies and scripts have to be modified for the new server.  Also any use of svn:externals property has to be modified.

2. With an old repository there is a good chance you will run into bugs in your data that cause svnsync or load to fail.  There are workaround for different failures but be prepared to run into them and account for finding the solutions.  SVN 1.8 and 1.9 have added various options to let you workaround some of the known bugs.  A common problem is having svn: properties that are not UTF-8 encoded or do not have LF line endings.  svnsync has this option to work around the encoding problem:

--source-prop-encoding ARG : convert translatable properties from encoding ARG
                             to UTF-8. If not specified, then properties are
                             presumed to be encoded in UTF-8.

And it automatically fixes the LF problems.

svnadmin load does not have any options to fix the problems, but you can add the   --bypass-prop-validation  option to ignore them and just carry the problems into your new repository.



--
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: upgrading server

David Chapman-10
In reply to this post by Andy So
On 7/25/2017 11:00 AM, Andy So wrote:

We have an old subversion version 1.4.3 (r23084) running on Solaris.

We would like to upgrade to use new hardware on Linux based OS (CentOS 6.9), possibly version 1.8.x or 1.9.x

 

Our plan is to installed and configure the latest SVN on CentOS 6.9. Then go through dump and load of the repository as described in various online post and documentations.  The repository is quite large…guessing the size to be in the order of 20-40GB

 

Before we start undertaking such tasks

1.      Does anyone know if there are there any problem/gotcha in migrating the repository?

2.      Does anyone know how long it would take to export the repository of this size?  This will give us an estimate how long to schedule down time and cut off time.

 

Thanks for any insight.


I recommend CentOS 7.x; CentOS 6.x is nearing the end of its support lifetime.  I also installed an SSD on my new machine - much faster than rotating media, and Subversion's "write once" philosophy (old revisions are essentially immutable) works well on an SSD.
-- 
    David Chapman      [hidden email]
    Chapman Consulting -- San Jose, CA
    Software Development Done Right.
    www.chapman-consulting-sj.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: upgrading server

Andrew Reedick-2
In reply to this post by Andy So
>> Does anyone know how long it would take to export the repository of this size?  This will give us an estimate how long to schedule down time and cut off time.

Svnsync is the easy option.

If you insist on doing a dump/load, then a) you can time a test run of a dump/load, and b) "svnadmin dump" allows you to specify revision ranges which means you can do incremental dumps.  You can dump/load the bulk of the repo now and then during the migration, run another dump/load to catch any new commits.
http://svnbook.red-bean.com/nightly/en/svn.reposadmin.maint.html#svn.reposadmin.maint.migrate

The only caveat I can think of is if someone changes revision properties (e.g. the commit message) between the time of the initial dump and the migration.  But you can track/prevent those with a hook script.  (And is another reason why svnsync is preferred.)


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: upgrading server

Ryan Schmidt-8
Don't forget that svnadmin dump/load and svnsync don't preserve hook scripts or other ancillary data that might be in your repository directory on the server, so if you have hook scripts, authorization rules or other customizations, carry those over to the new server manually. And of course any configuration files for httpd or svnserve.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: upgrading server

Nico Kadel-Garcia-2
In reply to this post by Andy So
On Tue, Jul 25, 2017 at 2:00 PM, Andy So <[hidden email]> wrote:
> We have an old subversion version 1.4.3 (r23084) running on Solaris.
>
> We would like to upgrade to use new hardware on Linux based OS (CentOS 6.9),
> possibly version 1.8.x or 1.9.x

If you're bumping Subversion release to 1.8.x or 1.9.x, I'd urge you
to go to CentOS 7, which at least starts with Subverison 1.7, and get
the latest RPM's from Wandisco.

> Our plan is to installed and configure the latest SVN on CentOS 6.9. Then go
> through dump and load of the repository as described in various online post
> and documentations.  The repository is quite large…guessing the size to be
> in the order of 20-40GB

See above. Installing the latest Subversion on CentOS 6 gets into some
fairly awkward library dependencies, especially for the "serf"
libraries which are notably out of date on CentOS 6.

> Before we start undertaking such tasks
>
> 1.      Does anyone know if there are there any problem/gotcha in migrating
> the repository?
>
> 2.      Does anyone know how long it would take to export the repository of
> this size?  This will give us an estimate how long to schedule down time and
> cut off time.

Definitely use svnsync, if possible, so you have no downtime and can
write-lock the original repository and do a DNS based switchover.
Rollback, if there are *any* mismatched writes to the new repository,
means starting over with all working copies, which can be quite
painful.

> Thanks for any insight.

You might consider less of an extreme version update. The more
versions you update between, I've noted more likelihood of
incompatible workflows such as the changes in the "svn:external"
formats.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: upgrading server

Johan Corveleyn-3
In reply to this post by Mark Phippard-3
On Tue, Jul 25, 2017 at 9:00 PM, Mark Phippard <[hidden email]> wrote:

> On Tue, Jul 25, 2017 at 2:00 PM, Andy So <[hidden email]> wrote:
>>
>> We have an old subversion version 1.4.3 (r23084) running on Solaris.
>>
>> We would like to upgrade to use new hardware on Linux based OS (CentOS
>> 6.9), possibly version 1.8.x or 1.9.x
>>
>>
>>
>> Our plan is to installed and configure the latest SVN on CentOS 6.9. Then
>> go through dump and load of the repository as described in various online
>> post and documentations.  The repository is quite large…guessing the size to
>> be in the order of 20-40GB
>>
>>
>>
>> Before we start undertaking such tasks
>>
>> 1.      Does anyone know if there are there any problem/gotcha in
>> migrating the repository?
>>
>> 2.      Does anyone know how long it would take to export the repository
>> of this size?  This will give us an estimate how long to schedule down time
>> and cut off time.
>>
>>
>>
>> Thanks for any insight.
>
>
>
> Dump and load is a good idea because it lets the repository be rewritten
> using the newer code and repository format which will give you a smaller
> repository that will run a little faster.  That said you do not HAVE to
> dump/load.  You have other options:
>
> 1. Just move the repository folder to the new server.  Perhaps using tar and
> then moving the archive.
>
> 2. Instead of using dump/load, use svnsync.  This gives all the benefits of
> the dump/load but allows you to shrink your downtime to almost nothing.
> Just svnsync the repository to the new server.  This will probably take a
> long time, but it does not matter since the original server can be running
> while it happens.  At the time of your choosing, do a final svnsync, and
> then shutdown the old server and use the new one.
>
> 3. Do option 1 now and then do a dump/load or svnsync at some future time
> that is more convenient for downtime.  It will probably run faster since it
> is on new and better hardware too.
>
> There are some gotchas no matter what:
>
> 1.  Does the new server have a new hostname or do you intend to update DNS
> to point to new server?  If you are not doing the latter, then all of your
> existing working copies and scripts have to be modified for the new server.
> Also any use of svn:externals property has to be modified.
>
> 2. With an old repository there is a good chance you will run into bugs in
> your data that cause svnsync or load to fail.  There are workaround for
> different failures but be prepared to run into them and account for finding
> the solutions.  SVN 1.8 and 1.9 have added various options to let you
> workaround some of the known bugs.  A common problem is having svn:
> properties that are not UTF-8 encoded or do not have LF line endings.
> svnsync has this option to work around the encoding problem:
>
> --source-prop-encoding ARG : convert translatable properties from encoding
> ARG
>                              to UTF-8. If not specified, then properties are
>                              presumed to be encoded in UTF-8.
>
> And it automatically fixes the LF problems.
>
> svnadmin load does not have any options to fix the problems, but you can add
> the   --bypass-prop-validation  option to ignore them and just carry the
> problems into your new repository.

Indeed, dump/load is a bit more limited than svnsync regarding
normalization. Also, be careful: both dump/load and svnsync do not
migrate your hook scripts, config files and locks (server side locks
of the svn:needs-lock type).

I've just extended the FAQ entry about dump/load with some of the
known caveats, and explained a procedure to do dump/load with minimal
downtime. See:

http://subversion.apache.org/faq.html#dumpload

--
Johan
Loading...