Subversion Server Replacement Query

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

Subversion Server Replacement Query

Saleh, Mai

Hi Subversion Support,

 

Please advise as we need to move subversion from old server with OS RH5U3 to RH7U3, all of our repositories are hosted on NFS share.

SVN version on old server is 1.6.x and new server has version 1.9.7.

 

Plan is to rsync old shared area hosting repositories to new shared area and copy configuration files to the same path on new server. So repositories and configuration files will be in exact paths. Then we will rename new server with original server name.

 

 

Initial test succeeded, but we need to know is there anything that we need to take care of, do we have to use dump and load as that will take lots of time, due to history and size of repositories.

 

Do users need to create a new working copies?

 

 

 

 

Thanks & BRs

 

Mai Saleh

IT Global Technologies & Infrastructure

Software Tools Engineer

 

Reply | Threaded
Open this post in threaded view
|

Re: Subversion Server Replacement Query

Branko Čibej
On 20.09.2017 11:06, Saleh, Mai wrote:
>
> Hi Subversion Support,
>
>  
>
> Please advise as we need to move subversion from old server with OS
> RH5U3 to RH7U3, all of our repositories are hosted on NFS share.
>

Don't Do That

> SVN version on old server is 1.6.x and new server has version 1.9.7.
>
>  
>
> Plan is to rsync old shared area hosting repositories to new shared
> area and copy configuration files to the same path on new server. So
> repositories and configuration files will be in exact paths. Then we
> will rename new server with original server name.
>
>  
>
>  
>
> Initial test succeeded, but we need to know is there anything that we
> need to take care of,
>

Yes. Do not put your repositories on an NFS share. That is not
guaranteed to work correctly (because file renames over NFS are not atomic).

> do we have to use dump and load as that will take lots of time, due to
> history and size of repositories.
>

No, the 1.9 server can use repositories created by version 1.6, but you
will not get all the new features and performance enhancements; see:

http://subversion.apache.org/docs/release-notes/1.7.html#new-feature-compatibility-table
http://subversion.apache.org/docs/release-notes/1.8.html#new-feature-compatibility-table
http://subversion.apache.org/docs/release-notes/1.9.html#new-feature-compatibility-table

>  Do users need to create a new working copies?
>

No.

-- Brane
Reply | Threaded
Open this post in threaded view
|

Re: Subversion Server Replacement Query

corneil.duplessis@gmail.com
In reply to this post by Saleh, Mai
In short:
Don't host on NFS or any remote filesystem. 
It is better to dump and import, you can do the dump with old or new version but will only benefit from all new features and stability with the repository in the newer version.



On 20 September 2017 at 11:06, Saleh, Mai <[hidden email]> wrote:

Hi Subversion Support,

 

Please advise as we need to move subversion from old server with OS RH5U3 to RH7U3, all of our repositories are hosted on NFS share.

SVN version on old server is 1.6.x and new server has version 1.9.7.

 

Plan is to rsync old shared area hosting repositories to new shared area and copy configuration files to the same path on new server. So repositories and configuration files will be in exact paths. Then we will rename new server with original server name.

 

 

Initial test succeeded, but we need to know is there anything that we need to take care of, do we have to use dump and load as that will take lots of time, due to history and size of repositories.

 

Do users need to create a new working copies?

 

 

 

 

Thanks & BRs

 

Mai Saleh

IT Global Technologies & Infrastructure

Software Tools Engineer

 


Reply | Threaded
Open this post in threaded view
|

RE: Subversion Server Replacement Query

Saleh, Mai

Hi Corneil,

 

Thanks a million for your reply.

It has been years since we use NFS, so no worries about that.

 

Can I know please what will be the risk to use rsync as it worked in our tests.

 

Do users need to create a new working copies after moving to new version?

 

 

 

 

Thanks & BRs

 

Mai Saleh

IT Global Technologies & Infrastructure

Software Tools Engineer

 

From: [hidden email] [mailto:[hidden email]]
Sent: Wednesday, September 20, 2017 12:23 PM
To: Saleh, Mai <[hidden email]>
Cc: [hidden email]
Subject: Re: Subversion Server Replacement Query

 

In short:

Don't host on NFS or any remote filesystem. 

It is better to dump and import, you can do the dump with old or new version but will only benefit from all new features and stability with the repository in the newer version.

 


 

Corneil du Plessis

about.me/corneil

 

 

On 20 September 2017 at 11:06, Saleh, Mai <[hidden email]> wrote:

Hi Subversion Support,

 

Please advise as we need to move subversion from old server with OS RH5U3 to RH7U3, all of our repositories are hosted on NFS share.

SVN version on old server is 1.6.x and new server has version 1.9.7.

 

Plan is to rsync old shared area hosting repositories to new shared area and copy configuration files to the same path on new server. So repositories and configuration files will be in exact paths. Then we will rename new server with original server name.

 

 

Initial test succeeded, but we need to know is there anything that we need to take care of, do we have to use dump and load as that will take lots of time, due to history and size of repositories.

 

Do users need to create a new working copies?

 

 

 

 

Thanks & BRs

 

Mai Saleh

IT Global Technologies & Infrastructure

Software Tools Engineer

 

 

Reply | Threaded
Open this post in threaded view
|

Re: Subversion Server Replacement Query

Ralph Seichter
In reply to this post by Saleh, Mai
> Hi Subversion Support,

Funny way to put it. ;-) This is a public mailing list of (mostly)
volunteers.

> Please advise as we need to move subversion from old server with OS
> RH5U3 to RH7U3, all of our repositories are hosted on NFS share.

Don't use network based file systems. Really, don't.

> Plan is to rsync old shared area hosting repositories to new shared
> area and copy configuration files to the same path on new server.
> [...]
> do we have to use dump and load as that will take lots of time, due
> to history and size of repositories.

Set up the new server as a mirror (with local data store, in new format,
not yet accessible to your users) of the existing server, use svnsync to
replicate the data, over hours or days if necessary. It does not matter
how long it takes to sync, since the existing server can continue to
operate, and you can synchronise in portions if required. From the docs:

  Also noteworthy is that svnsync performs careful bookkeeping that
  allows it to be safely interrupted and restarted without ruining the
  integrity of the mirrored data.

Once you are ready to switch, close user access to the old server, run
svnsync one final time, change DNS names, and have the new server accept
user connections.

> Do users need to create a new working copies?

That is a necessity whenever the repository UUID changes.

-Ralph
Reply | Threaded
Open this post in threaded view
|

Re: Subversion Server Replacement Query

corneil.duplessis@gmail.com
In reply to this post by Saleh, Mai
The user's working copies should be fine. In my experience older clients are fine with newer servers and newer clients are fine with older workspace formats.


On 20 September 2017 at 12:29, Saleh, Mai <[hidden email]> wrote:

Hi Corneil,

 

Thanks a million for your reply.

It has been years since we use NFS, so no worries about that.

 

Can I know please what will be the risk to use rsync as it worked in our tests.

 

Do users need to create a new working copies after moving to new version?

 

 

 

 

Thanks & BRs

 

Mai Saleh

IT Global Technologies & Infrastructure

Software Tools Engineer

 

From: [hidden email] [mailto:[hidden email]]
Sent: Wednesday, September 20, 2017 12:23 PM
To: Saleh, Mai <[hidden email]>
Cc: [hidden email]
Subject: Re: Subversion Server Replacement Query

 

In short:

Don't host on NFS or any remote filesystem. 

It is better to dump and import, you can do the dump with old or new version but will only benefit from all new features and stability with the repository in the newer version.

 


 

Corneil du Plessis

 

 

On 20 September 2017 at 11:06, Saleh, Mai <[hidden email]> wrote:

Hi Subversion Support,

 

Please advise as we need to move subversion from old server with OS RH5U3 to RH7U3, all of our repositories are hosted on NFS share.

SVN version on old server is 1.6.x and new server has version 1.9.7.

 

Plan is to rsync old shared area hosting repositories to new shared area and copy configuration files to the same path on new server. So repositories and configuration files will be in exact paths. Then we will rename new server with original server name.

 

 

Initial test succeeded, but we need to know is there anything that we need to take care of, do we have to use dump and load as that will take lots of time, due to history and size of repositories.

 

Do users need to create a new working copies?

 

 

 

 

Thanks & BRs

 

Mai Saleh

IT Global Technologies & Infrastructure

Software Tools Engineer

 

 


Reply | Threaded
Open this post in threaded view
|

RE: Subversion Server Replacement Query

Saleh, Mai

I will really appreciate, if you could help in “Can I know please what will be the risk to use rsync as it worked in our tests.

Why do we have to take dump and load, that will take week due to number and size, if you can clarify why not just take a copy of current  repositories to new shared area via rsync, will be great J

 

Thanks & BRs

 

Mai Saleh

IT Global Technologies & Infrastructure

Software Tools Engineer

 

From: [hidden email] [mailto:[hidden email]]
Sent: Wednesday, September 20, 2017 2:15 PM
To: Saleh, Mai <[hidden email]>
Cc: [hidden email]
Subject: Re: Subversion Server Replacement Query

 

The user's working copies should be fine. In my experience older clients are fine with newer servers and newer clients are fine with older workspace formats.


 

Corneil du Plessis

about.me/corneil

 

 

On 20 September 2017 at 12:29, Saleh, Mai <[hidden email]> wrote:

Hi Corneil,

 

Thanks a million for your reply.

It has been years since we use NFS, so no worries about that.

 

Can I know please what will be the risk to use rsync as it worked in our tests.

 

Do users need to create a new working copies after moving to new version?

 

 

 

 

Thanks & BRs

 

Mai Saleh

IT Global Technologies & Infrastructure

Software Tools Engineer

 

From: [hidden email] [mailto:[hidden email]]
Sent: Wednesday, September 20, 2017 12:23 PM
To: Saleh, Mai <[hidden email]>
Cc: [hidden email]
Subject: Re: Subversion Server Replacement Query

 

In short:

Don't host on NFS or any remote filesystem. 

It is better to dump and import, you can do the dump with old or new version but will only benefit from all new features and stability with the repository in the newer version.

 


 

Corneil du Plessis

 

 

On 20 September 2017 at 11:06, Saleh, Mai <[hidden email]> wrote:

Hi Subversion Support,

 

Please advise as we need to move subversion from old server with OS RH5U3 to RH7U3, all of our repositories are hosted on NFS share.

SVN version on old server is 1.6.x and new server has version 1.9.7.

 

Plan is to rsync old shared area hosting repositories to new shared area and copy configuration files to the same path on new server. So repositories and configuration files will be in exact paths. Then we will rename new server with original server name.

 

 

Initial test succeeded, but we need to know is there anything that we need to take care of, do we have to use dump and load as that will take lots of time, due to history and size of repositories.

 

Do users need to create a new working copies?

 

 

 

 

Thanks & BRs

 

Mai Saleh

IT Global Technologies & Infrastructure

Software Tools Engineer

 

 

 

Reply | Threaded
Open this post in threaded view
|

Re: Subversion Server Replacement Query

Nico Kadel-Garcia-2


On Wed, Sep 20, 2017 at 8:27 AM, Saleh, Mai <[hidden email]> wrote:

I will really appreciate, if you could help in “Can I know please what will be the risk to use rsync as it worked in our tests.

Why do we have to take dump and load, that will take week due to number and size, if you can clarify why not just take a copy of current  repositories to new shared area via rsync, will be great J


Storing the Subversion server repository on NFS is a separate issue. If your repository is oversized, it can be a real temptation to put it on a separate storage server, but it carries its own risks and can leave your repository corrupted, accidentally, as two write operations occur close to each other but are not immediately *visiable* to another process checking for Subversion records. This has nothing to do with your problem you asked about, it's just good advice for your Subversion servers.

If you can lock the current server and rsync over, then upgrade in place, *maybe* that will work. It's also likely to work better switching one repo, or small sets of repos at a time, if you have many, many repositories. Bu if it's one huge repo, it's also a chance to split it to more manageable pieces.

You say that dump and load "will take a week". That hints that it's too damn big and it's time to trim. Dump and load gives you an opportunity to clear obsolete and bulky content, and switch servers to a new, slimmer, possibly much more responsive service. Workers would definitely have to check out new working copies. But that's typically a good idea anyway, after a major back-end shift, because subtle changes in handling of things like EOL in comments can create adventures in the migration.

The "svnmirror" approach is commonly used for these problems. Start a mirror on your Subversion 1.9 server, allowing it to catch up with the production Subversion. That can run for the days or week, in the background, and should automatically give you a working Subversion 1.9 server. *Then* turn off access to the old server. especially write access. *Then* switch your DNS or web services to point to the new server. *Then* make damn sure that old server is disabled, preferably with the entire Subversion repo set aside. *Then* turn on write to the new server, ideally with a new URL to make clear that this is a *new* service, and to make people think before they just continue writing to the old service. There are enough feature changes between 1.6 and 1.9 to justify this, I think.

I'd recommend using a new URL for access to the old server, and leaving the old server disabled. *Maybe* leave it with a separate URL for read-only access, just in case the new server fails and you need access to an obsolete repo.


 

 

Thanks & BRs

 

Mai Saleh

IT Global Technologies & Infrastructure

Software Tools Engineer

 

From: [hidden email] [mailto:[hidden email]]
Sent: Wednesday, September 20, 2017 2:15 PM
To: Saleh, Mai <[hidden email]>
Cc: [hidden email]
Subject: Re: Subversion Server Replacement Query

 

The user's working copies should be fine. In my experience older clients are fine with newer servers and newer clients are fine with older workspace formats.


 

Corneil du Plessis

 

 

On 20 September 2017 at 12:29, Saleh, Mai <[hidden email]> wrote:

Hi Corneil,

 

Thanks a million for your reply.

It has been years since we use NFS, so no worries about that.

 

Can I know please what will be the risk to use rsync as it worked in our tests.

 

Do users need to create a new working copies after moving to new version?

 

 

 

 

Thanks & BRs

 

Mai Saleh

IT Global Technologies & Infrastructure

Software Tools Engineer

 

From: [hidden email] [mailto:[hidden email]]
Sent: Wednesday, September 20, 2017 12:23 PM
To: Saleh, Mai <[hidden email]>
Cc: [hidden email]
Subject: Re: Subversion Server Replacement Query

 

In short:

Don't host on NFS or any remote filesystem. 

It is better to dump and import, you can do the dump with old or new version but will only benefit from all new features and stability with the repository in the newer version.

 


 

Corneil du Plessis

 

 

On 20 September 2017 at 11:06, Saleh, Mai <[hidden email]> wrote:

Hi Subversion Support,

 

Please advise as we need to move subversion from old server with OS RH5U3 to RH7U3, all of our repositories are hosted on NFS share.

SVN version on old server is 1.6.x and new server has version 1.9.7.

 

Plan is to rsync old shared area hosting repositories to new shared area and copy configuration files to the same path on new server. So repositories and configuration files will be in exact paths. Then we will rename new server with original server name.

 

 

Initial test succeeded, but we need to know is there anything that we need to take care of, do we have to use dump and load as that will take lots of time, due to history and size of repositories.

 

Do users need to create a new working copies?

 

 

 

 

Thanks & BRs

 

Mai Saleh

IT Global Technologies & Infrastructure

Software Tools Engineer

 

 

 


Reply | Threaded
Open this post in threaded view
|

RE: Subversion Server Replacement Query

Saleh, Mai

Hi Nico,

 

Thanks a million for your detailed reply, I will be testing svnmirror solution then if it worked as expected, will use it.

 

 

Thanks & BRs

 

Mai Saleh

IT Global Technologies & Infrastructure

Software Tools Engineer

 

From: Nico Kadel-Garcia [mailto:[hidden email]]
Sent: Wednesday, September 20, 2017 3:08 PM
To: Saleh, Mai <[hidden email]>
Cc: [hidden email]; [hidden email]
Subject: Re: Subversion Server Replacement Query

 

 

 

On Wed, Sep 20, 2017 at 8:27 AM, Saleh, Mai <[hidden email]> wrote:

I will really appreciate, if you could help in “Can I know please what will be the risk to use rsync as it worked in our tests.”

Why do we have to take dump and load, that will take week due to number and size, if you can clarify why not just take a copy of current  repositories to new shared area via rsync, will be great J

 

Storing the Subversion server repository on NFS is a separate issue. If your repository is oversized, it can be a real temptation to put it on a separate storage server, but it carries its own risks and can leave your repository corrupted, accidentally, as two write operations occur close to each other but are not immediately *visiable* to another process checking for Subversion records. This has nothing to do with your problem you asked about, it's just good advice for your Subversion servers.

 

If you can lock the current server and rsync over, then upgrade in place, *maybe* that will work. It's also likely to work better switching one repo, or small sets of repos at a time, if you have many, many repositories. Bu if it's one huge repo, it's also a chance to split it to more manageable pieces.

 

You say that dump and load "will take a week". That hints that it's too damn big and it's time to trim. Dump and load gives you an opportunity to clear obsolete and bulky content, and switch servers to a new, slimmer, possibly much more responsive service. Workers would definitely have to check out new working copies. But that's typically a good idea anyway, after a major back-end shift, because subtle changes in handling of things like EOL in comments can create adventures in the migration.

 

The "svnmirror" approach is commonly used for these problems. Start a mirror on your Subversion 1.9 server, allowing it to catch up with the production Subversion. That can run for the days or week, in the background, and should automatically give you a working Subversion 1.9 server. *Then* turn off access to the old server. especially write access. *Then* switch your DNS or web services to point to the new server. *Then* make damn sure that old server is disabled, preferably with the entire Subversion repo set aside. *Then* turn on write to the new server, ideally with a new URL to make clear that this is a *new* service, and to make people think before they just continue writing to the old service. There are enough feature changes between 1.6 and 1.9 to justify this, I think.

 

I'd recommend using a new URL for access to the old server, and leaving the old server disabled. *Maybe* leave it with a separate URL for read-only access, just in case the new server fails and you need access to an obsolete repo.

 

 

 

 

Thanks & BRs

 

Mai Saleh

IT Global Technologies & Infrastructure

Software Tools Engineer

 

From: [hidden email] [mailto:[hidden email]]
Sent: Wednesday, September 20, 2017 2:15 PM
To: Saleh, Mai <[hidden email]>
Cc: [hidden email]
Subject: Re: Subversion Server Replacement Query

 

The user's working copies should be fine. In my experience older clients are fine with newer servers and newer clients are fine with older workspace formats.


 

Corneil du Plessis

 

 

On 20 September 2017 at 12:29, Saleh, Mai <[hidden email]> wrote:

Hi Corneil,

 

Thanks a million for your reply.

It has been years since we use NFS, so no worries about that.

 

Can I know please what will be the risk to use rsync as it worked in our tests.

 

Do users need to create a new working copies after moving to new version?

 

 

 

 

Thanks & BRs

 

Mai Saleh

IT Global Technologies & Infrastructure

Software Tools Engineer

 

From: [hidden email] [mailto:[hidden email]]
Sent: Wednesday, September 20, 2017 12:23 PM
To: Saleh, Mai <[hidden email]>
Cc: [hidden email]
Subject: Re: Subversion Server Replacement Query

 

In short:

Don't host on NFS or any remote filesystem. 

It is better to dump and import, you can do the dump with old or new version but will only benefit from all new features and stability with the repository in the newer version.

 


 

Corneil du Plessis

 

 

On 20 September 2017 at 11:06, Saleh, Mai <[hidden email]> wrote:

Hi Subversion Support,

 

Please advise as we need to move subversion from old server with OS RH5U3 to RH7U3, all of our repositories are hosted on NFS share.

SVN version on old server is 1.6.x and new server has version 1.9.7.

 

Plan is to rsync old shared area hosting repositories to new shared area and copy configuration files to the same path on new server. So repositories and configuration files will be in exact paths. Then we will rename new server with original server name.

 

 

Initial test succeeded, but we need to know is there anything that we need to take care of, do we have to use dump and load as that will take lots of time, due to history and size of repositories.

 

Do users need to create a new working copies?

 

 

 

 

Thanks & BRs

 

Mai Saleh

IT Global Technologies & Infrastructure

Software Tools Engineer

 

 

 

 

Reply | Threaded
Open this post in threaded view
|

RE: Subversion Server Replacement Query

Saleh, Mai
In reply to this post by Nico Kadel-Garcia-2

Hi Nico,

 

Is svnmirror means using svnsync command or it’s something different?

 

Thanks & BRs

 

Mai Saleh

IT Global Technologies & Infrastructure

Software Tools Engineer

 

From: Saleh, Mai
Sent: Wednesday, September 20, 2017 3:23 PM
To: 'Nico Kadel-Garcia' <[hidden email]>
Cc: [hidden email]; [hidden email]
Subject: RE: Subversion Server Replacement Query

 

Hi Nico,

 

Thanks a million for your detailed reply, I will be testing svnmirror solution then if it worked as expected, will use it.

 

 

Thanks & BRs

 

Mai Saleh

IT Global Technologies & Infrastructure

Software Tools Engineer

 

From: Nico Kadel-Garcia [[hidden email]]
Sent: Wednesday, September 20, 2017 3:08 PM
To: Saleh, Mai <[hidden email]>
Cc: [hidden email]; [hidden email]
Subject: Re: Subversion Server Replacement Query

 

 

 

On Wed, Sep 20, 2017 at 8:27 AM, Saleh, Mai <[hidden email]> wrote:

I will really appreciate, if you could help in “Can I know please what will be the risk to use rsync as it worked in our tests.”

Why do we have to take dump and load, that will take week due to number and size, if you can clarify why not just take a copy of current  repositories to new shared area via rsync, will be great J

 

Storing the Subversion server repository on NFS is a separate issue. If your repository is oversized, it can be a real temptation to put it on a separate storage server, but it carries its own risks and can leave your repository corrupted, accidentally, as two write operations occur close to each other but are not immediately *visiable* to another process checking for Subversion records. This has nothing to do with your problem you asked about, it's just good advice for your Subversion servers.

 

If you can lock the current server and rsync over, then upgrade in place, *maybe* that will work. It's also likely to work better switching one repo, or small sets of repos at a time, if you have many, many repositories. Bu if it's one huge repo, it's also a chance to split it to more manageable pieces.

 

You say that dump and load "will take a week". That hints that it's too damn big and it's time to trim. Dump and load gives you an opportunity to clear obsolete and bulky content, and switch servers to a new, slimmer, possibly much more responsive service. Workers would definitely have to check out new working copies. But that's typically a good idea anyway, after a major back-end shift, because subtle changes in handling of things like EOL in comments can create adventures in the migration.

 

The "svnmirror" approach is commonly used for these problems. Start a mirror on your Subversion 1.9 server, allowing it to catch up with the production Subversion. That can run for the days or week, in the background, and should automatically give you a working Subversion 1.9 server. *Then* turn off access to the old server. especially write access. *Then* switch your DNS or web services to point to the new server. *Then* make damn sure that old server is disabled, preferably with the entire Subversion repo set aside. *Then* turn on write to the new server, ideally with a new URL to make clear that this is a *new* service, and to make people think before they just continue writing to the old service. There are enough feature changes between 1.6 and 1.9 to justify this, I think.

 

I'd recommend using a new URL for access to the old server, and leaving the old server disabled. *Maybe* leave it with a separate URL for read-only access, just in case the new server fails and you need access to an obsolete repo.

 

 

 

 

Thanks & BRs

 

Mai Saleh

IT Global Technologies & Infrastructure

Software Tools Engineer

 

From: [hidden email] [mailto:[hidden email]]
Sent: Wednesday, September 20, 2017 2:15 PM
To: Saleh, Mai <[hidden email]>
Cc: [hidden email]
Subject: Re: Subversion Server Replacement Query

 

The user's working copies should be fine. In my experience older clients are fine with newer servers and newer clients are fine with older workspace formats.


 

Corneil du Plessis

 

 

On 20 September 2017 at 12:29, Saleh, Mai <[hidden email]> wrote:

Hi Corneil,

 

Thanks a million for your reply.

It has been years since we use NFS, so no worries about that.

 

Can I know please what will be the risk to use rsync as it worked in our tests.

 

Do users need to create a new working copies after moving to new version?

 

 

 

 

Thanks & BRs

 

Mai Saleh

IT Global Technologies & Infrastructure

Software Tools Engineer

 

From: [hidden email] [mailto:[hidden email]]
Sent: Wednesday, September 20, 2017 12:23 PM
To: Saleh, Mai <[hidden email]>
Cc: [hidden email]
Subject: Re: Subversion Server Replacement Query

 

In short:

Don't host on NFS or any remote filesystem. 

It is better to dump and import, you can do the dump with old or new version but will only benefit from all new features and stability with the repository in the newer version.

 


 

Corneil du Plessis

 

 

On 20 September 2017 at 11:06, Saleh, Mai <[hidden email]> wrote:

Hi Subversion Support,

 

Please advise as we need to move subversion from old server with OS RH5U3 to RH7U3, all of our repositories are hosted on NFS share.

SVN version on old server is 1.6.x and new server has version 1.9.7.

 

Plan is to rsync old shared area hosting repositories to new shared area and copy configuration files to the same path on new server. So repositories and configuration files will be in exact paths. Then we will rename new server with original server name.

 

 

Initial test succeeded, but we need to know is there anything that we need to take care of, do we have to use dump and load as that will take lots of time, due to history and size of repositories.

 

Do users need to create a new working copies?

 

 

 

 

Thanks & BRs

 

Mai Saleh

IT Global Technologies & Infrastructure

Software Tools Engineer

 

 

 

 

Reply | Threaded
Open this post in threaded view
|

Re: Subversion Server Replacement Query

corneil.duplessis@gmail.com
In reply to this post by Saleh, Mai
The subversion documentation doesn't advise copying the repositories to a new 'machine' or installation.
I have always used dump and load. I repository that was converted from CVS with more than 50k files, 2-3million lines of code and 15years old was 3GB in side and took 2 hours to dump and load.
I am not sure what you have that will take a week.


On 20 September 2017 at 14:27, Saleh, Mai <[hidden email]> wrote:

I will really appreciate, if you could help in “Can I know please what will be the risk to use rsync as it worked in our tests.

Why do we have to take dump and load, that will take week due to number and size, if you can clarify why not just take a copy of current  repositories to new shared area via rsync, will be great J

 

Thanks & BRs

 

Mai Saleh

IT Global Technologies & Infrastructure

Software Tools Engineer

 

From: [hidden email] [mailto:[hidden email]]
Sent: Wednesday, September 20, 2017 2:15 PM
To: Saleh, Mai <[hidden email]>
Cc: [hidden email]
Subject: Re: Subversion Server Replacement Query

 

The user's working copies should be fine. In my experience older clients are fine with newer servers and newer clients are fine with older workspace formats.


 

Corneil du Plessis

 

 

On 20 September 2017 at 12:29, Saleh, Mai <[hidden email]> wrote:

Hi Corneil,

 

Thanks a million for your reply.

It has been years since we use NFS, so no worries about that.

 

Can I know please what will be the risk to use rsync as it worked in our tests.

 

Do users need to create a new working copies after moving to new version?

 

 

 

 

Thanks & BRs

 

Mai Saleh

IT Global Technologies & Infrastructure

Software Tools Engineer

 

From: [hidden email] [mailto:[hidden email]]
Sent: Wednesday, September 20, 2017 12:23 PM
To: Saleh, Mai <[hidden email]>
Cc: [hidden email]
Subject: Re: Subversion Server Replacement Query

 

In short:

Don't host on NFS or any remote filesystem. 

It is better to dump and import, you can do the dump with old or new version but will only benefit from all new features and stability with the repository in the newer version.

 


 

Corneil du Plessis

 

 

On 20 September 2017 at 11:06, Saleh, Mai <[hidden email]> wrote:

Hi Subversion Support,

 

Please advise as we need to move subversion from old server with OS RH5U3 to RH7U3, all of our repositories are hosted on NFS share.

SVN version on old server is 1.6.x and new server has version 1.9.7.

 

Plan is to rsync old shared area hosting repositories to new shared area and copy configuration files to the same path on new server. So repositories and configuration files will be in exact paths. Then we will rename new server with original server name.

 

 

Initial test succeeded, but we need to know is there anything that we need to take care of, do we have to use dump and load as that will take lots of time, due to history and size of repositories.

 

Do users need to create a new working copies?

 

 

 

 

Thanks & BRs

 

Mai Saleh

IT Global Technologies & Infrastructure

Software Tools Engineer

 

 

 


Reply | Threaded
Open this post in threaded view
|

RE: Subversion Server Replacement Query

Saleh, Mai

Hi Corneil,

 

We have many repos some with 70GB size J.

 

Thanks & BRs

 

Mai Saleh

IT Global Technologies & Infrastructure

Software Tools Engineer

 

From: [hidden email] [mailto:[hidden email]]
Sent: Wednesday, September 20, 2017 3:47 PM
To: Saleh, Mai <[hidden email]>
Cc: [hidden email]
Subject: Re: Subversion Server Replacement Query

 

The subversion documentation doesn't advise copying the repositories to a new 'machine' or installation.

I have always used dump and load. I repository that was converted from CVS with more than 50k files, 2-3million lines of code and 15years old was 3GB in side and took 2 hours to dump and load.

I am not sure what you have that will take a week.


 

Corneil du Plessis

about.me/corneil

 

 

On 20 September 2017 at 14:27, Saleh, Mai <[hidden email]> wrote:

I will really appreciate, if you could help in “Can I know please what will be the risk to use rsync as it worked in our tests.”

Why do we have to take dump and load, that will take week due to number and size, if you can clarify why not just take a copy of current  repositories to new shared area via rsync, will be great J

 

Thanks & BRs

 

Mai Saleh

IT Global Technologies & Infrastructure

Software Tools Engineer

 

From: [hidden email] [mailto:[hidden email]]
Sent: Wednesday, September 20, 2017 2:15 PM
To: Saleh, Mai <[hidden email]>
Cc: [hidden email]
Subject: Re: Subversion Server Replacement Query

 

The user's working copies should be fine. In my experience older clients are fine with newer servers and newer clients are fine with older workspace formats.


 

Corneil du Plessis

 

 

On 20 September 2017 at 12:29, Saleh, Mai <[hidden email]> wrote:

Hi Corneil,

 

Thanks a million for your reply.

It has been years since we use NFS, so no worries about that.

 

Can I know please what will be the risk to use rsync as it worked in our tests.

 

Do users need to create a new working copies after moving to new version?

 

 

 

 

Thanks & BRs

 

Mai Saleh

IT Global Technologies & Infrastructure

Software Tools Engineer

 

From: [hidden email] [mailto:[hidden email]]
Sent: Wednesday, September 20, 2017 12:23 PM
To: Saleh, Mai <[hidden email]>
Cc: [hidden email]
Subject: Re: Subversion Server Replacement Query

 

In short:

Don't host on NFS or any remote filesystem. 

It is better to dump and import, you can do the dump with old or new version but will only benefit from all new features and stability with the repository in the newer version.

 


 

Corneil du Plessis

 

 

On 20 September 2017 at 11:06, Saleh, Mai <[hidden email]> wrote:

Hi Subversion Support,

 

Please advise as we need to move subversion from old server with OS RH5U3 to RH7U3, all of our repositories are hosted on NFS share.

SVN version on old server is 1.6.x and new server has version 1.9.7.

 

Plan is to rsync old shared area hosting repositories to new shared area and copy configuration files to the same path on new server. So repositories and configuration files will be in exact paths. Then we will rename new server with original server name.

 

 

Initial test succeeded, but we need to know is there anything that we need to take care of, do we have to use dump and load as that will take lots of time, due to history and size of repositories.

 

Do users need to create a new working copies?

 

 

 

 

Thanks & BRs

 

Mai Saleh

IT Global Technologies & Infrastructure

Software Tools Engineer

 

 

 

 

Reply | Threaded
Open this post in threaded view
|

RE: Subversion Server Replacement Query

Saleh, Mai
In reply to this post by Nico Kadel-Garcia-2

Also mirror, does not get hooks, we have to reconfigure it right.

 

Thanks & BRs

 

Mai Saleh

IT Global Technologies & Infrastructure

Software Tools Engineer

 

From: Saleh, Mai
Sent: Wednesday, September 20, 2017 3:45 PM
To: 'Nico Kadel-Garcia' <[hidden email]>
Cc: '[hidden email]' <[hidden email]>; '[hidden email]' <[hidden email]>
Subject: RE: Subversion Server Replacement Query

 

Hi Nico,

 

Is svnmirror means using svnsync command or it’s something different?

 

Thanks & BRs

 

Mai Saleh

IT Global Technologies & Infrastructure

Software Tools Engineer

 

From: Saleh, Mai
Sent: Wednesday, September 20, 2017 3:23 PM
To: 'Nico Kadel-Garcia' <[hidden email]>
Cc: [hidden email]; [hidden email]
Subject: RE: Subversion Server Replacement Query

 

Hi Nico,

 

Thanks a million for your detailed reply, I will be testing svnmirror solution then if it worked as expected, will use it.

 

 

Thanks & BRs

 

Mai Saleh

IT Global Technologies & Infrastructure

Software Tools Engineer

 

From: Nico Kadel-Garcia [[hidden email]]
Sent: Wednesday, September 20, 2017 3:08 PM
To: Saleh, Mai <[hidden email]>
Cc: [hidden email]; [hidden email]
Subject: Re: Subversion Server Replacement Query

 

 

 

On Wed, Sep 20, 2017 at 8:27 AM, Saleh, Mai <[hidden email]> wrote:

I will really appreciate, if you could help in “Can I know please what will be the risk to use rsync as it worked in our tests.”

Why do we have to take dump and load, that will take week due to number and size, if you can clarify why not just take a copy of current  repositories to new shared area via rsync, will be great J

 

Storing the Subversion server repository on NFS is a separate issue. If your repository is oversized, it can be a real temptation to put it on a separate storage server, but it carries its own risks and can leave your repository corrupted, accidentally, as two write operations occur close to each other but are not immediately *visiable* to another process checking for Subversion records. This has nothing to do with your problem you asked about, it's just good advice for your Subversion servers.

 

If you can lock the current server and rsync over, then upgrade in place, *maybe* that will work. It's also likely to work better switching one repo, or small sets of repos at a time, if you have many, many repositories. Bu if it's one huge repo, it's also a chance to split it to more manageable pieces.

 

You say that dump and load "will take a week". That hints that it's too damn big and it's time to trim. Dump and load gives you an opportunity to clear obsolete and bulky content, and switch servers to a new, slimmer, possibly much more responsive service. Workers would definitely have to check out new working copies. But that's typically a good idea anyway, after a major back-end shift, because subtle changes in handling of things like EOL in comments can create adventures in the migration.

 

The "svnmirror" approach is commonly used for these problems. Start a mirror on your Subversion 1.9 server, allowing it to catch up with the production Subversion. That can run for the days or week, in the background, and should automatically give you a working Subversion 1.9 server. *Then* turn off access to the old server. especially write access. *Then* switch your DNS or web services to point to the new server. *Then* make damn sure that old server is disabled, preferably with the entire Subversion repo set aside. *Then* turn on write to the new server, ideally with a new URL to make clear that this is a *new* service, and to make people think before they just continue writing to the old service. There are enough feature changes between 1.6 and 1.9 to justify this, I think.

 

I'd recommend using a new URL for access to the old server, and leaving the old server disabled. *Maybe* leave it with a separate URL for read-only access, just in case the new server fails and you need access to an obsolete repo.

 

 

 

 

Thanks & BRs

 

Mai Saleh

IT Global Technologies & Infrastructure

Software Tools Engineer

 

From: [hidden email] [mailto:[hidden email]]
Sent: Wednesday, September 20, 2017 2:15 PM
To: Saleh, Mai <[hidden email]>
Cc: [hidden email]
Subject: Re: Subversion Server Replacement Query

 

The user's working copies should be fine. In my experience older clients are fine with newer servers and newer clients are fine with older workspace formats.


 

Corneil du Plessis

 

 

On 20 September 2017 at 12:29, Saleh, Mai <[hidden email]> wrote:

Hi Corneil,

 

Thanks a million for your reply.

It has been years since we use NFS, so no worries about that.

 

Can I know please what will be the risk to use rsync as it worked in our tests.

 

Do users need to create a new working copies after moving to new version?

 

 

 

 

Thanks & BRs

 

Mai Saleh

IT Global Technologies & Infrastructure

Software Tools Engineer

 

From: [hidden email] [mailto:[hidden email]]
Sent: Wednesday, September 20, 2017 12:23 PM
To: Saleh, Mai <[hidden email]>
Cc: [hidden email]
Subject: Re: Subversion Server Replacement Query

 

In short:

Don't host on NFS or any remote filesystem. 

It is better to dump and import, you can do the dump with old or new version but will only benefit from all new features and stability with the repository in the newer version.

 


 

Corneil du Plessis

 

 

On 20 September 2017 at 11:06, Saleh, Mai <[hidden email]> wrote:

Hi Subversion Support,

 

Please advise as we need to move subversion from old server with OS RH5U3 to RH7U3, all of our repositories are hosted on NFS share.

SVN version on old server is 1.6.x and new server has version 1.9.7.

 

Plan is to rsync old shared area hosting repositories to new shared area and copy configuration files to the same path on new server. So repositories and configuration files will be in exact paths. Then we will rename new server with original server name.

 

 

Initial test succeeded, but we need to know is there anything that we need to take care of, do we have to use dump and load as that will take lots of time, due to history and size of repositories.

 

Do users need to create a new working copies?

 

 

 

 

Thanks & BRs

 

Mai Saleh

IT Global Technologies & Infrastructure

Software Tools Engineer

 

 

 

 

Reply | Threaded
Open this post in threaded view
|

RE: Subversion Server Replacement Query

Johan Corveleyn-3
[ If possible, please use bottom-posting or inline replying on this list, and preferably use plain-text. More below. ]


Dump+load or svnsync into the new FSFS format is not necessary, but like others have said, it might give you new features and the most optimal format available (this might give you space savings and very likely improved performance).

This FAQ may be useful:

As noted there:

- svnsync is better at normalizing some issues with properties (among others the EOL problem in comments that Nico hinted at).

- Apart from hooks and conf you also have to copy over locks (these are not svnsynced nor included in dump+load).

If you don't want to load into a new format back-end (because it takes too long and you're not interested in the improvements):

- I think you can safely use rsync if your repository is offline / read-only, or wrapped in the "svnadmin freeze" command (see http://svnbook.red-bean.com/vi/1.8/svn.ref.svnadmin.c.freeze.html).

- You might also want to look into "svnadmin hotcopy".


Also interesting to know: the next version of Subversion (1.10, currently in alpha, not yet ready for production use!) will most likely again include a new version of the FSFS format (with LZ4 compression (a bit larger, but usually a lot faster) so might be again a good candidate to plan a dump+load or svnsync, if you want it). And it will add a couple of improvements to "svnadmin load":

- New option --no-flush-to-disk to make it a lot faster (by not flushing after every revision).

- New option --normalize-props to perform the same EOL normalization on properties as svnsync does.

(Disclaimer: since 1.10 is not final yet, things may still change ... nothing is set in stone yet)

Final release of 1.10 is still some months away though.

-- 
Johan
Reply | Threaded
Open this post in threaded view
|

Re: Subversion Server Replacement Query

Nico Kadel-Garcia-2
In reply to this post by Saleh, Mai


On Wed, Sep 20, 2017 at 9:44 AM, Saleh, Mai <[hidden email]> wrote:

Hi Nico,

 

Is svnmirror means using svnsync command or it’s something different?

 

Thanks & BRs

 

Mai Saleh

IT Global Technologies & Infrastructure

Software Tools Engineer


That's exactly what I meant. If you're a complete weasel, like certain people who've done considerable consulting work and shall remain unnamed, you also use "svnadmin hotcopy" on the old repo and back that up before you touch *anything*. If your old server is old, or if you do any work on the old system with write privileges to the file system, it's too easy to make a mistake. And that hotcopy gets you a tarballable backup from which you can review the config files, especially svnserve.conf and any post-* scripts.