when will file ./svn/tmp/svn-XXXXXX be created? and why?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

when will file ./svn/tmp/svn-XXXXXX be created? and why?

Keva-Slient

we use svn managing our config files. 

in client side environment, check out repos in client side, processes could open the config file and read or write the config file. some of them could keep file handler for a long time.

now we get some problems:  the config file will be moved to .svn/tmp directory and be deleted. 

for example: online_assist process:
online_assist will open some config files in /wns/etc/config/ directory. /wns/etc/config/ is a repo.
use losf to see which files online_assist are opening.

#lsof |grep online_assist |grep .svn/tmp
704 /wns/sbin/online_assist /wns/etc/config/.svn/tmp/svn-ynboPV (deleted)
704 /wns/sbin/online_assist /wns/etc/config/.svn/tmp/svn-sTknMU (deleted)
704 /wns/sbin/online_assist /wns/etc/config/.svn/tmp/svn-39HEsA (deleted)

online_assist should open config files which are not in .svn/tmp directory.
in usual, online_assist should open config files like : /wns/etc/config/db/ap_data.db ...

#pidof online_assist
704
#ls -l /proc/704/fd/* -l |grep config |sort -k 10
lrwx------ 1 root root 64 Dec 20 08:22 /proc/704/fd/19 -> /wns/etc/config/.svn/tmp/svn-39HEsA (deleted)
lrwx------ 1 root root 64 Dec 20 08:22 /proc/704/fd/18 -> /wns/etc/config/.svn/tmp/svn-sTknMU (deleted)
lrwx------ 1 root root 64 Dec 20 08:22 /proc/704/fd/15 -> /wns/etc/config/.svn/tmp/svn-ynboPV (deleted)
lrwx------ 1 root root 64 Dec 20 08:22 /proc/704/fd/23 -> /wns/etc/config/db/china_province_city.db
lrwx------ 1 root root 64 Dec 20 08:22 /proc/704/fd/20 -> /wns/etc/config/db/phonelocation.db
lrwx------ 1 root root 64 Dec 20 08:22 /proc/704/fd/16 -> /wns/etc/config/wac/data/ouiol.db
lrwx------ 1 root root 64 Dec 20 08:22 /proc/704/fd/21 -> /wns/etc/config/wac/data/ouiol.db
lrwx------ 1 root root 64 Dec 20 08:22 /proc/704/fd/17 -> /wns/etc/config/wac/data/sta_login_stat.db
lrwx------ 1 root root 64 Dec 20 08:22 /proc/704/fd/22 -> /wns/etc/config/wac/data/wti.db

some files opened by online_assist in  /wns/etc/config are correct, like /wns/etc/config/wac/data/ouiol.db. but some are moved to /wns/etc/config/.svn/tmp and deleted.

why is this ?

why would the file opend by process be moved to .svn/tmp
and
why would the file moved to .svn/tmp be deleted?


------------------
.........
Vanity...My favourite sin.
    --<<The Devil's Advocate>>
 
Reply | Threaded
Open this post in threaded view
|

RE: when will file ./svn/tmp/svn-XXXXXX be created? and why?

Bert Huijben-5

In many cases Subversion will create files there, then write its contents and when done move the file to the required location in your working copy. This makes sure that you will never see half written files during operations… and by that loose valuable information.

 

              Bert

 

From: Keva-Slient [mailto:[hidden email]]
Sent: donderdag 21 december 2017 08:29
To: users <[hidden email]>;
王运 <[hidden email]>; bluecat <[hidden email]>
Subject: when will file ./svn/tmp/svn-XXXXXX be created? and why?

 

 

we use svn managing our config files. 

 

in client side environment, check out repos in client side, processes could open the config file and read or write the config file. some of them could keep file handler for a long time.

 

now we get some problems:  the config file will be moved to .svn/tmp directory and be deleted. 

 

for example: online_assist process:

online_assist will open some config files in /wns/etc/config/ directory. /wns/etc/config/ is a repo.

use losf to see which files online_assist are opening.

 

#lsof |grep online_assist |grep .svn/tmp

704      /wns/sbin/online_assist /wns/etc/config/.svn/tmp/svn-ynboPV (deleted)

704      /wns/sbin/online_assist /wns/etc/config/.svn/tmp/svn-sTknMU (deleted)

704      /wns/sbin/online_assist /wns/etc/config/.svn/tmp/svn-39HEsA (deleted)

 

online_assist should open config files which are not in .svn/tmp directory.

in usual, online_assist should open config files like : /wns/etc/config/db/ap_data.db ...

 

#pidof online_assist

704

#ls -l /proc/704/fd/* -l |grep config |sort -k 10

lrwx------ 1 root root 64 Dec 20 08:22 /proc/704/fd/19 -> /wns/etc/config/.svn/tmp/svn-39HEsA (deleted)

lrwx------ 1 root root 64 Dec 20 08:22 /proc/704/fd/18 -> /wns/etc/config/.svn/tmp/svn-sTknMU (deleted)

lrwx------ 1 root root 64 Dec 20 08:22 /proc/704/fd/15 -> /wns/etc/config/.svn/tmp/svn-ynboPV (deleted)

lrwx------ 1 root root 64 Dec 20 08:22 /proc/704/fd/23 -> /wns/etc/config/db/china_province_city.db

lrwx------ 1 root root 64 Dec 20 08:22 /proc/704/fd/20 -> /wns/etc/config/db/phonelocation.db

lrwx------ 1 root root 64 Dec 20 08:22 /proc/704/fd/16 -> /wns/etc/config/wac/data/ouiol.db

lrwx------ 1 root root 64 Dec 20 08:22 /proc/704/fd/21 -> /wns/etc/config/wac/data/ouiol.db

lrwx------ 1 root root 64 Dec 20 08:22 /proc/704/fd/17 -> /wns/etc/config/wac/data/sta_login_stat.db

lrwx------ 1 root root 64 Dec 20 08:22 /proc/704/fd/22 -> /wns/etc/config/wac/data/wti.db

 

some files opened by online_assist in  /wns/etc/config are correct, like /wns/etc/config/wac/data/ouiol.db. but some are moved to /wns/etc/config/.svn/tmp and deleted.

 

why is this ?

 

why would the file opend by process be moved to .svn/tmp

and

why would the file moved to .svn/tmp be deleted?

 

 

------------------

.........

Vanity...My favourite sin.

    --<<The Devil's Advocate>>

 

Reply | Threaded
Open this post in threaded view
|

why my process open .svn/tmp/svn-XXXXXX file? and why are tmp files deleted? and why my process can't open the right file after tmp files deleted?

Keva-Slient
In reply to this post by Keva-Slient
we use svn managing our config files.

we lock config file when do committing or writing operation.
in client side, only commit operation executes. and svndump in server.

our process is always opening file like repo/someconfig/files . it's strang process opened tmp file like repo/.svn/tmp/svn-XXXXXX in some clients.  

and tmp file was deleted. our process report errors.

for example:
<8> Sundray-WAC / #lsof |grep online_assist | grep config |sort -k 3
704 /wns/sbin/online_assist /wns/etc/config/.svn/tmp/svn-39HEsA (deleted)
704 /wns/sbin/online_assist /wns/etc/config/.svn/tmp/svn-sTknMU (deleted)
704 /wns/sbin/online_assist /wns/etc/config/.svn/tmp/svn-ynboPV (deleted)


 

------------------
.........
Vanity...My favourite sin.
    --<<The Devil's Advocate>>
 

Reply | Threaded
Open this post in threaded view
|

Re: why my process open .svn/tmp/svn-XXXXXX file? and why are tmp files deleted? and why my process can't open the right file after tmp files deleted?

Johan Corveleyn-3
On Fri, Dec 22, 2017 at 3:33 AM, Keva-Slient <[hidden email]> wrote:

> we use svn managing our config files.
>
> we lock config file when do committing or writing operation.
> in client side, only commit operation executes. and svndump in server.
>
> our process is always opening file like repo/someconfig/files . it's strang
> process opened tmp file like repo/.svn/tmp/svn-XXXXXX in some clients.
>
> and tmp file was deleted. our process report errors.
>
> for example:
> <8> Sundray-WAC / #lsof |grep online_assist | grep config |sort -k 3
> 704 /wns/sbin/online_assist /wns/etc/config/.svn/tmp/svn-39HEsA (deleted)
> 704 /wns/sbin/online_assist /wns/etc/config/.svn/tmp/svn-sTknMU (deleted)
> 704 /wns/sbin/online_assist /wns/etc/config/.svn/tmp/svn-ynboPV (deleted)

Did you see Bert's reply to your earlier mail? You're asking almost
the same question.

    https://svn.haxx.se/users/archive-2017-12/0125.shtml

As Bert said: Subversion uses the .svn/tmp to create files, fill them
with content, and move them to the correct location in your working
copy (to avoid incomplete files appearing in your working copy). Apart
from that, svn also uses its tmp directory for other temporary files,
like creating temp files for diffing consecutive version of a file,
while running an 'svn blame' operation.

Basically, the .svn/tmp folder is an internal implementation detail of
Subversion. You shouldn't try to mess with it or write scripts around
it (the internals and behaviour of the .svn folder may change at any
time in newer svn releases).

Also: we have no idea here what "online-assist" is, and what it does.
That's not something from the Subversion project, as far as I know. On
this list we only know about SVN itself, so explain your question /
problem in terms of svn commands, what you see, and what you were
expecting instead.

--
Johan
Reply | Threaded
Open this post in threaded view
|

Re: why my process open .svn/tmp/svn-XXXXXX file? and why are tmpfiles deleted? and why my process can't open the right file after tmp files deleted?

Johan Corveleyn-3
[ Please keep the list [hidden email] in cc to keep the
discussion on the list. Also, we prefer plain-text mails instead of
html, and we don't like top-posting on this list, so put your reply at
the bottom, or inline to the thing you're replying to. More below. ]

On Fri, Dec 22, 2017 at 10:32 AM, Keva-Slient <[hidden email]> wrote:

> ------------------ 原始邮件 ------------------
> 发件人: "Johan Corveleyn";<[hidden email]>;
> 发送时间: 2017年12月22日(星期五) 下午5:07
> 收件人: "Keva-Slient"<[hidden email]>;
> 抄送: "users"<[hidden email]>;
> 主题: Re: why my process open .svn/tmp/svn-XXXXXX file? and why are tmpfiles
> deleted? and why my process can't open the right file after tmp files
> deleted?
>
> On Fri, Dec 22, 2017 at 3:33 AM, Keva-Slient <[hidden email]> wrote:
>> we use svn managing our config files.
>>
>> we lock config file when do committing or writing operation.
>> in client side, only commit operation executes. and svndump in server.
>>
>> our process is always opening file like repo/someconfig/files . it's
>> strang
>> process opened tmp file like repo/.svn/tmp/svn-XXXXXX in some clients.
>>
>> and tmp file was deleted. our process report errors.
>>
>> for example:
>> <8> Sundray-WAC / #lsof |grep online_assist | grep config |sort -k 3
>> 704 /wns/sbin/online_assist /wns/etc/config/.svn/tmp/svn-39HEsA (deleted)
>> 704 /wns/sbin/online_assist /wns/etc/config/.svn/tmp/svn-sTknMU (deleted)
>> 704 /wns/sbin/online_assist /wns/etc/config/.svn/tmp/svn-ynboPV (deleted)
>
> Did you see Bert's reply to your earlier mail? You're asking almost
> the same question.
>
>     https://svn.haxx.se/users/archive-2017-12/0125.shtml
>
> As Bert said: Subversion uses the .svn/tmp to create files, fill them
> with content, and move them to the correct location in your working
> copy (to avoid incomplete files appearing in your working copy). Apart
> from that, svn also uses its tmp directory for other temporary files,
> like creating temp files for diffing consecutive version of a file,
> while running an 'svn blame' operation.
>
> Basically, the .svn/tmp folder is an internal implementation detail of
> Subversion. You shouldn't try to mess with it or write scripts around
> it (the internals and behaviour of the .svn folder may change at any
> time in newer svn releases).
>
> Also: we have no idea here what "online-assist" is, and what it does.
> That's not something from the Subversion project, as far as I know. On
> this list we only know about SVN itself, so explain your question /
> problem in terms of svn commands, what you see, and what you were
> expecting instead.
>
> --
> Johan
>
> online-assist is our process name.
> we use svn to manage our config file and binary db files.
>
>
> program must keep the file open and will keep servicing all the time.
> we have to restart program if file in working copy moved to tmp and deleted.
> that's not what we want.
>
> we only use "commit" command in client side.
> is there any solution to avoid this tmp file problem?
> may i change the source code to satisfy our request if there is not any
> solution?
> and should i send this source code question to [hidden email] ?
>
> i'm now reading source code. i've found some code.
> static const struct work_item_dispatch dispatch_table[] = {
>   { OP_FILE_COMMIT, run_file_commit },
> }
> svn_io_file_move
> svn_io_open_unique_file3
> temp_file_create
>
> and one repository only has one client.
> only this client commit to server. it doesn't need updating. cause  there
> are not any others writing and committing to this repository.
> there will no conflicting and updating.
> so is it necessary to disable creating temporary file?

It's still not clear what online-assist is really doing. What svn
commands does it execute?

I don't understand why the deletes of the tmp files are a problem. Are
you trying to keep those tmp files open?? That would be bad. You
should NOT depend on those tmp files.

Or is the problem that you're keeping the WORKING files open (as
opposed to the tmp files corresponding to them), and you want to keep
them open even while you're committing them? And subversion replaces
those files from underneath you, when you're committing them?

A possible reason for SVN to replace your working file while when
you're committing it, is keyword-expansion [1]. If you have such
keywords in your source file, and you have the "svn:keywords" property
set on it, SVN really has to change your file when you're committing.

But maybe that "replace file" codepath is executed even when there are
no keywords present and the svn:keywords property is not set... That
might be fixable.

OTOH, there might be other reasons why SVN needs to touch the file
when you're committing it (I haven't looked at the code).

[ Meta: Apache Subversion is open source and has an Apache License 2.0
[2]. Basically, you're free to change the source code to your liking,
and create yourself a custom build / fork. If you want to propose a
change to the core project, you should start a discussion on
[hidden email]. ]

[1] http://svnbook.red-bean.com/nightly/en/svn.advanced.props.special.keywords.html
[2] https://www.apache.org/licenses/LICENSE-2.0

--
Johan