Discussion:
Mirror process timeout, process freeze
Alex
2013-10-03 20:41:50 UTC
Permalink
Hello,
I have been a recurring problem when using mirroring. Lftp program
sometimes hangs (I tested it on two different FTP servers).

Lftp version:
# lftp -v
LFTP | Version 4.4.9 | Copyright (c) 1996-2013 Alexander V. Lukyanov

ftpd #1 version:
pure-ftpd 1.0.36

ftpd #2 version:
vsftpd 2.3.2

The command to start:
/usr/bin/lftp -u "lftptest,viethoh3poFe" localhost -e "set ftp:ssl-allow
false ; set net:limit-rate 20971520:20971520 ; set ftp:list-options -a ;
debug ; mirror --delete --continue / /mnt/storage3/ftptest/mirror ; exit"

The process hangs for about two hours:
# ps axu | grep lftp
root 483 5.5 0.6 80068 27800 pts/18 S+ 11:59 6:17
/usr/bin/lftp -u lftptest,viethoh3poFe localhost -e set ftp:ssl-allow
false ; set net:limit-rate 20971520:20971520 ; set ftp:list-options -a ;
debug ; mirror --delete --continue / /mnt/storage3/ftptest/mirror ; exit


All files at ftp://lo.5252.ru/bugs/lftp-4.4.9/
---- CWD path to be sent is
`/www/storage3.files/components/com_content/views/article/tmpl'
---> CWD /www/storage3.files/components/com_content/views/article/tmpl
<--- 250 OK. Current directory is
/www/storage3.files/components/com_content/views/article/tmpl
---> TYPE A
<--- 200 TYPE is now ASCII
---> PASV
<--- 227 Entering Passive Mode (127,0,0,1,39,55)
---- Connecting data socket to (127.0.0.1) port 10039
---- Data connection established
---> LIST -a
<--- 150 Accepted data connection
<--- 226-Options: -a -l
<--- 226 8 matches total
---- Got EOF on data connection
---- Closing data socket
<--- 421 Timeout - try typing a little faster next time
---- Closing control socket
However, the process is still alive
write(2, "<--- 226 8 matches total\n", 25) = 25
brk(0x2f58000) = 0x2f58000
read(6, "drwxr-xr-x 2 3000 3000 "..., 65536) = 586
read(4, 0x170db30, 65536) = -1 EAGAIN (Resource
temporarily unavailable)
...
read(4, 0x170db30, 65536) = -1 EAGAIN (Resource
temporarily unavailable)
read(6, "", 65536) = 0
read(4, 0x170db30, 65536) = -1 EAGAIN (Resource
temporarily unavailable)
...
read(4, 0x170db30, 65536) = -1 EAGAIN (Resource
temporarily unavailable)
ioctl(2, TIOCGPGRP, [482]) = 0
getpgrp() = 482
write(2, "---- Got EOF on data connection\n", 32) = 32
ioctl(2, TIOCGPGRP, [482]) = 0
getpgrp() = 482
write(2, "---- Closing data socket\n", 25) = 25
close(6) = 0
read(4, 0x170db30, 65536) = -1 EAGAIN (Resource
temporarily unavailable)
...
read(4, 0x170db30, 65536) = -1 EAGAIN (Resource
temporarily unavailable)
stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1464, ...}) = 0
...
stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1464, ...}) = 0
brk(0x2f30000) = 0x2f30000
read(4, 0x170db30, 65536) = -1 EAGAIN (Resource
temporarily unavailable)
...
read(4, 0x170db30, 65536) = -1 EAGAIN (Resource
temporarily unavailable)
poll([{fd=4, events=POLLIN}], 1, 6) = 0 (Timeout)
ioctl(0, TIOCGPGRP, [482]) = 0
getpgrp() = 482
ioctl(1, TIOCGWINSZ, {ws_row=43, ws_col=127, ws_xpixel=0,
ws_ypixel=0}) = 0
write(1, "www/storage3.files/components/co"..., 61) = 61
write(1, "\r", 1) = 1
read(4, 0x170db30, 65536) = -1 EAGAIN (Resource
temporarily unavailable)
...
read(4, 0x170db30, 65536) = -1 EAGAIN (Resource
temporarily unavailable)
poll([{fd=4, events=POLLIN}], 1, 2317752) = 1 ([{fd=4, revents=POLLIN}])
read(4, "421 Timeout - try typing a littl"..., 65536) = 52
read(4, 0x170db30, 65536) = -1 EAGAIN (Resource
temporarily unavailable)
read(4, 0x170db30, 65536) = -1 EAGAIN (Resource
temporarily unavailable)
ioctl(2, TIOCGPGRP, [482]) = 0
getpgrp() = 482
ioctl(0, TIOCGPGRP, [482]) = 0
getpgrp() = 482
ioctl(1, TIOCGWINSZ, {ws_row=43, ws_col=127, ws_xpixel=0,
ws_ypixel=0}) = 0
write(1, " "..., 64) = 64
write(1, "\r", 1) = 1
write(2, "<--- 421 Timeout - try typing a "..., 56) = 56
ioctl(2, TIOCGPGRP, [482]) = 0
getpgrp() = 482
write(2, "---- Closing control socket\n", 28) = 28
close(4) = 0
poll(0, 0, 1) = 0 (Timeout)
ioctl(0, TIOCGPGRP, [482]) = 0
getpgrp() = 482
ioctl(1, TIOCGWINSZ, {ws_row=43, ws_col=127, ws_xpixel=0,
ws_ypixel=0}) = 0
write(1, "www/storage3.files/components/co"..., 61) = 61
write(1, "\r", 1) = 1
poll(0, 0, 299978) = 0 (Timeout)
poll(0, 0, 2017591
127.0.0.1 - lftptest [02/Oct/2013:12:16:04 +0400] "GET
/mnt/storage3/ftptest/content/www/storage3.files/components/com_content/
views/article/index.html" 200 4096
127.0.0.1 - lftptest [02/Oct/2013:12:16:04 +0400] "GET
/mnt/storage3/ftptest/content/www/storage3.files/components/com_content/
views/article/metadata.xml" 200 4096
127.0.0.1 - lftptest [02/Oct/2013:12:16:04 +0400] "GET
/mnt/storage3/ftptest/content/www/storage3.files/components/com_content/
views/article/view.html.php" 200 12288
--
Каногин Алекс
Alexander V. Lukyanov
2013-10-04 07:03:27 UTC
Permalink
Post by Alex
I have been a recurring problem when using mirroring. Lftp program
sometimes hangs (I tested it on two different FTP servers).
Can you please try this patch? Or get the snapshot from
http://lftp.yar.ru/ftp/devel/lftp-pre4.4.10-20131004.tar.gz

Also please send me the listing of the directory where lftp freezes.

--
Alexander.
Alex
2013-10-04 11:16:09 UTC
Permalink
Post by Alexander V. Lukyanov
Post by Alex
I have been a recurring problem when using mirroring. Lftp program
sometimes hangs (I tested it on two different FTP servers).
Can you please try this patch?
patching file src/NetAccess.cc
Hunk #2 FAILED at 575.
1 out of 2 hunks FAILED -- saving rejects to file src/NetAccess.cc.rej
The patch I manually changed since he has not been applied.
Post by Alexander V. Lukyanov
/bin/sh ../libtool --silent --tag=CXX --mode=compile
x86_64-pc-linux-gnu-g++ -DHAVE_CONFIG_H -I. -I../lib -I../lib
-I../trio -march=native -mtune=native -O2 -pipe -Wall -Wwrite-strings
-Woverloaded-virtual -fno-exceptions -fno-rtti -fno-implement-inlines
-c -o Resolver.lo Resolver.cc
NetAccess.cc:426:26: error: ‘class LsCache’ has no member named
‘UpdateFileSet’
NetAccess.cc:427:26: error: ‘class LsCache’ has no member named
‘UpdateFileSet’
make[1]: *** [NetAccess.lo] Error 1
make[1]: *** Waiting for unfinished jobs....
make[1]: Leaving directory
`/var/tmp/portage/net-ftp/lftp-4.4.9-r9/work/lftp-4.4.9/src'
make: *** [all-recursive] Error 1
You can send the correct patch for version 4.4.9?
Post by Alexander V. Lukyanov
Or get the snapshot from
http://lftp.yar.ru/ftp/devel/lftp-pre4.4.10-20131004.tar.gz
Snapshot compiled - mirroring test passed without hang-ups.
Many thanks, for efficiency.
Post by Alexander V. Lukyanov
Also please send me the listing of the directory where lftp freezes.
--
Alexander.
--
Каногин Алекс
Loading...