Discussion:
Why is "PROT P" issued twice?
Stephen Powell
2014-09-06 14:58:28 UTC
Permalink
I searched the mailing list archives, but I was unable to find anything
related to this. The lftp client is issuing the "PROT P" command to the
server twice, back to back. Why does it do this? Is there any way to
prevent it? In the overall grand scheme of things, this really isn't
very important, as it doesn't really hurt anything; but the client
should know that it just issued this command and therefore doesn't need
to issue it again.
--
.''`. Stephen Powell
: :' :
`. `'`
`-
Alexander Lukyanov
2014-09-08 07:53:13 UTC
Permalink
Please send the debug log (use command "debug" to turn it on).
Post by Stephen Powell
I searched the mailing list archives, but I was unable to find anything
related to this. The lftp client is issuing the "PROT P" command to the
server twice, back to back. Why does it do this? Is there any way to
prevent it? In the overall grand scheme of things, this really isn't
very important, as it doesn't really hurt anything; but the client
should know that it just issued this command and therefore doesn't need
to issue it again.
--
.''`. Stephen Powell
`. `'`
`-
_______________________________________________
lftp mailing list
http://univ.uniyar.ac.ru/mailman/listinfo/lftp
--
Alexander.
Stephen Powell
2014-09-08 23:30:53 UTC
Permalink
Post by Alexander Lukyanov
Please send the debug log (use command "debug" to turn it on).
For security reasons I have redacted or changed some things,
such as userids, passwords, domain names, IP addresses, etc.
This keeps me employed. This is lftp version 4.5.4, Readline 6.3,
GnuTLS 3.3.6, zlib 1.2.8.

Output of "set":

-----

set cmd:prompt "lftp \\S\\? \\u\\@\\h:\\w> "
set cmd:term-status/*rxvt* "\\e[11;0]\\e]2;\\T\\007\\e[11]"
set cmd:term-status/*screen* \\e_\\T\\e\\
set cmd:term-status/*xterm* "\\e[11;0]\\e]2;\\T\\007\\e[11]"
set color:dir-colors ""
set dns:order inet
set file:charset UTF-8
set ftp:auto-sync-mode "icrosoft FTP Service|MadGoat|MikroTik"
set ftp:ignore-pasv-address yes
set ftp:prefer-epsv yes
set ftp:ssl-allow-anonymous yes
set ftp:ssl-protect-data yes
set ftp:sync-mode/ftp.idsoftware.com on
set ftp:sync-mode/ftp.microsoft.com on
set ftp:sync-mode/sunsolve.sun.com on
set ssl:ca-file /etc/ssl/certs/ca-certificates.crt
set xfer:clobber true
set xfer:max-redirections 10
set xfer:verify-command /usr/share/lftp/verify-file

-----

Commands:

-----

open -u xxxxxxxx,yyyyyyyy -p nnnnn my.stupid.little.server.com
get -a -e myfile -o myfile
close
exit

-----

Debug output:

-----

---- Resolving host address...
---- 1 address found: nnn.nnn.nnn.nnn
---- Connecting to my.stupid.little.server.com (nnn.nnn.nnn.nnn) port nnnnn
<--- 220 (vsFTPd 2.3.5)
---> FEAT
<--- 211-Features:
<--- AUTH TLS
<--- EPRT
<--- EPSV
<--- MDTM
<--- PASV
<--- PBSZ
<--- PROT
<--- REST STREAM
<--- SIZE
<--- TVFS
<--- UTF8
<--- 211 End
---> AUTH TLS
<--- 234 Proceed with negotiation.
---> OPTS UTF8 ON
Certificate: C=XX,ST=xxxxxxxx,L=xxxxxxxx,O=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx,CN=my.stupid.little.server.com
Issued by: C=XX,ST=xxxxxxxx,L=xxxxxxxx,O=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx,CN=My Internal CA,EMAIL=***@my.email.domain.name.com
Trusted
<--- 200 Always in UTF8 mode.
---> USER xxxxxxxx
<--- 331 Please specify the password.
---> PASS yyyyyyyy
<--- 230 Login successful.
---> PWD
<--- 257 "/home/xxxxxxxx"
---> PBSZ 0
<--- 200 PBSZ set to 0.
---> PROT P
<--- 200 PROT now Private.
---> PROT P
<--- 200 PROT now Private.
---> SIZE myfile
<--- 213 313
---> MDTM myfile
<--- 213 20081202163039
---> EPSV
<--- 229 Entering Extended Passive Mode (|||mmmmm|).
---- Connecting data socket to (nnn.nnn.nnn.nnn) port mmmmm
---- Data connection established
---> RETR myfile
<--- 150 Opening ASCII mode data connection for myfile (313 bytes).
Certificate: C=XX,ST=xxxxxxxx,L=xxxxxxxx,O=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx,CN=my.stupid.little.server.com
Issued by: C=XX,ST=xxxxxxxx,L=xxxxxxxx,O=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx,CN=My Internal CA,EMAIL=***@my.email.domain.name.com
Trusted
---- Got EOF on data connection
---- Closing data socket
<--- 226 Transfer complete.
---> QUIT
<--- 221 Goodbye.
---- Closing control socket

-----
--
.''`. Stephen Powell
: :' :
`. `'`
`-
Alexander V. Lukyanov
2014-09-09 05:15:03 UTC
Permalink
Post by Stephen Powell
Post by Alexander Lukyanov
Please send the debug log (use command "debug" to turn it on).
For security reasons I have redacted or changed some things,
such as userids, passwords, domain names, IP addresses, etc.
This keeps me employed. This is lftp version 4.5.4, Readline 6.3,
GnuTLS 3.3.6, zlib 1.2.8.
Thanks! Here is a patch to fix the problem.

--
Alexander.
Stephen Powell
2014-09-10 01:37:16 UTC
Permalink
Post by Alexander V. Lukyanov
Thanks! Here is a patch to fix the problem.
Thanks. The supplied patch fixed the problem.

But now I'm having a different problem, which I don't think I was
having before. It appears to be trying to close the control socket
twice. Or maybe it's just printing out garbage from a message
buffer that it shouldn't be printing. (I've also upgraded to
lftp 4.5.5, by the way.) Here is what the tail end of the debug
output looks like now:

-----

---- Got EOF on data connection
---- Closing data socket
<--- 226 Transfer complete.
---> QUIT
---- Closing control socket
ng control socket
Goodbye.
---- Closing control socket
t

-----

Weird.
--
.''`. Stephen Powell
: :' :
`. `'`
`-
Alexander V. Lukyanov
2014-09-10 08:35:03 UTC
Permalink
Post by Stephen Powell
But now I'm having a different problem, which I don't think I was
having before. It appears to be trying to close the control socket
twice. Or maybe it's just printing out garbage from a message
buffer that it shouldn't be printing. (I've also upgraded to
lftp 4.5.5, by the way.) Here is what the tail end of the debug
-----
---- Got EOF on data connection
---- Closing data socket
<--- 226 Transfer complete.
---> QUIT
---- Closing control socket
ng control socket
Goodbye.
---- Closing control socket
t
-----
Looks like garbage after end of the string. I'll look into the problem.

--
Alexander.
Alexander V. Lukyanov
2014-09-10 09:11:23 UTC
Permalink
Post by Alexander V. Lukyanov
Post by Stephen Powell
But now I'm having a different problem, which I don't think I was
having before. It appears to be trying to close the control socket
twice. Or maybe it's just printing out garbage from a message
buffer that it shouldn't be printing. (I've also upgraded to
lftp 4.5.5, by the way.) Here is what the tail end of the debug
-----
---- Got EOF on data connection
---- Closing data socket
<--- 226 Transfer complete.
---> QUIT
---- Closing control socket
ng control socket
Goodbye.
---- Closing control socket
t
-----
Looks like garbage after end of the string. I'll look into the problem.
Please try this patch.

--
Alexander.
Stephen Powell
2014-09-11 23:19:53 UTC
Permalink
Post by Alexander V. Lukyanov
Please try this patch.
Well, the patch seems to help. I think I am now getting
legitimate output that I didn't always get before. But it did
not completely solve the problem. After some further
experimentation I was able to find a circumvention to the problem.
And this circumvention technique implies the true source of the
problem. But before I go any further, I need to supply some
additional information that I didn't mention before, because I
didn't think it was relevant, but it is now apparent that it is
relevant. I am invoking the command like this:

lftp -f getmyfile.lftp

In other words, I am supplying a script file for lftp to execute.
The contents of getmyfile.lftp look like this:

debug -o getmyfile.output 9
set xfer:clobber yes
open -u xxxxxxxx,yyyyyyyy -p nnnnn my.stupid.little.server.com
get -a -e myfile -o myfile
close

(The other changes to the default values of "set" variables are
being made in /etc/lftp.conf.) The significant point is that debug
output is being directed to a file by means of the "-o" option of
the debug command, instead of being written to the terminal. Now
here is the circumvention technique that I use to get around the
problem. If I manually erase (rm) the debug output file,
getmyfile.output, left over from the previous run of the command,
before invoking the command again, I get the expected output in
getmyfile.output. But if I don't erase the getmyfile.output file
before running the command, and this second run of the command
produces less debug output than the first run did, then there will
be left-over output from the previous run of the command at the end
of the file.

In other words, the first patch that you sent me, the one which
eliminated the duplicate "PROT P" command, did not *cause* the
second problem, rather, it *exposed* the second problem, which was
there all along, and is logically unrelated to the original problem.
I confirmed this in the following way. I erased the getmyfile.output
file, then ran the lftp command listed above. I then examined
getmyfile.output. Output was as expected. I then manually added
four lines to the end of this file with a text editor, each of which
said, "This is garbage.". I saved the changes and exited the editor.
Then I ran the above lftp command again, without erasing the
getmyfile.output file before running the lftp command. I then
examined getmyfile.output. Surely enough, those four lines at the
end, which said, "This is garbage.", were still there.

So the problem is that when debug output is directed to a file by
means of the "-o" option, any pre-existing file by that same name
needs to be erased first. This is not being done. In hindsight,
I should have opened a second thread for this; but I didn't realize
that the two problems were unrelated at the time. Sorry about that.
--
.''`. Stephen Powell
: :' :
`. `'`
`-
Alexander V. Lukyanov
2014-09-12 05:48:46 UTC
Permalink
Post by Stephen Powell
being made in /etc/lftp.conf.) The significant point is that debug
output is being directed to a file by means of the "-o" option of
the debug command, instead of being written to the terminal. Now
here is the circumvention technique that I use to get around the
problem. If I manually erase (rm) the debug output file,
getmyfile.output, left over from the previous run of the command,
before invoking the command again, I get the expected output in
getmyfile.output. But if I don't erase the getmyfile.output file
before running the command, and this second run of the command
produces less debug output than the first run did, then there will
be left-over output from the previous run of the command at the end
of the file.
Ok, I see now. This patch should fix the problem and the debug output
will be appended to the file as intended.

The problem was that the fcntl call cleared O_APPEND flag.

--
Alexander.
Stephen Powell
2014-09-12 22:52:21 UTC
Permalink
Post by Alexander V. Lukyanov
Ok, I see now. This patch should fix the problem and the debug output
will be appended to the file as intended.
The problem was that the fcntl call cleared O_APPEND flag.
The above patches solve the problem. Thanks so much!
--
.''`. Stephen Powell
: :' :
`. `'`
`-
Loading...