LFTP retry behavior when a file is not present

[Miller-Jacobson, Ben]

One of my coworkers recently encountered a problem in one of our scripts where the script would apparently hang after first being moved to a new machine in which the none of the old files it was supposed to move were yet present. We tracked it down to LFTP apparently not finding the file it is looking for and retrying indefinitely. Reviewing the man page, this looks like the expected behavior. However, it was not what we were expecting and in our case and a number of other plausible use cases not what is desired. It occurs to me that an option to disable retries or perhaps set a maximum number of retries might be a nice feature to add, if it doesn't already exist. We've got a fix for our issue, but this might be useful for other cases where reporting a failure is better than retrying indefinitely, especially for use in scripts where there isn't going to be someone around to kill the process.

Thanks,

Ben Miller-Jacobson

[Alexander V. Lukyanov]

Post by Miller-Jacobson, Ben
One of my coworkers recently encountered a problem in one of our scripts
where the script would apparently hang after first being moved to a new
machine in which the none of the old files it was supposed to move were yet
present. We tracked it down to LFTP apparently not finding the file it is
looking for and retrying indefinitely. Reviewing the man page, this looks
like the expected behavior. However, it was not what we were expecting and
in our case and a number of other plausible use cases not what is desired.
It occurs to me that an option to disable retries or perhaps set a maximum
number of retries might be a nice feature to add, if it doesn't already
exist. We've got a fix for our issue, but this might be useful for other
cases where reporting a failure is better than retrying indefinitely,
especially for use in scripts where there isn't going to be someone around
to kill the process.

Please turn on debug in lftp (command debug) and see the protocol messages.
FYI, 4xx codes in FTP protocol are transient, so lftp retries on them.
To limit the number of retries change the setting net:max-retries.