[ftputil] Using ftptuil with encryped connection.

Roger Demetrescu roger.demetrescu at gmail.com
Fri Apr 11 15:25:49 CEST 2014


Hi Stefan !


On Thu, Apr 10, 2014 at 5:54 PM, Stefan Schwarzer <sschwarzer at sschwarzer.net
> wrote:

> Hi Roger,
>
> On 2014-04-08 22:02, Roger Demetrescu wrote:
> > On Tue, Apr 8, 2014 at 3:56 PM, Stefan Schwarzer
> > <sschwarzer at sschwarzer.net>wrote:
> >> On 2014-04-08 01:07, Roger Demetrescu wrote:
> >> I assume you're basing your session class for `FTPHost`'s
> >> `session_factory` argument on `ftplib.FTP_TLS`. Is this
> >> correct?
> >
> > Nope.. I was basing it on ftpslib.FTP_TLS, because python
> > 2.6 doesn't have FTP_TLS.
>
> Ok.
>
> >> One thing that might help with debugging is connecting to
> >> the server with `ftplib.FTP_TLS` instead of
> >> `ftputil.FTPHost`. This removes one component from the mix
> >> and give you a more direct feedback.
> >>
> >> I recommend that you instantiate `ftplib.FTP_TLS`, activate
> >> debugging with `ftp_instance.set_debuglevel(2)` and then try
> >> to open the connection with `ftp_instance.login()`. The
> >> example in the documentation at
> >> https://docs.python.org/2/library/ftplib.html#ftplib.FTP_TLS
> >> also mentions that you need to call `ftp_instance.prot_p()`
> >> to get a secure connection.
> >>
> >> _If_ `ftplib.FTP_TLS` works on its own, please make another
> >> attempt with ftputil, but send the full tracebacks that
> >> ftputil prints. You might also want to switch on debugging
> >> on the wrapped `_session` object before actually logging in
> >> from your derived `FTP_TLS` class.
> >>
> >> Please let me know what results you get.
> >
> > Well, since my development environment is using python
> > 2.7, I have already did it.
> >
> > Here is the code when I use ftpslib.FTP  (from M2Crypto)
> >
> > ----------------------------8<-------------------------------
> > import ftputil
> > from M2Crypto import ftpslib
> >
> > from config import host, username, password
> >
> > class SSLFTPSession(ftpslib.FTP_TLS):
> >
> >     def __init__(self, host, userid, password):
> >         ftpslib.FTP_TLS.__init__(self)
> >         self.set_debuglevel(2)
> >         self.connect(host, 21)
> >         self.auth_tls()
> >         self.login(userid, password)
> >         self.prot_p()
> >
> > ----------------------------8<-------------------------------
> >
> > and here is the code adapted to use ftplib.FTP_TLS from
> > python 2.7 itself:
> >
> > ----------------------------8<-------------------------------
> > import ftplib
> > import ftputil
> >
> > from config import host, username, password
> >
> > class SSLFTPSession(ftplib.FTP_TLS):
> >
> >     def __init__(self, host, userid, password):
> >         ftplib.FTP_TLS.__init__(self)
> >         self.set_debuglevel(2)
> >         self.connect(host, 21)
> >         self.login(userid, password)
> >         self.prot_p()
> >
> > ----------------------------8<-------------------------------
> >
> > Now, the funny thing: both code give me the IDENTICAL log
> > then I create the instance of ftputil.FTPHost:
> >
> >  f = ftputil.FTPHost(host, username, password,
> > session_factory=SSLFTPSession)
> >
> > ----------------------------8<-------------------------------
> > *get* '220-IBM Portal\r\n'
> > *get* '220 \r\n'
> > *resp* '220-IBM Portal\n220 '
> > *cmd* 'AUTH TLS'
> > *put* 'AUTH TLS\r\n'
> > *get* '234 Proceed with negotiation.\r\n'
> > *resp* '234 Proceed with negotiation.'
> > *cmd* 'USER myusername'
> > *put* 'USER myusername\r\n'
> > *get* '331 Please specify the password.\r\n'
> > *resp* '331 Please specify the password.'
> > *cmd* 'PASS ********'
> > *put* 'PASS ********\r\n'
> > *get* '230 Login successful.\r\n'
> > *resp* '230 Login successful.'
> > *cmd* 'PBSZ 0'
> > *put* 'PBSZ 0\r\n'
> > *get* '200 PBSZ set to 0.\r\n'
> > *resp* '200 PBSZ set to 0.'
> > *cmd* 'PROT P'
> > *put* 'PROT P\r\n'
> > *get* '200 PROT now Private.\r\n'
> > *resp* '200 PROT now Private.'
> > *cmd* 'PWD'
> > *put* 'PWD\r\n'
> > *get* '257 "/"\r\n'
> > *resp* '257 "/"'
> >
> > ----------------------------8<-------------------------------
> >
> > I used a diff program to make sure all this log was identical between
> > ftpslib.FTP_TLS and ftplib.FTP_TLS...
> >
> > Now, then I try to do a simple  f.listdir('.'), here comes
> > the difference:
> >
> > # --------------- using ftpslib.FTP_TLS
> > *cmd* u'CWD /'
> > *put* u'CWD /\r\n'
> > *get* '500 Unknown command.\r\n'
> > *resp* '500 Unknown command.'
> >
> ---------------------------------------------------------------------------
> > InaccessibleLoginDirError                 Traceback (most recent call
> last)
> > <ipython-input-1-1646fe8e3e06> in <module>()
> > ----> 1 f.listdir('.')
> >
> > # --------------- using ftplib.FTP_TLS
> > *cmd* u'CWD /'
> > *put* u'CWD /\r\n'
> > *get* '250 Directory successfully changed.\r\n'
> > *resp* '250 Directory successfully changed.'
> > *cmd* u'CWD /'
> > *put* u'CWD /\r\n'
> > *get* '250 Directory successfully changed.\r\n'
> > *resp* '250 Directory successfully changed.'
> > *cmd* 'TYPE A'
> > *put* 'TYPE A\r\n'
> > *get* '200 Switching to ASCII mode.\r\n'
> > *resp* '200 Switching to ASCII mode.'
> > *cmd* 'PASV'
> > *put* 'PASV\r\n'
> > *get* '227 Entering Passive Mode (***,***,***,***,250,31).\r\n'
> > *resp* '227 Entering Passive Mode (***,***,***,***,250,31).'
> > *cmd* u'LIST -a'
> > *put* u'LIST -a\r\n'
> > *get* '150 Here comes the directory listing.\r\n'
> > *resp* '150 Here comes the directory listing.'
> > *retr* 'drwxrwxrwx    9 0        0            4096 Apr 02 19:44 .\r\n'
> > *retr* 'drwxrwxrwx    9 0        0            4096 Apr 02 19:44 ..\r\n'
> > *retr* 'drwxrwxrwx    3 0        0            4096 Nov 27 18:36
> backup\r\n'
> > *retr* 'drwxrwxrwx    2 0        0            4096 Nov 27 18:37 css\r\n'
> > <<< LOTS OF DIRECTORIES AND FILES >>>
> > *retr* ''
> > *get* '226 Directory send OK.\r\n'
> > *resp* '226 Directory send OK.'
> > *cmd* u'CWD /'
> > *put* u'CWD /\r\n'
> > *get* '250 Directory successfully changed.\r\n'
> > *resp* '250 Directory successfully changed.'
> >
> > Funny (and scary), isn't it ?   :)
>
> Let's say, it's quite interesting. ;-)
>
> I mean, apart from the concrete commands, there are many
> things that might be different under the hood. Especially,
> how the data connection is set up.
>
> Since ftputil "converts" ftplib (and M2Crypto) exceptions to
> its own exceptions, some information might have gone lost.
> I highly recommend you try to use M2Crypto directly for a
> test, i. e. _not_ via ftputil. The exception you get there
> might be more helpful than ftputil's.
>


I did it.. and it worked (but can't live without ftputil   :)  )


---------------------8<--------------------
import ftputil

from M2Crypto import ftpslib

from config import host, username, password

f = ftpslib.FTP_TLS()
f.set_debuglevel(2)
f.connect(host, 21)
f.auth_tls()
f.login(username, password)
f.prot_p()
f.retrlines('LIST')
---------------------8<--------------------


The resulting log:

---------------------8<--------------------
*get* '220-IBM Portal\r\n'
*get* '220 \r\n'
*resp* '220-IBM Portal\n220 '
*cmd* 'AUTH TLS'
*put* 'AUTH TLS\r\n'
*get* '234 Proceed with negotiation.\r\n'
*resp* '234 Proceed with negotiation.'
*cmd* 'USER myusername'
*put* 'USER myusername\r\n'
*get* '331 Please specify the password.\r\n'
*resp* '331 Please specify the password.'
*cmd* 'PASS ********'
*put* 'PASS ********\r\n'
*get* '230 Login successful.\r\n'
*resp* '230 Login successful.'
*cmd* 'PBSZ 0'
*put* 'PBSZ 0\r\n'
*get* '200 PBSZ set to 0.\r\n'
*resp* '200 PBSZ set to 0.'
*cmd* 'PROT P'
*put* 'PROT P\r\n'
*get* '200 PROT now Private.\r\n'
*resp* '200 PROT now Private.'
*cmd* 'TYPE A'
*put* 'TYPE A\r\n'
*get* '200 Switching to ASCII mode.\r\n'
*resp* '200 Switching to ASCII mode.'
*cmd* 'PASV'
*put* 'PASV\r\n'
*get* '227 Entering Passive Mode (xxx,xxx,xxx,xxx,250,175).\r\n'
*resp* '227 Entering Passive Mode (xxx,xxx,xxx,xxx,250,175).'
*cmd* 'LIST'
*put* 'LIST\r\n'
*get* '150 Here comes the directory listing.\r\n'
*resp* '150 Here comes the directory listing.'
*retr* 'drwxrwxrwx    3 0        0            4096 Nov 27 18:36 backup\r\n'
drwxrwxrwx    3 0        0            4096 Nov 27 18:36 backup
*retr* 'drwxrwxrwx    2 0        0            4096 Nov 27 18:37 css\r\n'
<LOT OF DIRS AND FILES>
*get* '226 Directory send OK.\r\n'
*resp* '226 Directory send OK.'

---------------------8<--------------------






>
> Another thing: The example code I found under
>
> http://stackoverflow.com/questions/3907826/how-to-upload-a-file-on-ftps-server-using-m2crypto
> does _not_ use `auth_tls`, so you might try that.
>

I needed to add `auth_tls`, otherwise I get this:


*get* '220-IBM Portal\r\n'
*get* '220 \r\n'
*resp* '220-IBM Portal\n220 '
*cmd* 'USER myusername'
*put* 'USER myusername\r\n'
*get* '331 Non-anonymous sessions must use encryption.\r\n'
*resp* '331 Non-anonymous sessions must use encryption.'
*cmd* 'PASS ********'
*put* 'PASS ********\r\n'
*get* '503 Login with USER first.\r\n'
*resp* '503 Login with USER first.'





>
> > I'll try to copy ftputil from python 2.7 and add it to my
> > project (running python 2.6) and see if it works.
>
> What did you get?
>

I did a copy of python 2.7 ftplib.py into a py27ftplib.py file in my
project (which is running python 2.6)

This new FTP class itself gave some errors when trying to list files (can't
remember now what was the messages).
But surely wasn't ftputil fault, because even a test using only FTP object
had failed too.

Obviously tring to use the new FTP_TLS didn't work too, because it is was
based on this py27ftplib.FTP.

BUT, when I did a monkey patch on py27ftplib.FTP_TLS to make is subcalss
python 2.6 FTP class, everything worked well.
Summary:

I am still using python 2.6 FTP class.
I am using python 2.7 FTP_TLS class, but making it subclass python 2.6 FTP
class


ftputil and I are now happy...   :)




>
> Apart from that it would be interesting to find out what is
> different in the M2Crypto case, i. e. why it fails exactly.
> I usually try to understand why something happens.
> Otherwise, I'm sure, ftputil would have a few bugs more. ;-)
>


Yeah, I do also like to understand things. But I was/am in a rush now...
and this solution is fine for me,
because I don't want to add another dependencies to the project.  (will
make it easier to migrate do
python 2.7).


Many thanks !!

Best regards,

Roger
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sschwarzer.net/pipermail/ftputil/attachments/20140411/d30065ec/attachment.html>


More information about the ftputil mailing list