I have a new SFTP job to get files from an FTP Server. The files are large (80mg, 150mg). I can get smaller files from the ftp site with no issue, but when attempting the larger files the job completes abnormally after 2 min 1 sec. each time. I can see the file is created on our local file system with 0 bytes, then when the FTP job fails, the 0 byte file is deleted.
Is there a limit to how large an ftp file can be in Tidal? How long an ftp job can run?
The error in the job audit is Problem with spool file for job XXXX_SFTPGet and an exit code of 127 (whatever that is).
In the log, the error is that the connection was closed. I have checked with the ftp host and their logs show that we are disconnecting unexpectedly also.
Below is an excerpt from the log
DEBUG [SFTPMessage] 6 Feb 2015 14:17:33.055 : Send : Name=SSH_FXP_STAT,Type=17,RequestID=12
DEBUG [SSH2Channel] 6 Feb 2015 14:17:33.055 : Transmit 44 bytes
DEBUG [ChannelDataWindow] 6 Feb 2015 14:17:33.055 : Remote window size decreased to 130808
DEBUG [PlainSocket] 6 Feb 2015 14:17:33.071 : RepeatCallback received 84 bytes
DEBUG [SSH2Connection] 6 Feb 2015 14:17:33.071 : ProcessPacket pt=SSH_MSG_CHANNEL_DATA
DEBUG [SFTPMessageFactory] 6 Feb 2015 14:17:33.071 : Received message (type=105,len=37)
DEBUG [SFTPMessageStore] 6 Feb 2015 14:17:33.071 : AddMessage(12) - added to store
DEBUG [SFTPMessage] 6 Feb 2015 14:17:33.071 : Reply : Name=SSH_FXP_ATTRS,Type=105,RequestID=12
DEBUG [SFTPMessage] 6 Feb 2015 14:17:33.071 : Send : Name=SSH_FXP_OPEN,Type=3,RequestID=13
DEBUG [SSH2Channel] 6 Feb 2015 14:17:33.071 : Transmit 56 bytes
DEBUG [ChannelDataWindow] 6 Feb 2015 14:17:33.071 : Remote window size decreased to 130752
DEBUG [PlainSocket] 6 Feb 2015 14:17:33.087 : RepeatCallback received 52 bytes
DEBUG [SSH2Connection] 6 Feb 2015 14:17:33.087 : ProcessPacket pt=SSH_MSG_CHANNEL_DATA
DEBUG [SFTPMessageFactory] 6 Feb 2015 14:17:33.087 : Received message (type=102,len=10)
DEBUG [SFTPMessageStore] 6 Feb 2015 14:17:33.087 : AddMessage(13) - added to store
DEBUG [SFTPMessage] 6 Feb 2015 14:17:33.087 : Reply : Name=SSH_FXP_HANDLE,Type=102,RequestID=13
DEBUG [SFTPMessage] 6 Feb 2015 14:17:33.087 : Send : Name=SSH_FXP_READ,Type=5,RequestID=14
DEBUG [SSH2Channel] 6 Feb 2015 14:17:33.087 : Transmit 26 bytes
DEBUG [ChannelDataWindow] 6 Feb 2015 14:17:33.087 : Remote window size decreased to 130726
DEBUG [PlainSocket] 6 Feb 2015 14:17:33.118 : RepeatCallback received 0 bytes
DEBUG [SFTPChannelReceiver] 6 Feb 2015 14:17:33.118 : Connection closed: (code=0)
ERROR [SFTPMessageStore] 6 Feb 2015 14:17:33.118 : Disconnected unexpectedly ( [errorcode=0])
ERROR [SFTPMessageStore] 6 Feb 2015 14:17:33.118 : EnterpriseDT.Net.Ftp.Ssh.SFTPException: [errorcode=0]
ERROR [SFTPMessageStore] 6 Feb 2015 14:17:33.118 : at EnterpriseDT.Net.Ftp.Ssh.SFTPMessageStore.CheckState()
ERROR [SFTPMessageStore] 6 Feb 2015 14:17:33.118 : at EnterpriseDT.Net.Ftp.Ssh.SFTPMessageStore.GetMessage(Int32 requestId)
