cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
2056
Views
0
Helpful
2
Replies
Patricia Hindes
Beginner

SFTP MGET of large files fails - connection closed - problem with spool file

I have a new SFTP job to get files from an FTP Server.  The files are large (80mg, 150mg).  I can get smaller files from the ftp site with no issue, but when attempting the larger files the job completes abnormally after 2 min 1 sec. each time.  I can see the file is created on our local file system with 0 bytes, then when the FTP job fails, the 0 byte file is deleted.

Is there a limit to how large an ftp file can be in Tidal?  How long an ftp job can run?

The error in the job audit is Problem with spool file for job XXXX_SFTPGet and an exit code of 127 (whatever that is).

In the log, the error is that the connection was closed.  I have checked with the ftp host and their logs show that we are disconnecting unexpectedly also.

Below is an excerpt from the log

DEBUG [SFTPMessage] 6 Feb 2015 14:17:33.055 : Send : Name=SSH_FXP_STAT,Type=17,RequestID=12

DEBUG [SSH2Channel] 6 Feb 2015 14:17:33.055 : Transmit 44 bytes

DEBUG [ChannelDataWindow] 6 Feb 2015 14:17:33.055 : Remote window size decreased to 130808

DEBUG [PlainSocket] 6 Feb 2015 14:17:33.071 : RepeatCallback received 84 bytes

DEBUG [SSH2Connection] 6 Feb 2015 14:17:33.071 : ProcessPacket pt=SSH_MSG_CHANNEL_DATA

DEBUG [SFTPMessageFactory] 6 Feb 2015 14:17:33.071 : Received message (type=105,len=37)

DEBUG [SFTPMessageStore] 6 Feb 2015 14:17:33.071 : AddMessage(12) - added to store

DEBUG [SFTPMessage] 6 Feb 2015 14:17:33.071 : Reply : Name=SSH_FXP_ATTRS,Type=105,RequestID=12

DEBUG [SFTPMessage] 6 Feb 2015 14:17:33.071 : Send : Name=SSH_FXP_OPEN,Type=3,RequestID=13

DEBUG [SSH2Channel] 6 Feb 2015 14:17:33.071 : Transmit 56 bytes

DEBUG [ChannelDataWindow] 6 Feb 2015 14:17:33.071 : Remote window size decreased to 130752

DEBUG [PlainSocket] 6 Feb 2015 14:17:33.087 : RepeatCallback received 52 bytes

DEBUG [SSH2Connection] 6 Feb 2015 14:17:33.087 : ProcessPacket pt=SSH_MSG_CHANNEL_DATA

DEBUG [SFTPMessageFactory] 6 Feb 2015 14:17:33.087 : Received message (type=102,len=10)

DEBUG [SFTPMessageStore] 6 Feb 2015 14:17:33.087 : AddMessage(13) - added to store

DEBUG [SFTPMessage] 6 Feb 2015 14:17:33.087 : Reply : Name=SSH_FXP_HANDLE,Type=102,RequestID=13

DEBUG [SFTPMessage] 6 Feb 2015 14:17:33.087 : Send : Name=SSH_FXP_READ,Type=5,RequestID=14

DEBUG [SSH2Channel] 6 Feb 2015 14:17:33.087 : Transmit 26 bytes

DEBUG [ChannelDataWindow] 6 Feb 2015 14:17:33.087 : Remote window size decreased to 130726

DEBUG [PlainSocket] 6 Feb 2015 14:17:33.118 : RepeatCallback received 0 bytes

DEBUG [SFTPChannelReceiver] 6 Feb 2015 14:17:33.118 : Connection closed:  (code=0)

ERROR [SFTPMessageStore] 6 Feb 2015 14:17:33.118 : Disconnected unexpectedly ( [errorcode=0])

ERROR [SFTPMessageStore] 6 Feb 2015 14:17:33.118 : EnterpriseDT.Net.Ftp.Ssh.SFTPException:  [errorcode=0]

ERROR [SFTPMessageStore] 6 Feb 2015 14:17:33.118 :    at EnterpriseDT.Net.Ftp.Ssh.SFTPMessageStore.CheckState()

ERROR [SFTPMessageStore] 6 Feb 2015 14:17:33.118 :    at EnterpriseDT.Net.Ftp.Ssh.SFTPMessageStore.GetMessage(Int32 requestId)

2 REPLIES 2
Marc Clasby
Beginner

I believe there is a limitation on FTP and what you are seeing is a timeout built into the 3rd party application that tidal uses (I feel like it was hardcoded and it would be a big deal to change but this was before Cisco purchased tidal)  there may have been a tagent.ini setting that tweaks that but I can't find any details.

 

We wound up purchasing our own FTP software (ipswitch MOVEit Central & DMZ) because we also had the need to host as well as Get/Put to other FTP sites. It now does all our FTP and internal file delivery activity (we use it's api and call from tidal if we need to trigger inside a workflow)

 

 

We have no issues with large files and ftp. The issue is with SFTP.
Content for Community-Ad
This widget could not be displayed.