On April 21st, 2023, at 10:26 AM PST, Files.com received customer reports of elevated errors rates with wildcard searches on FTP, which resulted in an incident being declared. The Incident Management Team (IMT) convened and immediately began investigation.
The elevated error rates with wildcard searches on FTP was resolved on April 21st, 2023, at 10:36 AM PST, returning the platform to full functionality.
Files.com released a resolution Status Page posting on April 21st, 2023, at 11:00 AM PST, stating:
“FTP Service Only: Elevated Error Rates On Wildcard Searches: FTP only: We identified elevated error rates on the FTP service on Files.com in all regions. Attempts to list files using * (wildcard) searches were not returning results. This issue affected wildcard search results from 11:00 a.m. PDT on 4/20 until 10:36 a.m. PDT on 4/21. All services have been restored and are operating normally. This incident did not impact other network services such as API, SFTP, WebDAV, AS2, and others.”
This incident began when a code change was deployed to the Files.com API to make the operation of filtering operate consistently with the API documentation at http://developers.files.com/. However, it turns out that FPS-FTP, Files.com’s FTP server, was relying on undocumented API behavior in order to implement filtering, and therefore broke when the API change was deployed. The incident was resolved by updating FPS-FTP's use of the API to follow the documented API interface.
The root cause of this issue was the Files.com FTP team’s failure to rely on only documented API behavior.
As an additional note, although the Files.com FTP interface uses the Files.com API internally, it does not use a Files.com SDK as a wrapped around that API connection.
We recommend to our customers to always use a Files.com SDK because it provides protection against incorrect API usage, and if we used it ourselves for FTP, it may have prevented this incident. We are tracking a long-term project to have FPS always use a Files.com SDK rather than implementing API methods directly. We hope to have this project accomplished this year.
Extensive review and testing was conducted by Files.com staff to ensure this resolution, and we have already taken steps internally to prevent this issue from recurring in the future.
We greatly appreciate your patience and understanding as we resolved this issue. If you need additional assistance or continue to experience issues, please contact our Customer Support team.