AWS Transfer Family and GuardDuty Malware Protection for S3

Malware Protection for S3

How to prevent against malware with AWS Transfer Family and GuardDuty

Businesses frequently have to send material to other parties via the internet in a safe manner. A file transfer server that is open to the public puts the company at risk from malware-infected files that are submitted by threat actors or unintentionally by authorized users. Businesses can help ensure that files received over public channels are checked for malware before processing in order to reduce this risk.

As part of a larger transfer workflow, this post shows how to utilize AWS Transfer Family and Amazon GuardDuty to check files uploaded over a secure FTP (SFTP) server for malware. Instead of scanning using a container image, GuardDuty automatically updates malware signatures every 15 minutes, eliminating the need for manual patching to maintain the signatures current.

Prerequisites

You’ll need the following to implement the solution:

• An account on AWS: To implement this solution, you must have access to AWS. If you don’t already have a usable account, see Start developing on AWS today.
• AWS Command Line Interface (AWS CLI): Install and set up the AWS CLI so that it is linked to your AWS account. Use your environment’s access token and secret access key to configure the environment variables for your AWS account.
• Git: The example code will be downloaded from GitHub using Git.
• Terraform: The automation will be executed using Terraform. To download and install Terraform, adhere to the installation guidelines.

Overview of the solution

GuardDuty and Transfer Family are used in this solution. GuardDuty is an intelligent threat detection service, and Transfer Family offers a secure file transfer service that you can utilise to set up an SFTP server. GuardDuty protects AWS accounts, workloads, and data by keeping an eye out for unusual activity and harmful activities. The following steps are used in the solution at a high level:

• A Transfer Family SFTP server receives a file upload from a user.
• To run an AWS Step Functions workflow, a Transfer Family controlled workflow calls AWS Lambda.
  • Only after the file upload is successful does the workflow start.
  • In order to report a partial upload error, partial uploads to the SFTP server will trigger an error handling Lambda function.
• GuardDuty begins scanning after a step function state machine calls a Lambda function to transfer uploaded files to an Amazon Simple Storage Service (Amazon S3) bucket for processing.
• The step method receives a callback with the GuardDuty scan result.
• Files that are infected are either cleansed or relocated.
• Through an Amazon Simple Notification Service (Amazon SNS) subject, the workflow provides the user with the results. This could be an alert about a harmful upload or problem that occurred during the scan, or it could be a notification about a successful upload and a clean scan that can be processed further.

Architecture and walkthrough of the solution

The solution scans newly uploaded items to the S3 bucket using GuardDuty Malware Protection for S3. This feature of GuardDuty allows you to monitor for particular object prefixes or to configure a malware protection strategy for an S3 bucket at the bucket level.

From the time the file is uploaded until it is scanned and classified as either safe or infected, the processes listed below outline the workflow for this solution. From there, you can modify the steps according to your use case.

• Through Transfer Family, a file is uploaded via the SFTP protocol.
• If the upload is successful, the Managed Workflow Complete workflow is started, and the file is uploaded to the Unscanned S3 bucket using Transfer Family. This procedure calls the Step Function Invoker Lambda function and manages successful uploads.
• The Step Function By calling the GuardDuty Scan Lambda function, the invoker initiates the state machine and the process’s initial step.
• The file is moved to the Processing bucket by the GuardDuty Scan function. The files that will be scanned will come from this bucket.
• GuardDuty examines an item automatically when it detects object upload activity. A malware prevention strategy is developed for the Processing bucket in this implementation.
• GuardDuty publishes the scan result to Amazon EventBridge upon completion.
• Every time a scan event is finished, a Lambda Callback function is called by an EventBridge rule. The function will be called by EventBridge using an event containing the scan results. For an example, see Monitoring S3 item scans with Amazon EventBridge.
• Using the callback task integration paradigm, the Lambda Callback function alerts the GuardDuty Scan task. The GuardDuty scan’s output is provided to the Move File task after being returned to the GuardDuty Scan method.
• The Move File job will move the file to the Clean S3 bucket if the scan yields a clean result with no threats found, signifying that the file has been successfully scanned and is safe for additional processing.
• To alert the subscribers, the Move File function now posts a notice to the Success SNS topic.
• The Move File function will transfer the file to the Quarantine S3 bucket for additional examination if the outcome shows that the file is harmful. To alert the user to the upload of a potentially harmful file, the function will additionally remove the file from the Processing bucket and post a notice in the SNS’s Error topic.
• Transfer Family will initiate the Managed procedure Partial procedure if the file upload fails and is not fully uploaded.
• Controlled Workflow The Error Publisher function, which is used to report errors that arise anywhere in the workflow, is called by the Partial error handling workflow.
• The Error Publisher function determines the type of error and sets the error status appropriately, depending on whether it is due to a partial upload or a problem elsewhere in the workflow. After that, it will post an error message to the SNS’s Error Topic.
• If the file isn’t successfully scanned, the GuardDuty Scan task has a timeout to ensure that an event is broadcast to Error Topic, triggering a manual intervention to look into the matter further. The Error clean up Lambda function is called in the event that the GuardDuty Scan operation fails.
• Lastly, the Processing bucket has an S3 Lifecycle policy associated to it. To ensure that no file is left in the Processing bucket for longer than a day, this is done.

Code base

A sample implementation of this solution utilising Terraform and Python-based Lambda functions may be found in the GitHub AWS-samples project.AWS CloudFormation can build the same solution. The code contains the elements required to implement the complete process in order to showcase the capabilities of the GuardDuty malware prevention plan and Transfer Family.

Install the solution

How to apply this solution in testing.

• Git clones the repository to your working directory.
• Open the cloned project directory and navigate to the root directory.
• In Terraform locals.tf, change S3 bucket, SFTP server, and other variables to your liking.
• Execute the Terraform plan.
• If everything appears to be in order, build the resources by running Terraform Apply and selecting Yes.

Clean up

Cleaning up the resources you produced after testing and investigating the answer is crucial to preventing needless expenses. Go to the root directory of your cloned project and execute the following command to remove the resources produced by this solution:

Terraform-created resources, like as the SFTP server, S3 buckets, Lambda functions, and other elements, will be deleted with this command. When asked, select “yes” to confirm the deletion.

In conclusion

You can ensure that files received over SFTP and transferred to your S3 bucket are inspected for risks and are safe for additional processing by following the steps described in the post. By ensuring that public uploads are securely scanned before being sent to other parts of your system, the solution lowers the exposure surface.