Weekly Journal 11 - S3 Logging

Synopsis

This has been another long week of doing mostly managerial work. With only a couple of weeks left in March, we have to start planning for next quarter. I did get to spend some time learning about The different logging options for Amazon S3 and how enable them.

S3 Logging

CloudTrail

First up on the list of S3 logging options is AWS CloudTrail. CloudTrail creates an audit trail of API activity inside of your AWS account. CloudTrail has two parts, management events and data events. By default, only management events are recorded; data events requires additional configuration, and incurs additional costs.

In terms of S3, CloudTrail management events will log activity related to the provisioning and configuration of S3 buckets. Examples would include events like creating a new bucket, assigning a policy to a bucket, and enabling bucket encryption. This information is valuable when you need to trace changes, and it is an aid to monitor for suspicious activity in your account.

The other half of CloudTrail is data events. Data events log access and changes to individual objects in an S3 bucket. This is how you can see who created or deleted objects, when they were last accessed and similar information. This translates into GetObject, PutObject and DeleteObject API calls. If you have a lot of activity in your S3 buckets, logging data events can get expensive. Fortunately, there are configuration options for limiting logging to specific prefixes and objects to help control costs.

You get 90 days of CloudTrail management events with a very basic viewer for free. You can store events beyond 90 days in S3, and load them into other analysis tools like Amazon Athena if you want to pay for it.

S3 Server Access Logging

The final logging option for S3 buckets is Server Access Logging. If you are accessing your S3 buckets using the REST API, this is where you can get information about the GET, PUT, and DELETE HTTP requests. These log events are useful for keeping track of the HTTP requests and debugging errors related to accessing an S3 bucket.

All three types of logging are useful, but you will want to tailor your use of each type of logging based on your project needs.

What’s Next?

The next few weeks are going to be focused on planning and preparations for next quarter, and conducting interviews. Maybe I can find something interesting to write about related to how we plan and prioritize work for next week.