Log Stream
Last modified on February 28, 2024
This feature is part of the Enterprise bundle. If it is not enabled for your organization, please contact StrongDM at the StrongDM Help Center.
Log Stream allows you to stream your StrongDM audit logs to a third-party object storage service, such as Amazon S3. Having logs streamed to an external storage location for ingestion into security information and event management (SIEM) applications can help provide insights and visibility to compliance, security, and other teams.
For more information about logging, please see Logs.
How to Configure Log Stream
Log Stream is enabled and configured in the Admin UI in Settings > Log Encryption & Storage.
Amazon S3
The following steps show you how to configure Log Stream for streaming to an Amazon S3 bucket.
Under Log Stream, select Yes for Enable Log Stream?.
Select the Provider name (
Amazon S3).Enter the Region of your target bucket (for example,
us-west-2).Enter the Bucket Name (for example,
unique-bucket-name).Enter the Key Prefix (for example,
sdm/logs).Your region, bucket name, and key prefix values are then populated in the bucket policy. Click Copy to save the bucket policy to your clipboard.
Example bucket policy:
{ "Version": "2012-10-17", "Statement": [ { "Sid": "StrongDMLogStream", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::910226215634:role/StrongDMLogStream" }, "Action": "s3:PutObject", "Resource": "arn:aws:s3:::unique-bucket-name/sdm/logs/*" } ] }In a separate browser tab or window, sign in to the AWS Management Console and open the Amazon S3 console. From Buckets, choose the name of the desired bucket. On the bucket’s Permissions tab, click Edit bucket policy to add the bucket policy that you copied.
After saving changes to your bucket policy, return to the Admin UI Log Stream settings and click Test to test that data from StrongDM can be written to your S3 bucket location. The test uses the provided bucket name and key prefix values to write a test object to
<BUCKET_NAME>/<KEY_PREFIX>/test.
A successful test displays the “Log Stream test successful” message. If the test fails, check that the region, bucket name, and key prefix values that you entered are correct, and ensure that the bucket policy is saved correctly in the Amazon S3 console.
When configuration is complete, your audit logs are streamed to the connected S3 bucket.
If an unrecoverable error occurs when streaming logs to the target storage location, such as if StrongDM is unable to stream some or all data to the target storage location, StrongDM administrators in your organization receive an email detailing the problem. In addition, the Log Stream settings provide more information about the stream status.
Moreover, if StrongDM detects a streaming failure, then logs will no longer be streamed to your configured location. Once you have resolved the connection issue, logs will automatically start streaming again. If you wish to, you can retrieve logs from the duration of the failure using the StrongDM API or CLI.
Log storage in S3
Logs that are streamed to S3 are batched into multiple log entries and written to a file.
The location and format of each file is as follows:
- Activities:
/activities/YYYY/MM/DD/HH/MM/<randomUUID>.json - Queries:
/queries/YYYY/MM/DD/HH/MM/<randomUUID>.json - Replays:
/replays/YYYY/MM/DD/HH/MM/<randomUUID>.json
This storage format of logs includes multiple log entries in a single file, batching groups of writes together. It also can include log entries from multiple different activities or queries in the same file. One or more JSON log entries are separated by a new line.
Entries are written immediately, unless a write is already in progress, in which case up to 1000 entries or approximately 100 MiB of entries are accumulated for the subsequent write.
If you have an automation that parses the log files, please ensure that the scripts match the format and location of the data.
Examples of Streamed Logs
When Log Stream is configured, several varieties of logs are streamed to the target storage location, including activities, queries for all resource types, and replay data for interactive sessions. This section provides examples of each log format.
Test
{
"formatVersion": "v1.0.0",
"logType": "activities",
"actorAccountID": "a-0000000000000000",
"timestamp": "0001-01-01T00:00:00Z"
}
Activity log example
{
"formatVersion": "v1.0.0",
"id": "at-33faa6fb65df876a",
"logType": "activities",
"activity": "user logged into the Admin UI",
"actorAccountID": "a-0abcdabcdab00000",
"actorEmail": "alice.glick@strongdm.com",
"actorFirstName": "Alice",
"actorLastName": "Glick",
"actorExternalID": "e-bca5454",
"description": "Alice Glick (alice.glick@strongdm.com) logged into the Admin UI.",
"ipAddress": "123.123.123.123",
"userAgent": "strongdm-cli/1.2.3",
"entities": [
{
"type": "user",
"id": "a-0abcdabcdab00000",
"email": "alice.glick@strongdm.com"
}
],
"timestamp": "2024-02-01T18:50:04.782960647Z"
}
{
"formatVersion": "v1.0.0",
"id": "at-44faa6fb65df876b",
"logType": "activities",
"activity": "User added to role",
"actorAccountID": "a-0abcdabcdab00000",
"actorEmail": "alice.glick@strongdm.com",
"actorFirstName": "Alice",
"actorLastName": "Glick",
"actorExternalID": "e-bca5454",
"description": "Alice Glick (alice.glick@strongdm.com) assigned Bob Belcher (bob.belcher@strongdm.com) to DEVROLE.",
"ipAddress": "123.123.123.123",
"userAgent": "strongdm-cli/1.2.3",
"entities": [
{
"type": "user",
"id": "a-0abcdabcdab00000",
"email": "alice.glick@strongdm.com"
}
],
"timestamp": "2024-02-01T18:50:05.782960648Z"
}
{
"formatVersion": "v1.0.0",
"id": "at-55faa6fb65df876c",
"logType": "activities",
"activity": "user logged into the Admin UI",
"actorAccountID": "a-0defgdefgde00001",
"actorEmail": "bob.belcher@strongdm.com",
"actorFirstName": "Bob",
"actorLastName": "Belcher",
"actorExternalID": "e-def1212",
"description": "Bob Belcher (bob.belcher@strongdm.com) logged into the Admin UI.",
"ipAddress": "456.456.456.456",
"userAgent": "strongdm-cli/1.2.3",
"entities": [
{
"type": "user",
"id": "a-0defgdefgde00001",
"email": "bob.belcher@strongdm.com"
}
],
"timestamp": "2024-02-02T18:50:04.782960646Z"
}
Query log example
{
"formatVersion": "v1.0.0",
"logType": "queries",
"actorAccountID": "a-0abcdabcdab00000",
"actorFirstName": "Bob",
"actorLastName": "Belcher",
"actorEmail": "bob.belcher@strongdm.com",
"actorExternalID": "e-def1212",
"durationMs": "20",
"hash": "0da22222ba9b212ecfed33a17147c466ae0929ca",
"query": "select name from users",
"queryCategory": "cloud",
"resourceID": "rs-0cegcegcegceg00",
"resourceName": "MySQL",
"resourceTags": {
"env": "dev"
},
"resourceType": "mysql",
"sourceIp": "1.11.222.333",
"timestamp": "2024-02-01T13:13:20.895597162Z",
"uuid": "0CEGCEGCEGCEGCEGCEGCE1234ceg"
}
{
"formatVersion": "v1.0.0",
"logType": "queries",
"actorAccountID": "a-0defgdefgde00001",
"actorFirstName": "Bob",
"actorLastName": "Belcher",
"actorEmail": "bob.belcher@strongdm.com",
"actorExternalID": "e-bca5454",
"durationMs": "20",
"hash": "0da22222ba9b212ecfed33a17147c466ae0929fb",
"query": "select name from users",
"queryCategory": "cloud",
"resourceID": "rs-0abcdabcdabcd00",
"resourceName": "AWS Development",
"resourceTags": {
"env": "dev"
},
"resourceType": "aws",
"sourceIp": "1.23.456.789",
"timestamp": "2024-02-02T13:10:20.895597163Z",
"uuid": "0ABCDABCDABCDABCDABCD1234abc"
}
{
"formatVersion": "v1.0.0",
"logType": "queries",
"actorAccountID": "a-0mnomnomnom00003",
"actorFirstName": "Bob",
"actorLastName": "Belcher",
"actorEmail": "bob.belcher@strongdm.com",
"actorExternalID": "e-bca5454",
"durationMs": "42",
"hash": "0da22222ba9b212ecfed44b17147c466ae0929fb",
"query": "select name from users",
"queryCategory": "cloud",
"resourceID": "rs-0abcdabcdabcd00",
"resourceName": "AWS Development",
"resourceTags": {
"env": "dev"
},
"resourceType": "aws",
"sourceIp": "1.23.456.789",
"timestamp": "2024-02-03T12:10:20.895597166Z",
"uuid": "0ABCDABCDABCDABCDABCD1234abc"
}
Replay log example
{
"formatVersion": "v1.0.0",
"logType": "replays",
"chunkID": "1",
"events": [
{
"data": "base64binarydata",
"durationMs": "46"
}
],
"hash": "0da22222ba9b212ecfed33a17147c466ae0929fb",
"queryUUID": "sABCDABCDABCDABCDABCD1234abc",
"timestamp": "2024-02-26T21:14:12.377835194Z"
}
{
"formatVersion": "v1.0.0",
"logType": "replays",
"chunkID": "2",
"events": [
{
"data": "base64binarydata",
"durationMs": "50"
}
],
"hash": "0ab33333ba9b212ecfed33a17147c466ae0929fz",
"queryUUID": "sDEFDEFDEFDEFDEFDEFDE1234def",
"timestamp": "2024-02-26T21:15:12.377835194Z"
}
{
"formatVersion": "v1.0.0",
"logType": "replays",
"chunkID": "3",
"events": [
{
"data": "base64binarydata",
"durationMs": "54"
}
],
"hash": "0lk44444ba9b212ecfed33a17147c466ae0929xy",
"queryUUID": "sXYZXYZXYZXYZXYZXYZXYZ567xyz",
"timestamp": "2024-02-26T21:16:12.377835194Z"
}
Additional Information
When Log Stream is enabled, what data, if any, is stored with StrongDM?
The Log Encryption and Storage settings determine what is stored with StrongDM.
Setting the Store with StrongDM? option to No prevents any data (replay or other queries) from being sent to or stored with StrongDM (that is, the data won’t leave the relays). Only metadata (such as user, resource, and total replay time) is sent and stored, even if Log Stream is enabled.
For the Remote encryption? setting, choosing to store logs with StrongDM with an encryption key will encrypt the query or replay data with the key before it leaves the relay but will still store it (encrypted) with StrongDM and Log Stream.
Setting the Retain SSH, RDP, and Kubernetes Replay Data sent to StrongDM? setting to No prevents replay data from persisting in StrongDM. As long as Log Stream is enabled, replay data will be sent to StrongDM and will be stored in the external storage location. Only non-replay queries and replay metadata will be stored in StrongDM.
Activities will be sent to Log Stream regardless of the Log Encryption and Storage settings.