Introduction

In my previous post, I discussed using Amazon Data Firehose and CloudWatch Metric Streams to ingest metrics into Dynatrace. In this post, I’ll compare this method with another approach: using the Dynatrace ActiveGate agent. I’ll also walk you through the steps to set up ActiveGate for monitoring your AWS environment.

Dynatrace ActiveGate is a multi-purpose module that is capable of performing monitoring tasks (using API) for a wide range of technologies, including but not limited to AWS, VMware, Azure, Kubernetes, OpenShift, Cloud Foundry, Google Cloud, Oracle, SMTP, WMI, Prometheus, and many others.

Metric Ingestion Methods: A Quick Overview

1. Amazon Data Firehose and CloudWatch Metric Streams

• Real-time Data: Provides near real-time metric ingestion by streaming data directly from CloudWatch to a Data Firehose.
• AWS-native Integration: Leverages AWS-native services for streaming, processing, and delivering metrics and logs.
• High Scalability: Suitable for environments with a large number of resources and metrics.
• Complexity: Requires configuring multiple AWS services and continuous management of data streams.

2. Dynatrace ActiveGate Agent

• Ease of Setup: Involves deploying a lightweight agent (ActiveGate) and connecting it to AWS using IAM roles.
• Granular Control: Allows for selecting specific metrics and resources from AWS for monitoring.
• Dynamic Discovery: Automatically discover all AWS services in your environment.
• Dynatrace API: Provides endpoints for accessing Dynatrace API.
• Cost-effective: No additional AWS costs except for the standard CloudWatch API usage.

Comparison Table

Feature/Aspect	Kinesis Data Firehose & CloudWatch Metric Streams	Dynatrace ActiveGate Agent
Setup Complexity	Moderate to High	Low to Moderate
Real-time Streaming	Yes (Near real-time with Metric Streams)	Near real-time
Cost	Additional costs (Kinesis, Metric Streams)	No extra AWS costs (CloudWatch API)
Ease of Management	Requires ongoing management	Easier, automatic discovery
Customization	Highly customizable	Limited (depends on CloudWatch API)

Which Method Should You Use?

• Use Amazon Data Firehose and CloudWatch Metric Streams if you need real-time monitoring and have a large, complex AWS environment that benefits from direct integration with AWS-native streaming services.
• Use Dynatrace ActiveGate if you prefer a simpler, more cost-effective solution by reducing your interaction with Dynatrace to one single point. It also offers optimized traffic volume and less complexity of the network.

How to Implement Dynatrace ActiveGate for Monitoring AWS CloudWatch Metrics

Here are the steps to set up the integration using the ActiveGate approach:

Step 1: Launch an EC2 instance

Launch an EC2 instance using Ubuntu image for installing the ActiveGate agent. The instance type has to have at least 2GB memory, i.e. t2.small or higher to meet the minimum requirements for installing the agent

Log in to your Dynatrace environment and navigate to Deploy ActiveGate. On the Install Linux Environment ActivateGate page, select Create token in order to download the ActiveGate installer.

Copy the command with the token and paste it to the EC2 instance to download the agent installer

wget -O Dynatrace-ActiveGate-Linux-x86-1.297.7.sh "https://<your environment id>.live.dynatrace.com/api/v1/deployment/installer/gateway/unix/latest?arch=x86" --header="Authorization: Api-Token <your token string>"

Copy the run command and run the installer using root rights (i.e. sudo)

sudo /bin/bash Dynatrace-ActiveGate-Linux-x86-1.297.7.sh

Select Show deployment status from the same installation page to verify the deployment status of the ActiveGate agent

Step 2: Set Up AWS IAM Role for ActiveGate

1. Create an IAM role in AWS using CloudFormation template from cloud-snippets/role_based_access_no_AG_template.yml
2. Go to CloudFormation console, choose Stacks and create a new stack with new resources
3. Select Template is ready and upload the role_based_access_no_AG_template.yml template to create the stack for provisioning the required IAM role

Modify the newly launched EC2 to use the new IAM role just created

Create a second IAM role for monitoring with the account hosting the ActiveGate agent using the role_based_access_monitored_account_template.yml template.

Step 3: Connect ActiveGate to AWS

1. In the Dynatrace console, go to Settings > Cloud and virtualization > AWS.
2. Click Connect new instance.
3. For IAM role that Dynatrace should use to get monitoring data, enter Dynatrace_monitoring_role
4. For Amazon account ID, enter your amazon account ID
5. For Token, click Show and copy the value as the External ID for updating your Dynatrace_monitoring_role
    IAM role
6. For Resources to be monitored, select Monitor all resources

Go back to IAM role console, update the sts:ExternalId string under Trust relationships of the Dynatrace_monitoring_role

Step 4: Validate Data Ingestion

1. After successfully connecting the ActiveGate to AWS, navigate to AWS Classic in the Dynatrace console to view the discovered AWS resources and metrics.
2. Create dashboards, set up alerts, and explore auto-discovered metrics for detailed insights into your AWS environment.

Conclusion

Ingesting AWS CloudWatch metrics into Dynatrace can be accomplished through different methods, each with its own set of pros and cons. Using Amazon Data Firehose and CloudWatch Metric Streams provides real-time ingestion but involves more complexity and additional costs. On the other hand, deploying a Dynatrace ActiveGate agent offers a straightforward and cost-effective way for monitoring your AWS environment.

Thank you for reading my blog and I hope you like it!