9min

Sending Fargate Logs to ContainIQ

Create a Firehose stream in Kinesis

Document image
  • Change delivery stream name to ContainIQ
  • Select Direct Put for the source field
  • Select HTTP endpoint for the destination
  • Put https://api.containiq.com/ingest/logs/fargate into the HTTP endpoint URL
  • Put your API key into the access key field
  • Add the parameter cluster with the value equal to what was set when you initially deployed the ContainIQ agent

AWS configuration

Create the aws-observability namespace

Run kubectl create ns aws-observability.

Create the FluentBit config

Replace <aws-region> below with the region of your cluster.

Shell
|

Apply the config

Run kubectl apply -f fluentbit-config.yaml.

Create the role JSON

Replace resource below with the ARN from the Firehose stream you created in the first step.

Shell
|

Apply the role JSON

Run aws iam create-policy \ --policy-name FluentBitEKSFargate \ --policy-document file://allow_kinesis_put_permission.json.

Set the pod execution role

Replace <Your-Cluster-Name> below with the name of your EKS cluster. You can find this name by running aws eks list-clusters.

Replace <fargate-profile> below with the Fargate profile you want to get logs from. You can find this by running aws eks list-fargate-profiles --cluster-name <Your-Cluster-Name> .

POD_EXEC_ROLE=$(aws eks describe-fargate-profile \ --cluster-name <Your-Cluster-Name> \ --fargate-profile-name <fargate-profile> | jq -r '.fargateProfile.podExecutionRoleArn' | awk -F"/" '{print (NF>1)? $NF : ""}' )

Set the policy name

Shell
|

Attach the policy

Run aws iam attach-role-policy \ --policy-arn $POLICY_NAME \ --role-name $POD_EXEC_ROLE.