In certain situations, it may not be feasible to run the ContainIQ agent as a daemonset. For example, with AWS Fargate you cannot schedule daemonsets onto Fargate nodes, or with GKE autopilot you may run into limitations around Fluentd's host map. Fortunately, ContainIQ also supports collecting logs from a sidecar container.
Setup
Start by creating a ConfigMap with the necessary Fluentd config.
Replace /path-to-log-file with the path where you are logging to
If you would like to add additional fields to ContainIQ for improved searchability you will need to add the additional fields to the record section of the configuration. the following are available
pod_name ex: pod_name ${record["app-field-name-in-log"]}
container
stream
- name : fluentd
imagePullPolicy : Always
image : containiq/logging- agent
env :
- name : K8S_NODE_NAME
valueFrom :
fieldRef :
fieldPath : spec.nodeName
- name : INGEST_LOGS_ENDPOINT_URL
value : https: //api.containiq.com/ingest/logs
- name : NOTIFICATION_FILE_LOCATION
value : /containiq/notification- config.yaml
- name : FLUENT_KUBERNETES_METADATA_SKIP_LABELS
value : 'true'
- name : FLUENT_KUBERNETES_METADATA_SKIP_CONTAINER_METADATA
value : 'true'
- name : FLUENT_KUBERNETES_METADATA_SKIP_NAMESPACE_METADATA
value : 'true'
- name : FLUENT_KUBERNETES_METADATA_SKIP_MASTER_URL
value : 'true'
- name : FLUENT_CONTAINER_TAIL_EXCLUDE_PATH
value : /var/log/containers/fluent*
- name : FLUENTD_SYSTEMD_CONF
value : 'disable'
- name : CLUSTER_NAME
valueFrom :
configMapKeyRef :
name : cluster- config
key : cluster- name
resources :
limits :
memory : 200Mi
requests :
cpu : 100m
memory : 200Mi
volumeMounts :
- name : containiqconfig
mountPath : /containiq
- name : sidecarconf
mountPath : /fluentd/etc/conf.d
- name : varlog
mountPath : /path/to/logs
volumes :
- name : varlog
emptyDir : { }
- name : sidecarconf
configMap :
name : sidecarconf
- name : containiqconfig
projected :
sources :
- secret :
name : containiq- notification
Next add the Fluentd container as a sidecar in your deployment and mount the correct volumes.
Replace /path/to/logs with the location of your log file.
Add the varlog volume mount to the container you will be logging from.
- name : fluentd
imagePullPolicy : Always
image : containiq/logging- agent
env :
- name : K8S_NODE_NAME
valueFrom :
fieldRef :
fieldPath : spec.nodeName
- name : INGEST_LOGS_ENDPOINT_URL
value : https: //api.containiq.com/ingest/logs
- name : NOTIFICATION_FILE_LOCATION
value : /containiq/notification- config.yaml
- name : FLUENT_KUBERNETES_METADATA_SKIP_LABELS
value : 'true'
- name : FLUENT_KUBERNETES_METADATA_SKIP_CONTAINER_METADATA
value : 'true'
- name : FLUENT_KUBERNETES_METADATA_SKIP_NAMESPACE_METADATA
value : 'true'
- name : FLUENT_KUBERNETES_METADATA_SKIP_MASTER_URL
value : 'true'
- name : FLUENT_CONTAINER_TAIL_EXCLUDE_PATH
value : /var/log/containers/fluent*
- name : FLUENTD_SYSTEMD_CONF
value : 'disable'
- name : CLUSTER_NAME
valueFrom :
configMapKeyRef :
name : cluster- config
key : cluster- name
resources :
limits :
memory : 200Mi
requests :
cpu : 100m
memory : 200Mi
volumeMounts :
- name : containiqconfig
mountPath : /containiq
- name : sidecarconf
mountPath : /fluentd/etc/conf.d
- name : varlog
mountPath : /path/to/logs
volumes :
- name : varlog
emptyDir : { }
- name : sidecarconf
configMap :
name : sidecarconf
- name : containiqconfig
projected :
sources :
- secret :
name : containiq- notification
Finally, add the Fluentd service account under the spec:
spec :
serviceAccount : fluentd
serviceAccountName : fluentd
Pleas reach out to support@containiq.com with any questions.