Datadog Logging for MuleSoft using HTTP Appender

Note: While copy/paste the code snippet from blog please covert quotes and double quotes.

In this blog we will try to learn how we can send logs to Datadog from Mulesoft using HTTP Appender.

What is Datadog?

• Datadog is a monitoring service for cloud-scale applications, providing monitoring of servers, databases, tools, and services, through a SaaS-based data analytics platform.

Register with Datadog:

• Go to https://www.datadoghq.com/
• Click on Get Started Free if not already have account

https://DataDoghq.com

Generate Token:

• Go to https://www.datadoghq.com/
• Click On Login

Datadog Login Page

• Provide credentials
• Go to Gear Integrations -> APIs -> API Keys

API

• Copy Key

Enable Custom Logging For On Premise MuleSoft Runtime:

• Go to Project
• Open src/main/resoruces/log4j2.xml
• Add below xml tag in Configuration/Appenders

<Http name=”DataDog” url=”https://http-intake.logs.datadoghq.com/v1/input?host=${sys:hostName}&amp;ddsource=Mulesoft&amp;service=loggingApp&ddtags=env:Dev">
 <Property name=”Content-Type” value=”application/json” />
 <Property name=”DD-API-KEY” value=”71cc5085b82b1958d749b8e40ab17fc1" />
 <PatternLayout pattern=”%-5p %d [%t] [event: %X{correlationId}] %c: %m%n” />
</Http>

• URL follow below format

https://http-intake.logs.datadoghq.com/v1/input?host=${sys:hostName}&amp;ddsource=<SourceName>&amp;service=<ApplicationName>&ddtags=env:<Environment>

• Make sure you use <SourceName>, <ApplicationName> and <Environment> as log4j property also pass these as JVM argument So it’s not hard coded and can be changed at the time of deployment
• Add below xml tag in Configuration/Loggers/AsyncRoot

<AppenderRef ref=”DataDog” />

• Run Mulesoft Application
• See the Console Log

Mulesoft Console Log

• Go to https://www.datadoghq.com/
• Logs -> Search
• Click on Refresh

DataDog Logs -> Search

Enable Custom Logging For CloudHub Application:

Before enabling the logging for cloudhub application, you need Disable CloudHub logs. By default this option is not available and you need to raise a ticket with MuleSoft for providing this option.

Once you disabled cloudhub logs, MuleSoft is not responsible for below things

• MuleSoft is not responsible for lost logging data due to misconfiguration of your own log4j appender.
• MuleSoft is also not responsible for misconfigurations that result in performance degradation, running out of disk space, or other side effects.
• When you disable the default CloudHub application logs, then only the system logs are available. For application worker logs, please check your own application’s logging system. Downloading logs is not an option in this scenario.
• Only Asynchronous log appenders can be used, Synchronous appenders should not be used.
• Use asynchronous loggers and not synchronous ones is to avoid threading issues. Synchronous loggers can lock threads waiting for responses.

You need to create log4j2.xml at location src/main/resources.

You need to add a Http appender inlog4j.xml. Provide url to connect Datadog with token and type of logger, source, pattern layout etc.

<Http name=”DataDog” url=”https://http-intake.logs.datadoghq.com/v1/input?host=${sys:hostName}&amp;ddsource=Mulesoft&amp;service=loggingApp&ddtags=env:Dev">
 <Property name=”Content-Type” value=”application/json” />
 <Property name=”DD-API-KEY” value=”71cc5085b82b1958d749b8e40ab17fc1" />
 <PatternLayout pattern=”%-5p %d [%t] [event: %X{correlationId}] %c: %m%n” />
</Http>

You can add more loggers like Log4J2CloudhubLogAppender into your log4j2.xml to enable logging on the cloudhub log console of your application.

<Log4J2CloudhubLogAppender name=”CLOUDHUB”
 addressProvider=”com.mulesoft.ch.logging.DefaultAggregatorAddressProvider”
 applicationContext=”com.mulesoft.ch.logging.DefaultApplicationContext”
 appendRetryIntervalMs=”${sys:logging.appendRetryInterval}”
 appendMaxAttempts=”${sys:logging.appendMaxAttempts}”
 batchSendIntervalMs=”${sys:logging.batchSendInterval}”
 batchMaxRecords=”${sys:logging.batchMaxRecords}” memBufferMaxSize=”${sys:logging.memBufferMaxSize}”
 journalMaxWriteBatchSize=”${sys:logging.journalMaxBatchSize}”
 journalMaxFileSize=”${sys:logging.journalMaxFileSize}”
 clientMaxPacketSize=”${sys:logging.clientMaxPacketSize}”
 clientConnectTimeoutMs=”${sys:logging.clientConnectTimeout}”
 clientSocketTimeoutMs=”${sys:logging.clientSocketTimeout}”
 serverAddressPollIntervalMs=”${sys:logging.serverAddressPollInterval}”
 serverHeartbeatSendIntervalMs=”${sys:logging.serverHeartbeatSendIntervalMs}”
 statisticsPrintIntervalMs=”${sys:logging.statisticsPrintIntervalMs}”>
 <PatternLayout pattern=”[%d{MM-dd HH:mm:ss}] %-5p %c{1} [%t]: %m%n” />
</Log4J2CloudhubLogAppender>

Below is full log4j2.xml which can be used for your application for enabling custom logging on cloudhub and http appender for Datadog.

<?xml version=”1.0" encoding=”UTF-8"?>
<Configuration status=”INFO” name=”cloudhub”
 packages=”com.mulesoft.ch.logging.appender,org.apache.logging.log4j”>
 <Appenders>
 <Log4J2CloudhubLogAppender name=”CLOUDHUB”
 addressProvider=”com.mulesoft.ch.logging.DefaultAggregatorAddressProvider”
 applicationContext=”com.mulesoft.ch.logging.DefaultApplicationContext”
 appendRetryIntervalMs=”${sys:logging.appendRetryInterval}”
 appendMaxAttempts=”${sys:logging.appendMaxAttempts}”
 batchSendIntervalMs=”${sys:logging.batchSendInterval}”
 batchMaxRecords=”${sys:logging.batchMaxRecords}” memBufferMaxSize=”${sys:logging.memBufferMaxSize}”
 journalMaxWriteBatchSize=”${sys:logging.journalMaxBatchSize}”
 journalMaxFileSize=”${sys:logging.journalMaxFileSize}”
 clientMaxPacketSize=”${sys:logging.clientMaxPacketSize}”
 clientConnectTimeoutMs=”${sys:logging.clientConnectTimeout}”
 clientSocketTimeoutMs=”${sys:logging.clientSocketTimeout}”
 serverAddressPollIntervalMs=”${sys:logging.serverAddressPollInterval}”
 serverHeartbeatSendIntervalMs=”${sys:logging.serverHeartbeatSendIntervalMs}”
 statisticsPrintIntervalMs=”${sys:logging.statisticsPrintIntervalMs}”>
 <PatternLayout pattern=”[%d{MM-dd HH:mm:ss}] %-5p %c{1} [%t]: %m%n” />
 </Log4J2CloudhubLogAppender>
 <Http name=”DataDog” url=”https://http-intake.logs.datadoghq.com/v1/input?host=${sys:hostName}&amp;ddsource=Mulesoft&amp;service=loggingApp&ddtags=env:Dev">
 <Property name=”Content-Type” value=”application/json” />
 <Property name=”DD-API-KEY” value=”71cc5085b82b1958d749b8e40ab17fc1" />
 <PatternLayout pattern=”%-5p %d [%t] [event: %X{correlationId}] %c: %m%n” />
 </Http>
 </Appenders>
 <Loggers>
 <AsyncLogger name=”org.mule.runtime.core.internal.processor.LoggerMessageProcessor” level=”INFO” />
 <AsyncLogger name=”com.mulesoft.agent” level=”INFO” />
 <AsyncRoot level=”INFO”>
 <AppenderRef ref=”CLOUDHUB” />
 <AppenderRef ref=”DataDog” />
 </AsyncRoot>
 <AsyncLogger name=”com.gigaspaces” level=”ERROR” />
 <AsyncLogger name=”com.j_spaces” level=”ERROR” />
 <AsyncLogger name=”com.sun.jini” level=”ERROR” />
 <AsyncLogger name=”net.jini” level=”ERROR” />
 <AsyncLogger name=”org.apache” level=”WARN” />
 <AsyncLogger name=”org.apache.cxf” level=”WARN” />
 <AsyncLogger name=”org.springframework.beans.factory” level=”WARN” />
 <AsyncLogger name=”org.mule” level=”INFO” />
 <AsyncLogger name=”com.mulesoft” level=”INFO” />
 <AsyncLogger name=”org.jetel” level=”WARN” />
 <AsyncLogger name=”Tracking” level=”WARN” />
 <AsyncLogger name=”org.mule” level=”INFO” />
 <AsyncLogger name=”com.mulesoft” level=”INFO” />
 <AsyncLogger name=”org.mule.extensions.jms” level=”INFO” />
 <AsyncLogger name=”org.mule.service.http.impl.service.HttpMessageLogger” level=”INFO” />
 <AsyncLogger name=”org.mule.extension.salesforce” level=”INFO” />
 <AsyncLogger name=”org.mule.extension.ftp” level=”INFO” />
 <AsyncLogger name=”org.mule.extension.sftp” level=”INFO” />
 <AsyncLogger name=”com.mulesoft.extension.ftps” level=”INFO” />
 <AsyncLogger name=”org.mule.modules.sap” level=”INFO” />
 <AsyncLogger name=”com.mulesoft.extension.mq” level=”INFO” />
 <AsyncLogger name=”com.mulesoft.mq” level=”INFO” />
 <AsyncLogger name=”org.mule.extension.db” level=”INFO” />
 <AsyncLogger name=”httpclient.wire” level=”DEBUG” />
 <AsyncLogger name=”org.mule.transport.email” level=”DEBUG” />
 </Loggers>
</Configuration>

This is how you can enable Datadog logging using HTTP Appender for MuleSoft applications.