Logz.io Logging for MuleSoft using LogzioAppender

Note: While copy/paste the code snippet from blog please covert quotes and double quotes.

In this blog we will try to learn how we can send logs to Logz.io from Mulesoft using LogzioAppender.

What is Logz.io?

• Logz.io provides log management and log analysis services.
• The platform combines ELK as a cloud service and machine learning to derive new insights from machine data.

Register with Logz.io:

• Go to https://logz.io/
• Click on Free Trial if not already have account

Logz.io

Generate Token:

• Go to https://logz.io/
• Click On Login
• Provide credentials
• Go to Gear Button -> Settings -> General

Settings -> General

• Copy Token

Token

Enable Custom Logging For On Premise MuleSoft Runtime:

• Go to Project
• Open pom.xml and add below dependencies

<! —  https://mvnrepository.com/artifact/org.apache.logging.log4j/log4j-core →
<dependency>
 <groupId>org.apache.logging.log4j</groupId>
 <artifactId>log4j-core</artifactId>
 <version>2.8.2</version>
</dependency>
<! — https://mvnrepository.com/artifact/org.apache.logging.log4j/log4j-api →
<dependency>
 <groupId>org.apache.logging.log4j</groupId>
 <artifactId>log4j-api</artifactId>
 <version>2.8.2</version>
</dependency>
<dependency>
 <groupId>io.logz.log4j2</groupId>
 <artifactId>logzio-log4j2-appender</artifactId>
 <version>1.0.12</version>
</dependency>

• Open src/main/resoruces/log4j2.xml
• Add below xml tag in Configuration/Appenders

<LogzioAppender name=”Logzio”>
 <logzioToken>kILyrCdlJyvKFeCtSyuQMgSFkCACtWdd</logzioToken>
 <logzioType>MuleSoft</logzioType>
 <logzioUrl>https://listener.logz.io:8071</logzioUrl>
 <debug>false</debug>
 <drainTimeoutSec>1</drainTimeoutSec>
 <addHostname>true</addHostname>
</LogzioAppender>

• Add below xml tag in Configuration/Loggers

<Root level=”info”>
 <AppenderRef ref=”Logzio”/>
</Root>

• Run Application
• See the Console Log

Application Logger Console

• Go to https://logz.io/
• Kibana
• Click on Refresh

Kibana -> Logs

Enable Custom Logging For CloudHub Application:

Before enabling the logging for cloudhub application, you need Disable CloudHub logs. By default this option is not available and you need to raise a ticket with MuleSoft for providing this option.

Anypoint Settins

Once you disabled cloudhub logs, MuleSoft is not responsible for below things

• MuleSoft is not responsible for lost logging data due to misconfiguration of your own log4j appender.
• MuleSoft is also not responsible for misconfigurations that result in performance degradation, running out of disk space, or other side effects.
• When you disable the default CloudHub application logs, then only the system logs are available. For application worker logs, please check your own application’s logging system. Downloading logs is not an option in this scenario.
• Only Asynchronous log appenders can be used, Synchronous appenders should not be used.
• Use asynchronous loggers and not synchronous ones is to avoid threading issues. Synchronous loggers can lock threads waiting for responses.

Open pom.xml and add below dependencies
<! —  https://mvnrepository.com/artifact/org.apache.logging.log4j/log4j-core →
<dependency>
 <groupId>org.apache.logging.log4j</groupId>
 <artifactId>log4j-core</artifactId>
 <version>2.8.2</version>
</dependency>
<! — https://mvnrepository.com/artifact/org.apache.logging.log4j/log4j-api →
<dependency>
 <groupId>org.apache.logging.log4j</groupId>
 <artifactId>log4j-api</artifactId>
 <version>2.8.2</version>
</dependency>
<dependency>
 <groupId>io.logz.log4j2</groupId>
 <artifactId>logzio-log4j2-appender</artifactId>
 <version>1.0.12</version>
</dependency>

You need to create log4j2.xml at location src/main/resources.

MuleSoft Application Folder Structure

You need to add a LogzioAppender inlog4j.xml. Provide url to connect Logz.io with token and type of logger, source, pattern layout etc.

<LogzioAppender name=”Logzio”>
 <logzioToken>kILyrCdlJyvKFeCtSyuQMgSFkCACtWdd</logzioToken>
 <logzioType>MuleSoft</logzioType>
 <logzioUrl>https://listener.logz.io:8071</logzioUrl>
 <debug>false</debug>
 <drainTimeoutSec>1</drainTimeoutSec>
 <addHostname>true</addHostname>
</LogzioAppender>

You can add more loggers like Log4J2CloudhubLogAppender into your log4j2.xml to enable logging on the cloudhub log console of your application.

<Log4J2CloudhubLogAppender name=”CLOUDHUB”
 addressProvider=”com.mulesoft.ch.logging.DefaultAggregatorAddressProvider”
 applicationContext=”com.mulesoft.ch.logging.DefaultApplicationContext”
 appendRetryIntervalMs=”${sys:logging.appendRetryInterval}”
 appendMaxAttempts=”${sys:logging.appendMaxAttempts}”
 batchSendIntervalMs=”${sys:logging.batchSendInterval}”
 batchMaxRecords=”${sys:logging.batchMaxRecords}” memBufferMaxSize=”${sys:logging.memBufferMaxSize}”
 journalMaxWriteBatchSize=”${sys:logging.journalMaxBatchSize}”
 journalMaxFileSize=”${sys:logging.journalMaxFileSize}”
 clientMaxPacketSize=”${sys:logging.clientMaxPacketSize}”
 clientConnectTimeoutMs=”${sys:logging.clientConnectTimeout}”
 clientSocketTimeoutMs=”${sys:logging.clientSocketTimeout}”
 serverAddressPollIntervalMs=”${sys:logging.serverAddressPollInterval}”
 serverHeartbeatSendIntervalMs=”${sys:logging.serverHeartbeatSendIntervalMs}”
 statisticsPrintIntervalMs=”${sys:logging.statisticsPrintIntervalMs}”>
 <PatternLayout pattern=”[%d{MM-dd HH:mm:ss}] %-5p %c{1} [%t]: %m%n” />
</Log4J2CloudhubLogAppender>
Below is full log4j2.xml which can be used for your application for enabling custom logging on cloudhub and LogzioAppender for logz.io.
<?xml version=”1.0" encoding=”UTF-8"?>
<Configuration status=”INFO” name=”cloudhub”
 packages=”com.mulesoft.ch.logging.appender,org.apache.logging.log4j”>
 <Appenders>
 <Log4J2CloudhubLogAppender name=”CLOUDHUB”
 addressProvider=”com.mulesoft.ch.logging.DefaultAggregatorAddressProvider”
 applicationContext=”com.mulesoft.ch.logging.DefaultApplicationContext”
 appendRetryIntervalMs=”${sys:logging.appendRetryInterval}”
 appendMaxAttempts=”${sys:logging.appendMaxAttempts}”
 batchSendIntervalMs=”${sys:logging.batchSendInterval}”
 batchMaxRecords=”${sys:logging.batchMaxRecords}” memBufferMaxSize=”${sys:logging.memBufferMaxSize}”
 journalMaxWriteBatchSize=”${sys:logging.journalMaxBatchSize}”
 journalMaxFileSize=”${sys:logging.journalMaxFileSize}”
 clientMaxPacketSize=”${sys:logging.clientMaxPacketSize}”
 clientConnectTimeoutMs=”${sys:logging.clientConnectTimeout}”
 clientSocketTimeoutMs=”${sys:logging.clientSocketTimeout}”
 serverAddressPollIntervalMs=”${sys:logging.serverAddressPollInterval}”
 serverHeartbeatSendIntervalMs=”${sys:logging.serverHeartbeatSendIntervalMs}”
 statisticsPrintIntervalMs=”${sys:logging.statisticsPrintIntervalMs}”>
 <PatternLayout pattern=”[%d{MM-dd HH:mm:ss}] %-5p %c{1} [%t]: %m%n” />
 </Log4J2CloudhubLogAppender>
 <LogzioAppender name=”Logzio”>
 <logzioToken>kILyrCdlJyvKFeCtSyuQMgSFkCACtWdd</logzioToken>
 <logzioType>MuleSoft</logzioType>
 <logzioUrl>https://listener.logz.io:8071</logzioUrl>
 <debug>false</debug>
 <drainTimeoutSec>1</drainTimeoutSec>
 <addHostname>true</addHostname>
 </LogzioAppender>
 </Appenders>
 <Loggers>
 <AsyncLogger name=”org.mule.runtime.core.internal.processor.LoggerMessageProcessor” level=”INFO” />
 <AsyncLogger name=”com.mulesoft.agent” level=”INFO” />
 <AsyncRoot level=”INFO”>
 <AppenderRef ref=”CLOUDHUB” />
 </AsyncRoot>
 <AsyncLogger name=”com.gigaspaces” level=”ERROR” />
 <AsyncLogger name=”com.j_spaces” level=”ERROR” />
 <AsyncLogger name=”com.sun.jini” level=”ERROR” />
 <AsyncLogger name=”net.jini” level=”ERROR” />
 <AsyncLogger name=”org.apache” level=”WARN” />
 <AsyncLogger name=”org.apache.cxf” level=”WARN” />
 <AsyncLogger name=”org.springframework.beans.factory” level=”WARN” />
 <AsyncLogger name=”org.mule” level=”INFO” />
 <AsyncLogger name=”com.mulesoft” level=”INFO” />
 <AsyncLogger name=”org.jetel” level=”WARN” />
 <AsyncLogger name=”Tracking” level=”WARN” />
 <AsyncLogger name=”org.mule” level=”INFO” />
 <AsyncLogger name=”com.mulesoft” level=”INFO” />
 <AsyncLogger name=”org.mule.extensions.jms” level=”INFO” />
 <AsyncLogger name=”org.mule.service.http.impl.service.HttpMessageLogger” level=”INFO” />
 <AsyncLogger name=”org.mule.extension.salesforce” level=”INFO” />
 <AsyncLogger name=”org.mule.extension.ftp” level=”INFO” />
 <AsyncLogger name=”org.mule.extension.sftp” level=”INFO” />
 <AsyncLogger name=”com.mulesoft.extension.ftps” level=”INFO” />
 <AsyncLogger name=”org.mule.modules.sap” level=”INFO” />
 <AsyncLogger name=”com.mulesoft.extension.mq” level=”INFO” />
 <AsyncLogger name=”com.mulesoft.mq” level=”INFO” />
 <AsyncLogger name=”org.mule.extension.db” level=”INFO” />
 <AsyncLogger name=”httpclient.wire” level=”DEBUG” />
 <AsyncLogger name=”org.mule.transport.email” level=”DEBUG” />
 <Root level=”info”>
 <AppenderRef ref=”Logzio”/>
 </Root>
 </Loggers>
</Configuration>

This is how you can enable Log.io logging using LogzioAppender for MuleSoft applications.

Note: If the above configuration does not work please and you get issues like below.

2020–04–16 19:39:09,443 WrapperListener_start_runner ERROR Appenders contains an invalid element or attribute “LogzioAppender”
2020–04–16 19:39:09,453 WrapperListener_start_runner ERROR Unable to locate appender “Logzio” for logger config “root”

Please try to add below entries in log4j2.xml.

• Add attribute packages=”io.logz.log4j2" in configuration

Please try to add below entries in pom.xml

• Add below dependency

<dependency>
 <groupId>io.logz.sender</groupId>
 <artifactId>logzio-sender</artifactId>
 <version>1.1.1</version>
</dependency>