Incremental Mining of System Log Format

Abstract

In modern computer system, system logs are important for problem determination in troubleshooting. Especially in the troubleshooting of systems, system administrators need to understand overview of the problems and identify the root causes quickly, and system logs can help the system administrators. However large numbers of unfamiliar system logs when are generated problems occur, and it's difficult to understand and use them. Most of the existing methods for interpreting system logs don't work immediately and are not useful for troubleshooting situations. We have devised a new method for mining log formats and retrieving log types and parameters in incremental log messages. By creating a structured tree using the nodes generated from log messages, we created a method for mining and refining log format continuously in realtime. Our experiments shows that our method can identify the formats of real system logs without prior knowledge.