lucene-log4j solves a recurrent problem that production support team face whenever a live incident happens: filtering production log statements to match a session/transaction/user ID. via lucene-log4j – Specialized Log4j file rolling appender which indexes log with Lucene – Google Project Hosting.
Disco is a lightweight, open-source framework for distributed computing based on the MapReduce paradigm. Disco is powerful and easy to use, thanks to Python. Disco distributes and replicates your data, and schedules your jobs efficiently. Disco even includes the tools you need to index billions of data points and query them in real-time. Disco was …
Harvesters watch log files for changes, send new log messages to the server, which broadcasts to web clients. Log messages are tagged with stream, node, and log level information based on user configuration. via Log.io – Real-time log monitoring in your browser.
You have logs. Billions of lines of data. You shipped, dated it, parsed it and stored it. And you used Logstash to do it. Now what do you do with it? Now you make sense of it. See everything you want, and nothing you don’t. Chart it and rank it and play with the numbers. …
OpenTSDB is a distributed, scalable Time Series Database (TSDB) written on top of HBase. OpenTSDB was written to address a common need: store, index and serve metrics collected from computer systems (network gear, operating systems, applications) at a large scale, and make this data easily accessible and graphable. via OpenTSDB – A Distributed, Scalable Monitoring …
“Fluentd” is an open-source tool to collect events and logs. 150+ plugins instantly enables you to store the massive data for Log Search (like ElasticSearch), Big Data Analytics, and Archiving (MongoDB, S3, Hadoop). via Fluentd: Open Source Log Management.
Flume is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data. It has a simple and flexible architecture based on streaming data flows. It is robust and fault tolerant with tunable reliability mechanisms and many failover and recovery mechanisms. It uses a simple extensible data model that …
logstash is a tool for managing events and logs. You can use it to collect logs, parse them, and store them for later use (like, for searching). Speaking of searching, logstash comes with a web interface for searching and drilling into all of your logs. via logstash – open source log management.
OSSIM™ and other cool projects. Play, share, and enjoy.
Our open source projects focus on collaborative threat intelligence and incident response. This includes OSSIM, the world’s most widely used open source SIEM product.
via OSSIM: Open Source SIEM & Open Threat Exchange Projects.
Splunk is a great tool for searching logs, but its high cost makes it prohibitive for many teams. In this article, we present a free and open source alternative to Splunk by combining three open source projects: Elasticsearch, Kibana, and Fluentd.