Logstash is a great tool for managing events and logs. You can use it to collect logs, parse them, and store them for later use (like, for searching). It is fully free and fully open source. The license is Apache 2.0, meaning you are pretty much free to use it however you want in whatever way.
But for now, Logstash doesn't support output log to Hadoop Hdfs, it's a really big missing fearture, so you if you are Logstash's fan, you have 3 way to put your log to Hadoop Hdfs from Logstash
- Hadoop HttpFS (or WebHDFS)
- Forward you log message to Hadoop HttpFs with Logstash output (http, exec, ...)
- Hadoop NFS (Network file system)
- Store you log in a directory that mounted to Hadoop Hdfs, ues with Logstash output file
- Third party tools
- Forward you log message to other tools that supported to streaming log to Hadoop Hdfs, use with Logstash output (tcp, http, exec, ..)