Jump to content

Fluentd

From Wikipedia, the free encyclopedia
Fluentd
Developer(s)Treasure Data
Initial release10 October 2011;
13 years ago (2011-10-10)
Stable release
1.16.6 / August 20, 2024; 3 months ago (2024-08-20)[1]
Repository
Written inC, Ruby
Operating systemLinux (Amazon Linux, CentOS, RHEL), macOS (10.9 and above), Ruby, Windows (7 and above)
TypeLogging tool
LicenseApache 2.0
Websitewww.fluentd.org Edit this at Wikidata

Fluentd is a cross-platform open-source data collection software project originally developed at Treasure Data. It is written primarily in the C programming language with a thin-Ruby wrapper that gives users flexibility.[2]

Overview

[edit]

Fluentd was positioned for "big data," semi- or un-structured data sets. It analyzes event logs, application logs, and clickstreams.[3] According to Suonsyrjä and Mikkonen, the "core idea of Fluentd is to be the unifying layer between different types of log inputs and outputs.",[4] Fluentd is available on Linux, macOS, and Windows.[5]

History

[edit]

Fluentd was created by Sadayuki Furuhashi as a project of the Mountain View-based firm Treasure Data. Written primarily in Ruby, its source code was released as open-source software in October 2011.[6][7] The company announced $5 million of funding in 2013.[8] Treasure Data was then sold to Arm Ltd. in 2018.[9]

Users

[edit]

Fluentd was one of the data collection tools recommended by Amazon Web Services in 2013, when it was said to be similar to Apache Flume or Scribe.[10] Google Cloud Platform's BigQuery recommends Fluentd as the default real-time data-ingestion tool, and uses Google's customized version of Fluentd, called google-fluentd, as a default logging agent.[11][12]

Fluent Bit

[edit]

Fluent Bit is a log processor and log forwarder which is being developed as a CNCF sub-project under the umbrella of Fluentd project.[13] Fluentd is written in C and Ruby and built as a Ruby gem so it consumes some amount of memory resources. On the other hand, since Fluent Bit is written only in C and has no dependencies, the consumed memory usage much decreased compared to Fluentd which makes it easy to run on the embedded Linux and container environment.[14]

References

[edit]
  1. ^ "Releases - fluent/fluentd" – via GitHub.
  2. ^ https://docs.fluentd.org/quickstart [bare URL]
  3. ^ Pasupuleti, Pradeep and Purra, Beulah Salome (2015). Data Lake Development with Big Data. pp. 44–45; 48. Packt. ISBN 1785881663
  4. ^ Suonsyrjä, Sampo and Mikkonen, Tommi "Designing an Unobtrusive Analytics Framework for Monitoring Java Applications", pp. 170–173 in Software Measurement. Springer. ISBN 3319242857
  5. ^ Fluentd.org. "Download Fluentd". Retrieved 10 March 2016.
  6. ^ Mayer, Chris (30 October 2013). "Treasure Data: Breaking down the Hadoop barrier". JAXenter
  7. ^ Fluentd.org. "What is Fluentd?". Retrieved 10 March 2016.
  8. ^ Derrick Harris (July 23, 2013). "Treasure Data raises $5M, fuses Hadoop and data warehouse in Amazon's cloud". GigaOm.
  9. ^ "Arm unit Treasure Data to seek buyer or IPO before Nvidia sale". Nikkei Asia. November 19, 2020. Retrieved August 2, 2021.
  10. ^ Parviz Deyhim (August 2013). "Best Practices for Amazon EMR" (PDF). Amazon Web Services. p. 12. Archived from the original (PDF) on 2016-03-26. Retrieved March 24, 2017.
  11. ^ Google Cloud Platform (2016). "Real-time logs analysis using Fluentd and BigQuery". Retrieved 10 March 2016.
  12. ^ Google Cloud Platform (2016). "The Logging Agent". Retrieved 10 March 2016.
  13. ^ "Fluent Bit". fluentbit.io. Retrieved 2021-12-05.
  14. ^ "Fluentd & Fluent Bit - Fluent Bit: Official Manual". Retrieved 2021-12-05.

Further reading

[edit]
[edit]
Retrieved from "https://en.wikipedia.org/w/index.php?title=Fluentd&oldid=1255932796"