Apache Kafka is a distributed event store and stream-processing platform that is used for high-performance data pipelines, streaming analytics, and data integration at scale. It is an open-source system developed by the Apache Software Foundation and was originally created to handle real-time data feeds at LinkedIn in 2011. Kafka is written in Java and Scala and aims to provide a unified, high-throughput, low-latency platform for handling real-time data feeds.
Kafka can connect to external systems via Kafka Connect, and provides the Kafka Streams libraries for stream processing applications. Kafkas out-of-the-box Connect interface integrates with hundreds of event sources and event sinks including Postgres, JMS, Elasticsearch, AWS S3, and more. Kafka is a highly scalable, fault-tolerant distributed system, allowing it to be deployed for data integration at scale.
Kafka is used by thousands of companies for high-performance data pipelines, streaming analytics, and data integration at scale. More than 80% of all Fortune 100 companies trust and use Kafka. Kafka is one of the five most active projects of the Apache Software Foundation, with hundreds of meetups around the world.
