The headless knowledge structure is an natural emergence of the separation of knowledge storage, administration, optimization, and entry from the companies that write, course of, and question it. With this structure, you may handle your knowledge from a single logical location, together with permissions, schema evolution, and desk optimizations. And, to prime it off, it makes regulatory compliance lots easier, as a result of your knowledge resides in a single place, as a substitute of being copied round to each processing engine that wants it.
We name it a “headless” knowledge structure due to its similarity to a “headless server,” the place you need to use your individual monitor and keyboard to log in. If you wish to course of or question your knowledge in a headless knowledge structure, you’ll have to deliver your individual processing or querying “head” and plug it into the info — for instance, Trino, Presto, Apache Flink, or Apache Spark.
A headless knowledge structure can embody a number of knowledge codecs, with knowledge streams and tables as the 2 most typical. Streams present low-latency entry to incremental knowledge, whereas tables present environment friendly bulk-query capabilities. Collectively, they provide the flexibility to decide on the format that’s best suited on your use instances, whether or not it’s operational, analytical, or someplace in between.
First, let’s check out streaming within the headless knowledge structure.