Whereas Iceberg solves this drawback, open requirements are wanted in different areas as properly. We’re now seeing a brand new battlefield rising within the space of knowledge catalogs, which play a crucial function in a multi-engine structure. Catalogs make operations on tables dependable by supporting atomic transactions. Because of this information engineers and the pipelines they construct can modify tables concurrently, and queries on these tables produce correct outcomes. To perform this, all Iceberg-table learn and write operations, even from completely different engines, are routed by way of a catalog.
SaaS suppliers and hyperscalers can use the catalog as a option to create buyer stickiness, however enterprises are getting clever to this. They perceive that, simply as Iceberg supplies a typical format for tables, an open catalog customary will allow them to select one of the best software for the job and maximize the worth of their information.
Open requirements are good for enterprise, good for patrons, and good for the broader ecosystem. Enterprises have complicated information architectures, and open requirements enable them to make use of information throughout these platforms with out incurring extra price and governance challenges. Open requirements additionally promote innovation, as a result of they drive firms to compete on an implementation and permit clients to decide on between them.