Modern enterprises run on data, but moving this data around and giving it the right shape so it can be used in specific applications remains a complex undertaking. Definity, which is launching out of stealth Wednesday and announcing a $4.5 million seed funding round, wants to give these companies the tools to observe, fix and optimize their data pipelines.
The twist here is that unlike many of its competitors, it doesn’t only look at the data once it’s transformed and deposited somewhere — at which point is becomes hard to troubleshoot when things go awry — but while the data is still in motion.
The startup supports a wide variety of environments but focuses on Apache Spark-based applications (on-prem or top of managed services like Google’s Dataproc, AWS EMR or Databricks, for example), which is maybe no surprise given that all of the co-founders have a lot of experience with open source data-processing engines. CTO Ohad Raviv is a Spark contributor and the former big-data tech lead at PayPal. Roy Daniel, the company’s CEO, previously worked at FIS, while VP of R&D Tom Bar-Yacov was formerly a data engineering manager at PayPal.
In an interview, Daniel stressed that the company focuses on the data transformation plane on top of a data lake or warehouse, not the data ingestion part of the pipeline. Some of the issues the team experienced during its time working for these large enterprises include data quality problems brought about by inconsistent data, schema changes and stale data. “Those are data quality issues that propagate downstream,” he said. “They affect the business, whether it’s models that are working on top of bade data now, or dashboards or BI that is broken and all of a sudden, the CFO is like, what’s going on?”