The problem we saw
Big Data frameworks like Hadoop were not designed for the query intensive needs of BI tools. These frameworks excel at ingesting large amounts of data but are high-latency systems with query processing. Typically when a BI dashboard refreshes, it issues multiple simultaneous queries. The database then performs a full scan of the entire data set for each query. This is far too slow for the interactive-speed demands of business users.
How we solved it
We built a query acceleration engine from nuts to bolts designed for the unique needs of BI on Big Data. Our engine is a transparent middle tier between a BI tool and a Big Data platform. We combined a full-index architecture with auto cubes that work in tandem to cover all types of BI queries. Jethro serves 1000s of concurrent users and scales to billions of rows.