How Apache Spark, Scala, and Functional Programming Made Hard Problems Easy at Barclays | Cloudera Engineering Blog

At Barclays, our team recently built an application called Insights Engine to execute an arbitrary number N of near-arbitrary SQL-like queries and execute them in a way that can scale with increasing N. The queries were non-trivial, each constituting 200-300 lines of SQL, and they were running over a large dataset for hours as Apache Hive scripts. Yet, we need to execute 50 queries in less than an hour for our use case.

Source: How Apache Spark, Scala, and Functional Programming Made Hard Problems Easy at Barclays | Cloudera Engineering Blog

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.