Amazon FinSpace managed Spark clusters now support Apache Spark 3.1.2. Apache Spark 3 has query optimization features like dynamic partition pruning to optimize joins such as joining a large fact table of trades with a smaller dimension table of execution centers. It also includes changes to be more compatible with the ANSI SQL standard and features 30 new built-in functions. FinSpace Spark clusters make it simple for analysts to launch, connect, resize and terminate clusters. FinSpace Spark clusters are available in five sizes, so you can select a configuration that is suitable for your workload, and they include a time series analytics library for preparing and analyzing historical financial time series data at petabtyte scale.

Source: AWS