The Apache Spark framework is a popular add on to Hadoop for handling big data, particularly for building machine learning algorithms.
Until now though it's been hard to effectively monitor Spark performance. That's about to change as Brooklyn-based performance management company Sematext is launching its SPM for Spark.
The tool monitors key metrics for all Spark components -- master, workers, driver, and executor. It also includes alerting, anomaly detection, custom dashboards, log correlation and event graphing. SPM for Spark can be installed on premises or used from the cloud.
"Spark usage has been going through the roof," says Otis Gospodnetic, Sematext’s founder and CEO. "And engineers and DevOps folks handling Spark have not had a good monitoring tool at their disposal. By releasing the first Spark monitoring product to market with SPM, we have just filled a big hole in the Spark ecosystem".
SPM works seamlessly with centralized logging, log management and analytics solution Logsene, to provide a single pane of glass for performance monitoring, centralized log management, alerts, anomalies, custom events, and custom key performance indicators.
John Tripier, Alliance and Ecosystems lead at Databricks says, "One of the critical factors in the success of Spark has been the large developer community contributing to it, and the growing number of applications using Spark. We're very excited to have Sematext join this community and contribute their expertise with a comprehensive monitoring solution like SPM".
You can find out more about SPM for Spark's features with a downloadable datasheet or sign up for a free trial on the Sematext website.
Image Credit: alphaspirit / Shutterstock