bury the hammer

• about
• projects
• speaking
• blog
• cv
• contact

Testing a Spark Application

Data engineering / analysis applications are just like any other software. You need to test them. But how?

I wrote something over at the OpenCredo blog about Testing Spark.

It covers a few different technologies, mostly focused around data engineering (Cassandra, Spark, Scala). The thrust of it is that you can test any kind of application. Particularly data processing applications, which traditionally aren’t built by software engineers, and don’t tend to be tested.