Massively Parallel Processing - Not So Strange After All

One of my favorite action scenes is from part one of a recent and well-known blockbuster movie highlighting an epic clash in outer space between a power-hungry, finger-snapping supervillain and several of Earth's uniquely gifted superheroes. Strangely enough, the blue body-building nemesis would not succumb to spider webs, fusion-powered body armor, or magic tricks. However, once the mystical sorcerer realized he needed more power and resources to defeat his archenemy, he cast a spell that created a massive number of clones. Their coordinated strike was more than the evil titan could endure.

At this point, you are probably questioning the relevance of an intergalactic battle between good and evil. For a moment, please consider the following:

Data is generated, gathered, and consumed at mind-boggling rates in the modern world. Not only do companies have the burden of continuously collecting and storing massive amounts of data, but they must also employ a strategy that efficiently processes these data sets. VMware Data Solutions provides a modernized data analytics and operations platform that addresses this challenge of data proliferation. A key component of VMware Data Solutions, Greenplum, utilizes Massively Parallel Processing, an industry-standard technique used in big data analysis.

What is Massively Parallel Processing?

In technical terms, Massively Parallel Processing is a system in which multiple processors have a single task divided between them. Each processor functions independently yet simultaneously in executing its part of the task. The combined output from each of the processors constitutes the overall result. Massively Parallel Processing is the "divide and conquer" methodology.

Referring to the timeless war described at the beginning, the initial effort by each superhero was a solitary act that either did not yield the desired outcome or could not be completed in the required time. As illustrated below, this would be analogous to a single processor attempting to execute a complex task.

Conversely, if you take that same complex task and utilize Massively Parallel Processing, you can yield desirable results in a fraction of the time.

Strangely enough, it doesn't take a doctor to understand that a complex task can rarely be completed quickly by a singular effort. However, many coordinated efforts working concurrently on the same task typically yield optimal results. The path to becoming a master of the analytic arts is through the effective use of Massively Parallel Processing. Make that your End Game!