Creating order out of chaos – in real time

Yes, you read that right. It is now possible to create order out of the chaos in real time. PayPal has developed a unique approach for analyzing their chaotic real-time server data and producing organized, intelligent results.  I am just coming out of a HPC user’s forum presentation where PayPal announced this breakthrough approach using HP’s Moonshot server powered by TI processors.   We at TI are happy to play a part in the creation of this novel application brought about by our technology in close collaboration with PayPal and HP. Sounds like too much to digest? Read on.

First a simple way to look at the problem – we are swimming in data of all sorts. Some of this data can be analyzed long after it is generated and produce useful insights, while some data is time critical and needs to be analyzed quickly to produce useful information. The breakthrough solution presented today by PayPal is a solution to the latter problem, real-time data analysis. 

The challenge confronted here is a data set that must be consumed as soon as it is created in order to yield maximum value. In PayPal’s example, they are dealing with a data set that must be analyzed as soon as it is generated and if this can be done, it could help solve problems in areas such as fraud, risk assessment, forecasting, and business analytic. Just imagine the value of on the fly, real time analysis to businesses when you can identify a pattern and act on it instantaneously. The saying “time is money” is as true as ever and acting quickly on trends exhibited by big data analysis is an untapped resource with tremendous potential 

PayPal’s novel approach is to convert events represented in a plain text format into a numeric format which can be analyzed in real-time using mathematical techniques with hardware specifically designed to operate on such numeric data. The first instantiation of this approach uses ProLiant m800 cartridges powered by TI’s 66AK2Hx processor that integrates eight c66x DSP cores and four ARM® Cortex®-A15 cores using TI’s KeyStone™ II architecture running in HP’s Moonshot server platform. The 66AK2Hx SoC is ideally suited for this type of application as it possesses some unique advantages to aid in real time processing. These advantages are:

1)      C66x DSP cores that have great signal processing performance as well as very low latency response times and can receive, process, and return packet data very quickly.

2)      An integrated I/O fabric that moves data quickly and with low latency. The C66AK2H IO fabric utilizes sRIO that has 10x lower hop to hop latency than Ethernet I/O.

3)      Additional KeyStone II architecture elements such as the Multicore Navigator and TeraNet which further enable low latency data movement within and across devices.

Each ProLiant m800 cartridge contains four 66AK2Hx devices each of which can be considered a distinct server node. Each Moonshot chassis can hold 45 m800 cartridges thus allowing 180 C66AK2H server nodes to work on incoming data streams in real time. The 66AK2Hx processor’s integrated sRIO seamlessly integrates with Moonshot’s programmable I/O fabric enhancing the ability of the system to use the large number of distributed nodes for real time processing of large data sets.

We are excited to partner with PayPal and HP to give birth to this approach. You can  watch this video where PayPal’s Principle Architect Ryan Quick describes the types of real time data processing problems PayPal is solving and discusses the reasons behind working with TI and HP. 

There is a lot of excitement in store so stay tuned over the coming weeks as we continue to battle the forces of entropy in real time.