Jul
20

The Big Data era is ending ; welcome to the one of Fast Data

No single company is now immune from what we call Uberisation. A start-up or a company coming from a completely different sector of activity can, at any point, call into question situations that seemed to be have been established for decades. More agile, relieved of the burdens imposed on existing stakeholders, these competitors are taking advantage of historic players’ inertia in order to destabilise them. Just like the new entrants, the latter must demonstrate agility and opportunism. Data figures prominently among the arms at their disposal, with Big Data and also a new discipline that is beginning to gain power: Fast Data.

All the big French companies have now begun their digital transformation. They are reinforcing the role of data in their strategy and in the lifecycle of their products. Certain companies have created Big Data cellules and are hiring expensive Data Scientists. Some have already put Data Lakes in place in order to centralise their data and support their initial projects. The latest edition of the Big Data Paris show clearly demonstrated that many French companies are now committed to the Big Data revolution. This show also saw companies’ increasingly clear interest in what is known as Fast Data.

Big Data and its flagship technology, Hadoop, gave companies the means to analyse huge volumes of data. Now, the companies that are most advanced in their Big Data strategy would really like to undertake these analyses not in “batches” — i.e. staggered in time — but in real time or in much shorter timeframes. The term “Fast Data” is not totally new, but the technology has matured and numerous solutions are available on the market, both from commercial software publishers and the Open Source ecosystem.

The Big Data era is ending ; welcome to the one of Fast Data

Velocity already featured in the 3V that defined Big Data at its origins. It has now become a reality with the new-generation solutions. (Source: Teradata)

 

A solution to react to the client in real time

 

The potential applications of Fast Data are innumerable. Online advertising, the stock market and IT security are some of the sectors that have already adopted the approach, but there are many others. With Fast Data, you no longer need to wait 24 hours for a response to a loan application as it lets the bank carry out the risk calculation during the meeting with the financial advisor. Similarly, a telecoms operator is going to be able to reduce their churn rate (subscription cancellations) by reacting as soon as they see a client’s first signs of potential ‘infidelity’. For sales people, the move to Fast Data will represent a huge step forward in the way in which they are able to interact with their leads or existing clients. Big Data allowed them to calculate client scoring and thus determine the products for which this presented the greatest appetite. By moving into the Fast Data dimension, the software will be able to calculate — in real time — the “Next Best Action” to undertake to make a sale. It’s a precious calculation for displaying the right offer on the e-Commerce website or on the salesperson’s screen when they’re talking to their contact. Making the right offer at the right time, proposing a discount rate that will win over the lead; such is the promise of Fast Data.

 

The technique rises to the real-time challenge

 

With solutions that analyse data “in-memory” such as Spark, VoltDB, HANA from SAP or GoldenGate from Oracle, it has become possible to process or analyse “on the fly”, that’s to say at the very moment the data is generated by a sensor, a connected object or a client interaction. This real-time processing enables what specialists call data streaming.

Nevertheless, we should not consider Fast Data as Big Data’s gravedigger. Many experts agree that Hadoop will not totally disappear to the benefit of these new-generation solutions. We are starting to see the appearance of architectures with a two-speed information system; these have classic Big Data on one side, for storing and processing big volumes of data at night or when programmed to do so, and Fast Data on the other side, for information that necessitates immediate processing, even if this means algorithms that are less efficient and less precise but much faster.

In an article published by the Decideo website, Cédric Despres, head of Big Data services at Ysance puts the proportion of company data that require fast processing at 20%. Considering that Fast Data implementation is twice as expensive as Big Data processing, CDO are going to have to learn to handle their accelerator pedal with skill in order to best exploit these fast but expensive resources.

 

The Big Data era is ending ; welcome to the one of Fast Data

Architects are starting to design two-speed Big Data architectures depending on whether it’s necessary to process data in real time or simply store it for later. (Source: VoltDB)

 

Decision-makers are also demanding a real-time perspective of the company’s activity

 

In addition to the handling of these problems of sales assistance, scoring or the processing of data from connected objects, Fast Data is going to have a much wider scope for companies than just the optimisation of processes or the creation of new digital services. In the Capgemini/EMC joint study “Big & Fast Data: The Rise of Insight-Driven Business”, the decision-makers interviewed emphasised en masse their need to have information in real time at the moment they need to make their decisions. The “Next Best Action” concept does not apply only to the sales person who has to make a recommendation to their client; it does not apply only to tactical decisions. The decision-makers also need to leave behind the dashboard approach recapitulating past activity history. They need fresh data and predictive information in order to make their strategic decisions. It is only in this manner that they will be able to acquire the agility that historical players are lacking so much in the face of the risk of uberisation.

The Big Data era is ending ; welcome to the one of Fast Data

3 in 4 decision-makers now require real-time data to support their decision making (Source: Capgemini/EMC)

 

DO YOU KNOW SPARKLANE FOR MARKETING ?

Hypertarget your marketing campaigns & achieve your demand generation objectives more easily!

Discover the offer sparklane

 

Alain Clapaud
About the Author
Alain Clapaud is a freelance journalist, focused on digital marketing and new technologies topics