When Big Data meets Blockchain

Sep 12, 2019

Big Data has been the driving force behind the evolution we've been following in Data Science, Machine Learning and IA over the last decade. It's hard to imagine a company that is not yet adopting or thinking of adopting a Big Data solution to leverage business and generate insights. Now another technology promises to revolutionize the way we conduct financial transactions, Blockchain, and this can create even more opportunities for those who work with Big Data solutions. Let's investigate what these opportunities are in this post.

What is Blockchain?

Blockchain is a distributed database system that acts as an "open book" (also called ledger) for storing and managing transactions. Each record in the database is called a block and contains details such as the timestamp of the transaction as well as a link to the previous block. This makes it impossible for anyone to change information about records retrospectively. In addition, due to the fact that the same transaction is registered in multiple distributed database systems, the technology is secure by design. Therefore, Blockchain is immutable - the information remains in the same state as long as the network exists.

Within the Bitcoin virtual currency, a Blockchain is the data structure that represents a financial accounting entry or a record of a transaction. Each transaction is digitally signed with the purpose of guaranteeing its authenticity and ensuring that no one adulterates it, so that the record itself and the transactions within it are considered of high integrity.

True magic comes, however, from the fact that these digital log entries are distributed between a deployment or infrastructure. These additional nodes and tiers in the infrastructure serve the purpose of providing consensus on the state of a transaction at any time since all such nodes and tiers have copies of the authenticated records distributed between them.

Blockchain is a technology that aims at decentralization as a security measure. They are distributed and shared records and data bases that have the function of creating a global index for all the transactions that occur in a given market. It functions as a reason book, only in a public, shared and universal way, which creates consensus and trust in direct communication between two parties, that is, without the intermediary of third parties. It is constantly growing as new complete blocks are added to it by a new set of records. Blocks are added to the blockchain in a linear and chronological way. Each node - any computer connected to this network has the task of validating and passing on transactions - gets a copy of the blockchain after joining the network. The blockchain has complete address and balance information directly from the genesis block to the most recently completed block.

The blockchain was first defined in the original bitcoin source code. Therefore, they are closely connected with regard to the emergence of both.

The original definition was created in 2008 with the publication of the article "Bitcoin: The Peer-to-Peer Electronic Cash System" published by Satoshi Nakamoto (whose real identity remains open although there is some speculation and pronouncements about it). In 2009 the code was released as open source.

Thus, in 2009 begins the bitcoin network when Satoshi Nakamoto mined the first bitcoins. Satoshi Nakamoto disappeared in 2011 - that is, he stopped participating in forums, articles and code contributions about bitcoin. But even with the absence of Satoshi Nakamoto, bitcoin continued to be developed, with the community's overall effort working to solve various problems in the code - including, for example, a technical flaw in 2013 that caused a bifurcation in the blockchain.

Being the technological base of the crypto-coins, the blockchain has received the interest of banks, companies and governmental organizations. Since then, modifications have been made from the original version and new applications have been tied to the blockchain.

Blockchain and Big Data

When we talk about blockchain in the context of Bitcoin, the connection with Big Data seems a bit tenuous. What if, instead of Bitcoin, the block chain was a reason book for other financial transactions? Or commercial contracts? Or stock trading?

The financial services industry is starting to look seriously at blockchain technology. Oliver Bussmann, CIO at UBS says that blockchain technology can "reduce transaction processing time from days to minutes."

Imagine block strings of this magnitude. Data Lakes blocks containing the complete history of all financial transactions, all available for review. Blockchain provides the integrity of the ledger, but not for analysis. This is where Big Data and the analysis tools that come with it come into play.

Opportunities for Big Data Analytics

Recently, a consortium of 47 Japanese banks teamed up with a blockchain Startup called Ripple to facilitate money transfers between bank accounts using blockchain. The main reason behind the move is to perform real-time transfers at a significantly low cost. One of the reasons why traditional real-time transfers are costly is by potential risk factors. Double-spend (which is a form of transaction failure where the same security token is used twice) is a problem with real-time transfers. With blockchains, this risk is largely avoided. Big Data Analytics would make it possible to identify patterns in consumer spending and identify risky transactions much faster than can be done today. This reduces the cost of real-time transactions.

In companies outside the financial sector, the main impetus for the adoption of Blockchain technologies has been security. In the healthcare, retail, and public administration industries, establishments began experimenting with the blockchain to prevent intrusions and data leaks. In healthcare, technology such as blockchain can ensure that multiple "signatures" are requested at all levels of data access. This can help prevent a repeat of events like the 2015 attack that led to the theft of more than 100 million patient records.



Contact Us

  • Nieuwe Kerkstraat 13 1018 DW, Amsterdam, Netherlands