Blockchain for DATA TRUST in ENTERPRISE DATA LAKE ENGINEERING world

Blockchain for DATA TRUST in ENTERPRISE DATA LAKE ENGINEERING world

Businesses & Regulatory compliance (e.g., GDPR) often need a system-of-record application with

  • (a) Auditing capability to keep track of critical data,
  • (b) establish Data provenance,
  • (c) Verifiability in order to evolve it to the maturity of TRUSTED SOR or Systems-of-engagement (DWH/ Data lake).

We all know, building TRUST (audit functionality & Verifiability) with Next Gen Distributed Relational databases/NOSQL Distributed Databases

  • requires custom development,
  • is time-consuming,
  • prone to human error,
  • as in case of relational databases/ NOSQL DBs are not inherently immutable, any unintended changes to the data are hard to track and verify. verifiability is tougher implementation ask.

BLOCKCHAIN NETWORK IS THE SILVER BULLET FOR SUCH SCENARIO IN DATA ENGINEERING AND DATA LAKE TRUST..

"Blockchain" vis-a-vis "Today’s Systems-of-Record": {A VERY IMP PERSPECTIVE for ENTERPRISE DATA PROCESSING & DATA LAKE ARCHITECTS}

"Today’s Systems-of-Record" is a
    - *'Distributed Database/ Ledger Management system'* 
    - wherein all the members of a **business network** transact with each other, 
    - but they maintain **separate records of their transactions**.


-Whereas "Blockchain" is the Future network which answers aforemention inherent challenges with traditional SOR 
    - with *'Distributed' & 'Decentralized' Database/ Ledger Management system* 
    - wherein auditing, record keeping and **Data provenance** comes baked in

SOR vs Blockchain from Business Network standpoint:

  • Today’s Systems-of-Record" (Distributed Database or Ledger)
  • Private programs & private Ledger for Every participant

image.png

  • "Blockchain" (The Future network of Distributed and Decentralized Database or Ledger)
  • Shared programs and shared ledgers for Every participant

image.png

Did you find this article valuable?

Support Harsh Vardhan by becoming a sponsor. Any amount is appreciated!