Application of Blockchain for DATA TRUST in ENTERPRISE DATA ENGINEERING
Businesses, Regulatory compliance (e.g., GDPR) often need a Data Lake / Data Foundation / System-of-Engagement to have Comprehensive Data Trust pillars (audit, provenance, Lineage, Verifiability). But, the Organization (a large FinTech Org) wants to have Future-gen architecture which can provide
enablement of the Comprehensive Data Trust pillars (audit, provenance, Lineage, Verifiability)@port-of-entry of System-of-Engagement
- auditing capability to keep track of critical data
- Data Provenance
- complete history of assets / Data Lineage
- Verifiability : a verification mechanism to verify integrity of past records
Post enablement of DATA TRUST pillars @port-of-entry, it should be able to trigger serverless-processing or serverless-workflows for
- caching the history for querying in Amazon Elasticsearch,
- transforming and loading the data into an Amazon Redshift cluster, and
- also storing the data in an Amazon S3 data lake.
for critical events/ datasets originating from Streaming sources / Batch sources
- for critical events originating from Streaming sources, such as "*credit* and *debit transactions* across bank accounts", and also
- for critical datasets originating from Batch sources, such as "*history of assets* for Loan lending to Orgs/customers like vehicle maintenance records /or/ Lending application records".
Solution Architecture using AWS QLDB
Architecture Solution using QLDB for above use case application:
Application of Blockchain for CIRCULAR SUPPLY CHAIN with IoT
Business Techo Problem statement
CIRCULAR SUPPLY CHAIN with IoT
Continuous Monitoring & Notification for SAFE ENVIRONMENTAL CONDITION for raw crop produce SHIPMENTs in TRANSPORTATION
right credit attribution to the Farmers
Enabling SMOOTH ADOPTION of blockchain (HL FABRIC)
how to right size the load for blockchain?
How to cost optimize the adoption of blockchain?
how to manage, deploy, auto-scale, descale container Blockchain network (HL FABRIC)?
how to monitor the nodes used for blockchain network cluster?
how to secure member's keys created for each member for accessing blockchain network? Possibly securing/fortifying keys in hardware using AWSCloud HSM.
Please Note The fundamental technology of blockchain is cryptography. Cryptography works on KEYS. Hence, Keys become very important part of the solution.
Solution Architecture using Hyperledger FABRIC DOCKER AWS EKS IOT HSM
Architecture Solution using AWS Managed Blockchain for above use case application:
Solution Architecture
Reference
[1]https://www.youtube.com/watch?v=gAxK6zYrfxI
[2] https://www.youtube.com/watch?v=mI6-qRagn_Q
[3] https://www.youtube.com/watch?v=5CwIt-Alqhg
[4] https://www.youtube.com/watch?v=WAIOBeQA2QQ
Thank you for joining! Stay connected with the latest updates and insights by visiting my website www.DeepHiveMind.com. Don't forget to follow me on social media for more tech tips and discussions. Let's continue exploring the exciting world of technology together! #TechTalks #StayConnected
