Hyperledger Fabric in Production (presented in Hyperledger Member Summit 2023 APAC)

Transcript

1. © Hitachi, Ltd. 2023. All rights reserved. Hyperledger Fabric in Production Research and Development Group, Hitachi, Ltd. Tatsuya Sato Hyperledger Member Summit 2023 APAC
2. © Hitachi, Ltd. 2023. All rights reserved. 1 About me • Tatsuya Sato from Hitachi in Japan – Chief Researcher • Blockchain Technology, System Operations – Contributor to Hyperledger Foundation • Hyperledger Japan Chapter Leads • Maintainer of fabric-samples since 2023/05 – Repository of sample collection for Fabric • Maintainer of Hyperledger Fabric Japanese Documentation Working Group • Labs Owner of OpsSC (Operations Smart Contract) for Hyperledger Fabric – Decentralized system operation tool for Hyperledger Fabric-based systems
3. © Hitachi, Ltd. 2023. All rights reserved. 2 1. Case Study Topic: What Hitachi Learned from Practical Use of Fabric 2. Future Topic: Towards Deploying a Fully-Decentralized Fabric-based System
4. © Hitachi, Ltd. 2023. All rights reserved. 3 1. Case Study Topic: What Hitachi Learned from Practical Use of Fabric 2. Future Topic: Towards Deploying a Fully-Decentralized Fabric-based System
5. © Hitachi, Ltd. 2023. All rights reserved. 4 1-1. Hitachi’s Blockchain businesses and activities Managed Service Managed Service Hyperledger Fabric Hyperledger Fabric Support Service Support Service Application Service for Blockchain Hitachi Blockchain Service for Hyperledger Fabric Finance Finance Manufacturing Manufacturing Public Sectors Public Sectors Business Solutions Application Framework Application Framework Business Template Business Template Development Framework Development Framework (R&D) Technology Development, Upstream Activities (Hitachi Solutions) ConsenSys Quorum Support Hyperledger Besu Hyperledger Besu Enterprise plugins Enterprise plugins Support Support Common APIs for blockchain-based systems Asset Management API Asset Management API Record Management API Record Management API • Providing Blockchain as a Service (BaaS) primarily using Hyperledger Fabric • Developing blockchain-based systems for client companies (e.g., Financial Institutions, Manufactures, Public Sectors) – Typically, these systems are implemented on top of the above BaaS we provide • Contributing to Hyperledger through activities like submitting patches to Fabric etc. …
6. © Hitachi, Ltd. 2023. All rights reserved. 5 • Hitachi and MUFG Bank Start a PoC on blockchain technology for check digitization in Singapore (2016/8/22 News release) • Hitachi and Mizuho Start a PoC on the integrated management of money flow, sales flow, and logistics, and the advancement of supply chain finance using blockchain technology (2020/12/28 News release) • Zenrin and Hitachi are cooperating with a feasibility study of tourism-oriented MaaS in Nagasaki city (2021/5/20 News release) • The Retail and Distribution Subcommittee of the Digital Currency Forum, in which Hitachi participates, has started a proof of concept to utilize digital currency for payment operations associated with transactions between retail and wholesale companies (2022/5/31 News release) • We have engaged with customers in various blockchain projects, encompassing both PoC and production stages. For example… 1-2. Blockchain-related Projects involving Hitachi • Natural resource traceability system using blockchain technology goes into production • Developed the "Hitachi Electronic Signature Service," which uses blockchain technology to enhance the authenticity of information and promote stamp-less systems, and has begun preliminary operations within Hitachi (2021/3/3 News release) • Developed a prototype of traceability management using “Hyperledger Fabric” to promote the use of blockchain technology in supply chain (2017/5/9 News release) • Hitachi, JPX, Nomura and BOOSTRY are cooperating with Hitachi toward its issuance of a “Digitally Tracked Green Bond” utilizing digital technologies such as IoT and a blockchain platform (2023/11/16 News release) (*) These are mostly news releases in Japanese translated into English by the presenter
7. © Hitachi, Ltd. 2023. All rights reserved. 6 1-3. Case study: Sustainable Finance Platform *1:https://www.hitachi.com/New/cnews/month/2023/11/231116.html *2:https://www.hitachi.co.jp/New/cnews/month/2023/10/1030a.html Main Features • To digitize the authentication and issuance of credits, we have built a platform that simplifies the collection, refinement, and application of target equipment. • Calculation of CO2 reduction amount based on operation data measured by IoT sensors, record to Blockchain and automatically data interaction to the certification body. Value • Simplified credit application and issuance • Ensuring credit transparency For the further spread and expansion of carbon credits, Hitachi supports the entire process from application to circulation of carbon credits and promotes the realization of a decarbonized society. Collect and refine facility operating data with MRV. Automate carbon credit application and issuance. MRV function for sustainable finance Sustainable Finance Platform (Hitachi) Invest Profit on credit sales Carbon Credit Markets Credit certification body ESG evaluation agency Operating data of facilities Companies Investors Application Data refinement Carbon Credit Carbon credit application support(*2) ESG reporting Support(*1) MRV: Monitoring, Reporting and Verification, ESG: Environment, Social and Governance Fabric will be used for the MRV function → Fabric is often used for data exchange and traceability management between multiple companies
8. © Hitachi, Ltd. 2023. All rights reserved. 7 1-4. Issues and insights in actual Fabric environments from development perspective • Issue 1 When we started using Fabric, the development of client applications and chaincodes was challenging and had a high barrier → Our solution We prepared a framework for common processes in client applications and business templates (e.g., a set of common chaincodes/APIs, reference implementations for each use case) for internal use, and partially offered these as part of our services – Examples of practical improvements in the application framework: • Dealing with app, chaincode, and peer errors on the client side is tough → Currently categorizing them with original error codes through a gritty approach of string matching against log messages • Issue 2 System and parameter design are complicated due to Fabric’s distributed nature – Timeout settings are dispersed across different locations, adding to the complexity → Our solution Identified the timeout setting parameters and tuned based on the relational equations among these parameters • It would be beneficial to have such practical design guides in the community's documents
9. © Hitachi, Ltd. 2023. All rights reserved. 8 1-4. Issues and insights in actual Fabric environments from operational perspective (1/2) • Issue 3 No portal/UI at the time of start of use, Fabric has complex operational procedures → Our solution Developed a portal, started offering it as a service, working to streamline our operations – Step 1 Improvement of basic operations (for each organization): Fabric uses a combination of CLI commands to build/operate Fabric network, so we developed a set of CLI-wrapped operations scripts for each operation scenario and gradually integrating them into our service portal – Step 2 Improvement of inter-organizational operations: We are starting to integrate OpsSC, a decentralized system operation tool for Fabric, into our service portal (e.g., Automating chaincode deployment across all organizations) (*)OpsSC again in a later slide
10. © Hitachi, Ltd. 2023. All rights reserved. 9 1-4. Issues and insights in actual Fabric environments from operational perspective (2/2) • Issue 4 Certificates expired in a production environment → manually renewed the certs – Lesson Learned Crucial to tailor parameters to fit actual operations instead of default settings • Example of effective setting Extend the expiration period from the default 1 year to 13 months. This adjustment enables a consistent update schedule every April, streamlining the renewal process • Issue 5 Needed to replace Hyperledger Explorer (used as a monitoring tool) – Reason 1. Explorer was moved to “End of Life” status (now rebooted as a Labs project) – Reason 2. A lot of periodic Query logs from Explorer interfere with Peer logs → Our solution Migrated from Explorer to Prometheus with Fabric Operations Service (introduced from v1.4)
11. © Hitachi, Ltd. 2023. All rights reserved. 10 1. Case Study Topic: What Hitachi Learned from Practical Use of Fabric 2. Future Topic: Towards Deploying a Fully-Decentralized Fabric-based System
12. © Hitachi, Ltd. 2023. All rights reserved. 11 • In Fabric, it is still difficult to deploy a fully-decentralized system →Undergoing functional enhancements to promote “decentralization” CA: Certificate Authority BFT: Byzantine Fault Tolerance CFT: Crash Fault Tolerance Rough requirements: There is NO SPOT (single point of trust) from the following 3 perspectives Identity TX execution system Management /Governance - Orderer (to create block) supports only CFT consensus algorithms (Raft) - Bad practice: centralized management by a single manager Assume organizations each prepare their own CA and issue their own identities with CA - Require proper endorsement policy settings BFT support planned in v3 (preview version was released) - Introduced new chaincode lifecycle model that eliminates SPOTs from v2.x Need to adjust config parameters offline ➡ OpsSC (Hyperledger Labs): On-chain end-to-end operational workflow Current status (in v2.5 LTS): Undergoing/near future: [1] Toward Fully-Decentralized System With Hyperledger Fabric, Hyperledger Global Forum, 2022. [2] Toward Fully-Decentralized System With Hyperledger Fabric, IEICE ComEX, 2023. 2-1. Activities toward fully-decentralization in Fabric
13. © Hitachi, Ltd. 2023. All rights reserved. 12 2-2. BFT Orderer Overview and Development Status • In the upcoming v3 of Fabric, a BFT Orderer is a key feature – Byzantine Fault Tolerance (BFT): A model that tolerates arbitrary node failures → This allows for coverage even in cases where there are malicious nodes • Development Status: Preview version (v3.0.0-preview) was released in 2023/09 – You can try samples to run a Fabric network with BFT Orderer in test-network of fabric-samples • Design principles: Prioritize latency, simplicity, and manageability rather than throughput • Algorithm: SmartBFT, a BFT consensus algorithm (similar to PBFT in v0.6) • Management: Can be configured in the almost same way as Raft Orderer [1] Y. Manevich, Understanding Byzantine Fault Tolerance in Hyperledger Fabric, Virtual Hyperledger meetup, 2023 Design principles (Adapted from [1]) Architecture (Adapted from [1]) BFT: Byzantine Fault Tolerance PBFT: Practical BFT
14. © Hitachi, Ltd. 2023. All rights reserved. 13 2-3. Toward Production Use of BFT Orderer • Development of BFT Orderer is progressing towards the v3.0 GA release in the community – Focusing primarily on performance improvements and test coverage improvements – This version is anticipated to be a version intended for production use • My opinion: Additionally, it would be beneficial to expand the materials on mastering the use of BFT Orderer – Enrichment of samples in system configurations where multiple organizations hold an Orderer node – Automation or scripting of operational procedures due to incorrect settings and/or procedures are critical
15. © Hitachi, Ltd. 2023. All rights reserved. 14 2-4. OpsSC for Hyperledger Fabric • OpsSC (Operations Smart Contract) enables inter-organizational system operations efficiently in a decentralized manner by defining operational workflows as smart contracts • For Fabric: Streamline end-to-end workflows for operating chaincodes or channels Org1 Org2 Org3 (1) Sharing/adjusting config parameters off-chain (2) Manual execution of commands by each org 1. Share/Adjust (Consistency required) Before (Typical BC network operation in Fabric v2.x) Peer Orderer Peer Orderer Peer Orderer Org1 Org2 Org3 2. Execute cmds. Consistency and/or coordination (e.g., Install CC->Approve CC ->Commit CC) Config parameters (e.g., CC source code, CC definition) 1. 2. 2. Org1 Org2 Peer Orderer OpsSC Agent OpsSC Chaincodes OpsSC API/Portal OpsSC API/Portal Org3 OpsSC API/Portal 1-1. Propose ops 1-2. Vote for proposal Peer Orderer OpsSC Agent Peer Orderer OpsSC Agent 2-1. Check the num of votes 2-3. Execute ops based on event 2-2. Issue event 2-3. 2-3. (1) Sharing/adjusting config parameters on-chain (OpsSC chaincodes) (2) Auto execution of commands by each org’s agent based on the proposal After (OpsSC for Fabric v2.x) Chaincode Operations https://github.com/hyperledger-labs/fabric-opssc Channel Operations Individual tasks can be executed in a decentralized manner End-to-end workflows can be executed in a decentralized manner (BFT Orderer will be supported)
16. © Hitachi, Ltd. 2023. All rights reserved. 15 2-5. A reference architecture of fully-decentralized Fabric-based system Fabric Comps. Fabric Comps. IaaS (Network, computing resources) Peer Peer Peer Application Channel Chaincode CA CA CA Org1 Org2 OrgN … Orderer Node Orderer Node Endorsement Policy Private Data Fabric Comps. Decentralized Mgmt. Tool Fabric CLI Fabric CLI Fabric CLI API Server API Server API Server Decentralized Mgmt. Tool Decentralized Mgmt. Tool Orderer Node BFT protocol User Exec. Mgmt. Propose Approve Adjust Execute Issue TX Approve (*) EP: Endorsement Policy CC: Chaincode, PD: Private Data BFT: Byzantine fault tolerant Admin • Expect that such a decentralized system will be ready to implement once the functions presented in the previous slides are provided • Refer to the following for details on this reference architecture – [1] Toward Fully-Decentralized System With Hyperledger Fabric, Hyperledger Global Forum, 2022. – [2] Toward Fully-Decentralized System With Hyperledger Fabric, IEICE ComEX, 2023.
17. © Hitachi, Ltd. 2023. All rights reserved. 16 3. Summary 1. Case Study Topic: What Hitachi Learned from Practical Use of Fabric – Introduced Hitachi’s Blockchain business and activities – Shared some issues and insights in actual Fabric environments from both development and operational perspectives 2. Future Topic: Towards Deploying a Fully-Decentralized Fabric System – Presented on the development status of BFT orderer in the community • The preview version is now available – Introduced OpsSC (Operations Smart Contract), a decentralized operation tool for Fabric • Current implementation: Support end-to-end operational workflows on chaincode and channel management • Future Update Plan: Channel management with BFT orderer
18. © Hitachi, Ltd. 2023. All rights reserved. 17 Trademarks • Linux Foundation, Hyperledger, Hyperledger Fabric and Kubernetes are registered trademarks or trademarks of The Linux Foundation. • GitHub is a registered trademark or trademark of GitHub Inc. • All other company names, product names, service names and other proper nouns are registered trademarks or trademarks of their respective companies. • The TM and 🄬 marks are not shown in the text and figures in this slide.
19. © Hitachi, Ltd. 2023. All rights reserved.