All Posts By

Hyperledger

(1.27.2019) Smartereum: US Based Agricultural Conglomerate, Cargill, Invests Digital Engineering Into The Development Of Hyperledger Grid

By | Hyperledger Grid, News
According to a press release on the 25th of January 2018, the US based agricultural conglomerate, Cargill will be investing its digital engineering resources into the development of Hyperledger Grid. The grid is one of the most recent projects by Hyperledger that is geared towards streamlining the agricultural supply chain using Hyperledger’s blockchain technology.
View the full article here.

Hyperledger Sawtooth events in Go

By | 网志, Hyperledger Sawtooth

This blog post shows how to use the Sawtooth Go SDK for event handling. We can create events in a transaction processor and write a client to subscribe to events via ZMQ. There’s also a web socket based event subscription feature not in scope of the topic discussed here. The article has information useful for developers, technical readers, architects, Sawtooth enthusiasts and anybody interested in blockchain.

Please refer to Sawtooth Go SDK reference for setting up the development environment. It includes information on how to write a client in Go.

Events and Event Subscription In Sawtooth

Thanks to the many open source contributors. The Hyperledger Sawtooth documentation describes in detail about event and event subscription. This article supplements the document. Sawtooth has SDK support in many languages, giving flexibility to the application developer. We will discuss the Go SDK usage in detail.

Sawtooth defines two core events sawtooth/block-commit and sawtooth/state-delta. Both occur when a block is committed. The former carries information about the block. The latter carries information about all state changes at the given address. In addition to the predefined events, Sawtooth allows application developers to define new events. A combination of sophisticated SDK and protobuf definitions abstract the complex network communications. The SDK exposes APIs to add new application-specific events. It also provides the mechanism for clients to subscribe to available events.

Figure 1: Representation of the Validator connection with Transaction Processor and Client via ZMQ.

Event: A savior for real world use cases

  1. Notification events such as state change and block commit help the design of interactive clients. It saves clients from having to poll the validator for the status of submitted batches.
  2. Events can be used  as a debugger for transaction processors. Transaction processors can register application-specific events allowing clients to debug the flow remotely..
  3. In addition to debugging, the asynchronous behavior allows for stateless application designs.
  4. Deferring the processing of events at a later point in time. Sawtooth Supply Chain is one such well known example, which makes use of this feature.
Caution beginners! Do not get carried away with overwhelming application specific use cases. Events are also being used for internal operation of Sawtooth. The horizon is open and our imagination is the only limit.

Adding Application-Specific Event In Go

Creating a custom event involves defining three parts, namely

  1. Define attributes: A list of key-value pairs, which later is used to filter during event subscriptions. There can be many values per attribute key.
import

("github.com/hyperledger/sawtooth-sdk-go/protobuf/events_pb2")

// Create a key-value map to define attribute

attributes := []events_pb2.Event_Attribute{

Key: "MyOwnKey", Value:"ValueIWantToPass",

})

2. Define data payload: Byte information for a specific event type.

payload := []byte{'f', 'u', 'n', '-', 'l', 'e', 'a', 'r', 'n', 'i', 'n', 'g'}

3. Define event type: Used as an identifier when subscribing for the events.

After defining these parameters, we can use the available API in the SDK to create an event and add to the context. Listing 1 shows and example of adding an event. Please note that any change to the context’s events will affect the way subscribed clients work.

import("github.com/hyperledger/sawtooth-sdk-go/processor")

func (self *OneOfMyTPHandler) Apply(

request *processor_pb2.TpProcessRequest,

context *processor.Context)

error {    // -- snip --

context.AddEvent(

"MyEventIdentifier",

attributes,

payload)

// -- snip --
}

Listing 1: Adding an event to the context.

Event-Subscription In Go

Subscribing to an event involves establishing a ZMQ connection with the validator. The event types specify which events are subscribed to. An optional filter attribute can be passed for each event when establishing the subscription. The Sawtooth Go SDK uses protobuf definitions for serializing messages exchanged between the client and the validator. The following four steps show the sample code snippets.

  1. Establish a ZMQ connection: To establish a ZMQ connection from a client as a DEALER. Detailed description can be found in ROUTER-DEALER mechanism in ZMQ. The Sawtooth SDK provides an API for establishing client connection with validator.
import (
"github.com/hyperledger/sawtooth-sdk-go/messaging"
"github.com/pebbe/zmq4"
)
zmq_context, err := zmq4.NewContext()
// Error creating a ZMQ context
if err != nil {
   return err
}

// Remember to replace <VALIDATOR-IP> with hostname
// or the IP where validator is listening on port 4004
zmq_connection, err := messaging.NewConnection(
   zmq_context,
   zmq4.DEALER,
   "tcp://<VALIDATOR-IP>:4004",
   //This specifies a server connection which needs
   //binding or client connection that needs to
   // establish a request to server
   false,
)
// ZMQ connection couldn't be established
if err != nil {
   return err
}
// Remember to close the connection when either done
// processing events processing or an error occursdefer zmq_connection.Close()
// -- snip ––

2. Construct EventFilter, EventSubscription and ClientEventsSubscribeRequest: The Event is said to be subscribed when both event_type and all the EventFilters in EventSubscription field match. EventFilters can be applied on attributes defined earlier. FilterType determines rules for  comparing the match string.

import("github.com/hyperledger/sawtooth-sdk-go/protobuf/events_pb2") 

// Define filters over attributes to be triggered when
// a string matches a particular filter type
filters := []*events_pb2.EventFilter{&events_pb2.EventFilter{   
   Key: "MyOwnKey", 
   MatchString: "MyUniqueString", 
   FilterType:  events_pb2.EventFilter_REGEX_ANY,
}} 
my_identifier_subscription := events_pb2.EventSubscription{   
   EventType: "MyEventIdentifier", 
   Filters: filters,
} 
// -- snip --
// Construct subscription request for the validator
request := client_event_pb2.ClientEventsSubscribeRequest{
   Subscriptions: []*events_pb2.EventSubscription{
     &my_identifier_subscription, 
     &my_another_identifier_subscription,
   },
}
// -- snip ––

3. Send request over ZMQ connection: The client’s event subscription request can be sent through an established ZMQ connection. Note that a correlation id returned by SendMsg() can be used to know if the validator as a ROUTER has response messages. Many events can be subscribed to at once.

import (
 "errors"
"github.com/golang/protobuf/proto"
 "github.com/hyperledger/sawtooth-sdk-go/protobuf/client_event_pb2"
 "github.com/hyperledger/sawtooth-sdk-go/protobuf/validator_pb2"
)
// Get serialized request using protobuf libraries
serialized_subscribe_request, err :=
    proto.Marshal(&request)
if err != nil {
  return err
}
 // Send the subscription request, get a correlation id
// from the SDK
corrId, err := zmq_connection.SendNewMsg(
    validator_pb2.Message_CLIENT_EVENTS_SUBSCRIBE_REQUEST,
    serialized_subscribe_request,
)
// Error requesting validator, optionally based on
// error type may apply retry mechanism here
if err != nil {   return err} // Wait for subscription status, wait for response of
// message with specific correlation id_, response, err :=zmq_connection.RecvMsgWithId(corrId)
if err != nil {
  return err
}
 // Deserialize received protobuf message as response
// for subscription requestevents_subscribe_response :=
    client_event_pb2.ClientEventsSubscribeResponse{}

err = proto.Unmarshal(
    response.Content,
    &events_subscribe_response)
if err != nil {
  return err
}
 // Client subscription is not successful, optional
// retries can be done later for subscription based on
// response cause
if events_subscribe_response.Status !=
  client_event_pb2.ClientEventsSubscribeResponse_OK {
  return errors.New("Client subscription failed")
}
 // Client event subscription is successful, remember to
// unsubscribe when either not required anymore or
// error occurs. Similar approach as followed for
// subscribing events can be used here.
defer func(){
    // Unsubscribe from events
    events_unsubscribe_request :=
      client_event_pb2.ClientEventsUnsubscribeRequest{}
    serialized_unsubscribe_request, err =
      proto.Marshal(&events_unsubscribe_request)
    if err != nil {
      return err
    }
    corrId, err = zmq_connection.SendNewMsg(

      validator_pb2.Message_CLIENT_EVENTS_UNSUBSCRIBE_REQUEST,
      Serialized_unsubscribe_request,
    )
    if err != nil {
      return err
    }
    // Wait for status
    _, unsubscribe_response, err :=
    zmq_connection.RecvMsgWithId(corrId)
    // Optional retries can be done depending on error
    // status
    if err != nil {
      return err
    }
    events_unsubscribe_response :=
      client_event_pb2.ClientEventsUnsubscribeResponse{}
    err =
      proto.Unmarshal(unsubscribe_response.Content, 
      &events_unsubscribe_response)
    if err != nil {
      return err
    }
    // Optional retries can be done depending on error
    // status
    if events_unsubscribe_response.Status !=
      client_event_pb2.ClientEventsUnsubscribeResponse_OK {
        return errors.New("Client couldn't unsubscribe successfully")
    }
}()
// -- snip ––

4. Event handling: The established ZMQ connection will send protobuf messages corresponding to the subscribed events.

import (
    "errors"
    "fmt"
    "github.com/golang/protobuf/proto"
    "github.com/hyperledger/sawtooth-sdk-go/protobuf/validator_pb2"
) 
// Listen for events in an infinite loop
fmt.Println("Listening to events.")
for {
  // Wait for a message on connection 
  _, message, err := zmq_connection.RecvMsg()
  if err != nil {
    return err
  }
  // Check if received is a client event message
  if message.MessageType !=
    validator_pb2.Message_CLIENT_EVENTS {
   return errors.New("Received a message not 
requested for")
  } 
  event_list := events_pb2.EventList{}
  err = proto.Unmarshal(message.Content, &event_list)
  if err != nil {
    return err
  }
  // Received following events from validator   
  for _, event := range event_list.Events {
    // handle event here
    fmt.Printf("Event received: %v\n", *event)
  }
}
// -- snip ––

Try it out!

References:

  1. Subscribing to Events, Using the Go SDK, from the Hyperledger Sawtooth website.
  2. Commits by arsulegai in sawtooth-cookiejar example.
  3. The Sawtooth Go SDK.

Chapter on Router-Dealer, ZMQ protocol.

Hyperledger Adds to Growing Line up of Groups Focused on Cross-Industry Development

By | 网志

In another step towards driving cross-industry cooperation in the development and use of blockchain technologies, Hyperledger has added two new groups to its growing list of Special Interest Groups (SIG). These two new groups, the Hyperledger Social Impact SIG and the Hyperledger Trade Finance SIG, join Healthcare and Public Sector focused SIGs.

SIGs gather community members from an industry segment to work on domain-specific problems and create an environment for open discussion, document co-creation and solution proposals. SIGs help specific vertical markets in their efforts to address problems specific to that particular community.

The Hyperledger Social Impact Special Interest Group

The Hyperledger Social Impact SIG is primarily focused on serving as a platform for exchanging ideas and exploration of ways to use blockchain technology in the context of social good. The Social Impact SIG will work with the other working groups and technical teams, especially in the areas of implementation.

The areas of focus include:

  • Identifying related use cases, current pilots and proofs of concept, production case studies, and the opportunity blockchain presents for social impact;
  • Sharing stories of successes, failures, lessons learned, opportunities and challenges;
  • Exploring cross-cutting concerns like security, privacy, and identity in global South contexts;
  • Identifying existing or needed common critical software components that would serve the particular needs of global social impact;
  • Working towards proposing solutions to the problems identified;
  • Identifying conferences or other opportunities to connect face to face, as well as submit talks or present as a group at an event.
  • Identifying the business and NGO community and building an inclusive platform for early adopters to contribute with their experiences.
  • Facilitating and raising awareness of the opportunities blockchain can bring to the field.

The group is currently working on developing an opportunity matrix to map out the landscape of activity in philanthropy and remittances, supply chain, governance and democracy, financial empowerment, and identity. We welcome your participating in this effort!

We also want to hear about your use cases in using Hyperledger in a social impact context, please let us know if you’d like to present at our next meeting by sending a message on our mailing list.

The Hyperledger Trade Finance Special Interest Group

The Hyperledger Trade Finance Special Interest Group is primarily focused on serving as a platform for exchanging ideas and exploration of ways to use blockchain technology in the context of trade finance.

Areas of focus for the group include:

  • Identifying related reference architectures (for example Trade Finance business and integration architecture, technical and infrastructure architecture), frameworks and models (OSI), use cases, current pilots and proofs of concept, and production case studies;
  • Sharing stories of successes, failures, opportunities and challenges;
  • Exploring and addressing cross-cutting architectural principles, options and decisions like performance and scalability, security, identity management and privacy, and identity in trade finance contexts;
  • Identifying existing or needed common critical software, middleware, and hardware components that would serve the particular needs of trade finance, including data transmission and processing from access point (device), through networks, and cloud computing deployment model options (private, public, hybrid)
  • Working towards proposing solutions to the problems identified;

How to get involved in Hyperledger Special Interest Groups

Participation in any SIG is open to everyone in the community. Each group has an open e-mail list, a Chat channel, and a wiki page. Live meetings are also held regularly via web teleconference. When needed, a task force can also be created within the SIG and have working sessions to discuss specific work items.
If you’re interested in joining a SIG, please subscribe to the SIG’s mailing list and start by saying “hi” to the community by sharing what your interests are on the topic. If you are looking for more information on the focus of an active SIG or proposed SIGs please visit wiki.hyperledger.org. We look forward to your participation and contributions!

Introduction to Sawtooth PBFT

By | 网志, Hyperledger Sawtooth

As of release 1.1, Hyperledger Sawtooth supports dynamic consensus through its consensus API and SDKs. These tools, which were covered in a previous blog post, are the building blocks that make it easy to implement different consensus algorithms as consensus engines for Sawtooth. We chose to implement the Raft algorithm as our first consensus engine, which we describe in another blog post. While our Raft implementation is an excellent proof of concept, it is not Byzantine-fault-tolerant, which makes it unsuitable for consortium-style networks with adversarial trust characteristics.

To fill this gap, we chose the Practical Byzantine Fault Tolerance (PBFT) consensus algorithm. We started work on the Sawtooth PBFT consensus engine in the summer of 2018 and continue to develop and improve on it as we work towards its first stable release. This blog post summarizes the PBFT algorithm and describes how it works in Sawtooth.

What Is PBFT?

PBFT dates back to a 1999 paper written by Miguel Castro and Barbara Liskov at MIT. Unlike other algorithms at the time, PBFT was the first Byzantine fault tolerant algorithm designed to work in practical, asynchronous environments. PBFT is thoughtfully defined, well established, and widely understood, which makes it an excellent choice for Hyperledger Sawtooth.

PBFT is similar to Raft in some general ways:

  • It is leader-based and non-forking (unlike lottery-style algorithms)
  • It does not support open-enrollment, but nodes can be added and removed by an administrator
  • It requires full peering (all nodes must be connected to all other nodes)

PBFT provides Byzantine fault tolerance, whereas Raft only supports crash fault tolerance. Byzantine fault tolerance means that liveness and safety are guaranteed even when some portion of the network is faulty or malicious. As long as a minimum percentage of nodes in the PBFT network are connected, working properly, and behaving honestly, the network will always make progress and will not allow any of the nodes to manipulate the network.

How Does PBFT Work?

The original PBFT paper has a detailed and rigorous explanation of the consensus algorithm. What follows is a summary of the algorithm’s key points in the context of Hyperledger Sawtooth. The original definition is broadly applicable to any kind of replicated system; by keeping this information blockchain-specific, we can more easily describe the functionality of the Sawtooth PBFT consensus engine.

Network Overview

A PBFT network consists of a series of nodes that are ordered from 0 to n-1, where n is the number of nodes in the network. As mentioned earlier, there is a maximum number of “bad” nodes that the PBFT network can tolerate. As long as this number of bad nodes—referred to as the constant f—is not exceeded, the network will work properly. For PBFT, the constant f is equal to one third of the nodes in the network. No more than a third of the network (rounded down) can be “out of order” or dishonest at any given time for the algorithm to work. The values of n and f are very important; you’ll see them later as we discuss how the algorithm operates.

Figure 1 — n and f in the PBFT algorithm

As the network progresses, the nodes move through a series of “views”. A view is a period of time that a given node is the primary (leader) of the network. In simple terms, each node takes turns being the primary in a never-ending cycle, starting with the first node. For a four-node network, node 0 is the primary at view 0, node 1 is the primary at view 1, and so on. When the network gets to view 4, it will “wrap back around” so that node 0 is the primary again.

In more technical terms, the primary (p) for each view is determined based on the view number (v) and the ordering of the nodes. The formula for determining the primary for any view on a given network is p = v mod n. For instance, on a four-node network at view 7, the formula p = 7 mod 4 means that node 3 will be the primary (7 mod 4 = 3).

In addition to moving through a series of views, the network moves through a series of “sequence numbers.” In the context of a Sawtooth blockchain, a sequence number is equivalent to a block number; thus, saying that a node is on sequence number 10 is the same as saying that the node is performing consensus on block 10 in the chain.

Each node maintains a few key pieces of information as part of its state:

  • The list of nodes that belong to the network
  • Its current view number
  • Its current sequence number (the block it is working on)
  • The phase of the algorithm it is currently in (see “Normal-Case Operation”)
  • A log of the blocks it has received
  • A log of all valid messages it has received from the other nodes

Normal-Case Operation

Figure 2 — Messages sent during normal operation of PBFT (Node 3 is faulty)

To commit a block and make progress, the nodes in a PBFT network go through three phases:

  1. Pre-preparing
  2. Preparing
  3. Committing

Figure 2 shows these phases for a simple four-node network. In this example, node 0 is the primary and node 3 is a faulty node (so it does not send any messages). Because there are four nodes in the network (n = 4), the value of f for the network is 4-13=1. This means the example network can tolerate only one faulty node.

Pre-preparing

To kick things off, the primary for the current view will create a block and publish it to the network; each of the nodes will receive this block and perform some preliminary verification to make sure that the block is valid.

After the primary has published a block to the network, it broadcasts a pre-prepare message to all of the nodes. Pre-prepare messages contain four key pieces of information: the ID of the block the primary just published, the block’s number, the primary’s view number, and the primary’s ID. When a node receives a pre-prepare message from the primary, it will validate the message and add the message to its internal log. Message validation includes verifying the digital signature of the message, checking that the message’s view number matches the node’s current view number, and ensuring that the message is from the primary for the current view.

The pre-prepare message serves as a way for the primary node to publicly endorse a given block and for the network to agree about which block to perform consensus on for this sequence number. To ensure that only one block is considered at a time, nodes do not allow more than one pre-prepare message at a given view and sequence number.

Preparing

Once a node has received a block and a pre-prepare message for the block, and both the block and message have been added to the node’s log, the node will move on to the preparing phase. In the preparing phase, the node will broadcast a prepare message to the rest of the network (including itself). Prepare messages, like pre-prepare messages, contain the ID and number of the block they are for, as well as the node’s view number and ID.

In order to move onto the next phase, the node must wait until it has received 2f + 1 prepare messages that have the same block ID, block number, and view number, and are from different nodes. By waiting for 2f + 1 matching prepare messages, the node can be sure that all properly functioning nodes (those that are non-faulty and non-malicious) are in agreement at this stage. Once the node has accepted the required 2f + 1 matching prepare messages and added them to its log, it is ready to move onto the committing phase.

Committing

When a node enters the committing phase, it broadcasts a commit message to the whole network (including itself). Like the other message types, commit messages contain the ID and number of the block they are for, along with the node’s view number and ID. As with the preparing phase, a node cannot complete the committing phase until it has received 2f + 1 matching commit messages from different nodes. Again, this guarantees that all non-faulty nodes in the network have agreed to commit this block, which means that the node can safely commit the block knowing that it will not need to be reverted. With the required 2f + 1 commit messages accepted and in its log, the node can safely commit the block.

Once the primary node has finished the committing phase and has committed the block, it will start the whole process over again by creating a block, publishing it, and broadcasting a pre-prepare message for it.

View Changing

In order to be Byzantine fault tolerant, a consensus algorithm must prevent nodes from improperly altering the network (to guarantee safety) or indefinitely halting progress (to ensure liveness). PBFT guarantees safety by requiring all non-faulty nodes to agree in order to move beyond the preparing and committing phases. To guarantee liveness, though, there must be a mechanism to determine if the leader is behaving improperly (such as producing invalid messages or simply not doing anything). PBFT provides the liveness guarantee with view changes.

Figure 3 — Messages sent for a view change in PBFT (Node 0 is the faulty primary, Node 1 is the new primary)

When a node has determined that the primary of view v is faulty (perhaps because the primary sent an invalid message or did not produce a valid block in time), it will broadcast a view change message for view v + 1 to the network. If the primary is indeed faulty, all non-faulty nodes will broadcast view change messages. When the primary for the new view (v + 1) receives 2f + 1 view change messages from different nodes, it will broadcast a new view message for view v + 1 to all the nodes. When the other nodes receive the new view message, they will switch to the new view, and the new primary will start publishing blocks and sending pre-prepare messages.

View changes guarantee that the network can move on to a new primary if the current one is faulty. This PBFT feature allows the network to continue to make progress and not be stalled by a bad primary node.

Want to Learn More?

This blog post only scratches the surface of the PBFT consensus algorithm. Stay tuned to the Hyperledger blog for more information on PBFT, including a future post about our extensions and additional features for Sawtooth PBFT.

In the meantime, learn more about PBFT in the original PBFT paper, read the Sawtooth PBFT RFC, and check out the Sawtooth PBFT source code on GitHub.

About the Author

Logan Seeley is a Software Engineer at Bitwise IO. He has been involved in a variety of Hyperledger Sawtooth projects, including the development of the consensus API, Sawtooth Raft, and Sawtooth PBFT.

Assembling the Future of Smart Contracts with Sawtooth Sabre

By | 网志, Hyperledger Sawtooth

Is WebAssembly the future of smart contracts? We think so. In this post, we will talk about Sawtooth Sabre, a WebAssembly smart contract engine for Hyperledger Sawtooth.

We first learned about WebAssembly a couple of years ago at Midwest JS, a JavaScript conference in Minneapolis. The lecture focused on using WebAssembly inside a web browser, which had nothing to do with blockchain or distributed ledgers. Nonetheless, as we left the conference, we were excitedly discussing the possibilities for the future of smart contracts. WebAssembly is a stack-based virtual machine, newly implemented in major browsers, that provides a sandboxed approach to fast code execution. While that sounds like a perfect way to run smart contracts, what really excited us was the potential for WebAssembly to grow a large ecosystem of libraries and tools because of its association with the browser community.

A smart contract is software that encapsulates the business logic for modifying a database by processing a transaction. In Hyperledger Sawtooth, this database is called “global state”. A smart contract engine is software that can execute a smart contract. By developing Sawtooth Sabre, we hope to leverage the WebAssembly ecosystem for the benefit of application developers writing the business logic for distributed ledger systems. We expect an ever-growing list of WebAssembly programming languages and development environments.

Unblocking Contract Deployment

The primary mechanism for smart contract development in Hyperledger Sawtooth is a transaction processor, which takes a transaction as input and updates global state. Sound like a smart contract? It is! If you implement business logic in the transaction processor, then you are creating a smart contract. If you instead implement support for smart contracts with a virtual machine (or interpreter) like WebAssembly, then you have created a smart contract engine.

If we can implement smart contracts as transaction processors, why bother with a WebAssembly model like Sabre? Well, it is really about deployment strategy. There are three deployment models for smart contracts:

  • Off-chain push: Smart contracts are deployed by pushing them to all nodes from a central authority on the network.
  • Off-chain pull: Smart contracts are deployed by network administrators pulling the code from a centralized location. Network administrators operate independently.
  • On-chain: Smart contracts are submitted to the network and inserted into state. Later, as transactions are submitted, the smart contracts are read from state and executed (generally in a sandboxed environment).

We won’t discuss off-chain push, other than to note that this strategy—having a centralized authority push code to everyone in the network—isn’t consistent with distributed ledgers and blockchain’s promise of distributing trust.

Off-chain pull is an opt-in strategy for updating software, and is widely used for Linux distribution updates. We use this model to distribute Sawtooth, including the transaction processors. By adding the Sawtooth apt repository on an Ubuntu system, you pull the software and install it via the apt-get command. Each software repository is centrally managed, though it is possible to have multiple software repositories configured and managed independently. This model has a practical problem—it requires administrators across organizations to coordinate software updates—which makes business logic updates more complicated than we would like.

On-chain smart contracts are installed on the blockchain with a transaction that stores the contract into an address in global state. The smart contract can later be executed with another transaction. The execution of the smart contract starts by loading it from global state and continues by executing the smart contract in a virtual machine (or interpreter). On-chain smart contracts have a big advantage over off-chain contracts: because the blockchain is immutable and the smart contract itself is now on the chain, we can guarantee that the same smart contract code was used to create the original block and during replay. Specifically, the transaction will always be executed using the same global state, including the stored smart contract. Because contracts are deployed by submitting transactions onto the network, we can define the process that controls the smart contract creation and deletion with other smart contracts! Yes, this is a little meta, but isn’t it great?

The on-chain approach seems superior, so why did we implement Hyperledger Sawtooth transaction processors with the off-chain model? Because our long-term vision—and a main focus for Sawtooth—has been smart contract engines that run on-chain smart contracts. Smart contract engines are more suitable for off-chain distribution, because they do not contain business logic, and are likely to be upgraded at the same time as the rest of the software.

Our initial transaction processor design reflected our goal for several types of smart contract engines. We later implemented one of them: Sawtooth Seth, a smart contract engine that runs Ethereum Virtual Machine (EVM) smart contracts. For us, Seth was a validation that our transaction processor design was flexible enough to implement radically different approaches for smart contracts. Like Ethereum, Seth uses on-chain smart contracts, so Seth is great if you want Ethereum characteristics and compatibility with tools such as Truffle. However, Seth is limited by Ethereum’s design and ecosystem, and does not expose all the features in our blockchain platform. We knew that we needed an additional approach for smart contracts in Hyperledger Sawtooth.

Crafting a Compatible Path Forward

Sawtooth Sabre, our WebAssembly smart contract engine, is our solution for native, on-chain smart contracts.  

The programming model for Sabre smart contracts is the same as that for transaction processors. A transaction processor has full control of data representation, both in global state and in transaction payloads (within certain determinism requirements). Hyperledger Sawtooth uses a global state Merkle-Radix tree, and the transaction processors handle addressing within the tree. A transaction processor can use different approaches for addressing, ranging from calculating an address with a simple field hash to organizing data within the tree in a complex way (to optimize for parallel execution, for example). Multiple transaction processors can access the same global state if they agree on the conventions used in that portion of state.

Sawtooth Sabre smart contracts use this same method for data storage, which means they can access global state in the same way that transaction processors do. In fact, smart contracts and transaction processors can comfortably coexist on the same blockchain.

The other major feature is SDK compatibility. The Sawtooth Sabre SDK API is compatible with the Hyperledger Sawtooth transaction processor API, which means that smart contracts written in Rust can switch between the Sawtooth SDK and the Sabre SDK with a simple compile-time flag. (Currently, Rust is the only supported Sabre SDK.) The details of running within a WebAssembly interpreter are hidden from the smart contract author. Because Sabre smart contracts use the same API as transaction processors, porting a transaction processor to Sabre is relatively easy—just change a few import statements to refer to the Sabre SDK instead of the Hyperledger Sawtooth SDK.

Now the choice between off-chain and on-chain smart contracts is a compile-time option. We use this approach regularly, because we can separate our deployment decisions from the decisions for smart contract development. Most of the transaction-processor-based smart contracts included in Hyperledger Sawtooth are now compatible with Sawtooth Sabre.

A Stately Approach to Permissioning

Hyperledger Sawtooth provides several ways to control which transaction processors can participate on a network. As explained above, transaction processors are deployed with the off-chain pull method. This method lets administrators verify the the transaction processors before adding them to the network. Note that Hyperledger Sawtooth requires the same set of transaction processors for every node in the network, which prevents a single node from adding a malicious transaction processor. Additional controls can limit the accepted transactions (by setting the allowed transaction types) and specify each transaction processor’s read and write access to global state (by restricting namespaces).

These permissions, however, are not granular enough for Sawtooth Sabre, which is itself a transaction processor. Sabre is therefore subject to the same restrictions, which would then apply to all smart contracts. Using the same permission control has several problems:

  • Sabre smart contracts are transaction-based, which means that a smart contract is created by submitting a transaction. This removes the chance to review a contract before it is deployed.
  • Sabre transactions must be accepted by the network to run smart contracts, but we cannot limit which smart contracts these transactions are for, because this information is not available to validator.
  • Sabre must be allowed to access the same areas of global state that the smart contracts can access.

An “uncontrolled” version of Sabre would make it too easy to deploy smart contracts that are  not inherently restricted to a to the permissions that the publisher of the smart contract selects.

Our solution in Sawtooth Sabre is to assign owners for both contracts and namespaces (a subset of global state). A contract has a set of owners and a list of namespaces that it expects to read from and write to. Each namespace also has an owner. The namespace owner can choose which contracts have read and write access to that owner’s area of state. If a contract does not have the namespace permissions it needs, a transaction run against the smart contract will fail. So, while the namespace owner and contract owner are not necessarily the same, there is an implied degree of trust and coordination between them.

Also, contracts are versioned. Only the owners of a contract are able to submit new versions to Sabre, which removes the chance that a malicious smart contract change could be accepted.

A Final Note About WebAssembly

On-chain WebAssembly isn’t limited to just smart contracts. For example, in Hyperledger Grid, we are using on-chain WebAssembly to execute smart permissions for organization-specific permissioning. Another example is smart consensus, which allows consensus algorithm updates to be submitted as a transaction. There are several more possibilities for on-chain WebAssembly as well.

In short, we think WebAssembly is awesome! Sawtooth Sabre combines WebAssembly with existing Hyperledger Sawtooth transaction processors to provide flexible smart contracts with all the benefits of both a normal transaction processor and on-chain smart-contract execution. Sabre also takes advantage of WebAssembly’s ability to maintain dual-target smart contracts, where the contract can be run as either a native transaction processor or a Sabre contract. And the permission control in Sawtooth Sabre allows fine-grained control over both contract changes and access to global state.

We are incredibly grateful for Cargill’s sponsorship of Sawtooth Sabre and Hyperledger Grid (a supply chain platform built with Sawtooth Sabre). We would also like to thank the following people who help make our blog posts a success: Anne Chenette, Mark Ford, David Huseby, and Jessica Rampen.

About the Authors

Andi Gunderson is a Software Engineer at Bitwise IO and maintainer on Hyperledger Sawtooth and Sawtooth Sabre.

Shawn Amundson is Chief Technology Officer at Bitwise IO, a Hyperledger technical ambassador, and a maintainer and architect on Hyperledger Sawtooth and Hyperledger Grid.

Developer showcase series: Raj Sadaye, Arizona State University

By | 网志, Developer Showcase, Hyperledger Composer, Hyperledger Fabric, Hyperledger Quilt

Back to our Developer Showcase Series to learn what developers in the real world are doing with Hyperledger technologies. Next up is Raj Sadaye.

What advice would you offer other technologists or developers interested in getting started working on blockchain?

One piece of advice I would offer someone interested in getting started with blockchain is to start working on a project or an application using the technology that they want to learn. It doesn’t have to be an elaborate or a complicated application, but could be something that has utility in the real world. While working on it, they might face difficulties or technical setbacks. The best way to tackle this is to reach out to the community of several other developers who are currently maintaining/ working on that technology. We can learn a lot by working on a project and reading the documentation thoroughly.

Give a bit of background on what you’re working on, and let us know what was it that made you want to get into blockchain?

My interest in blockchains developed when I was searching for a method to secure IoT device communication as well as make it de-centralized to increase speed. Blockchain technology turned out to be the perfect solution for this. Over the past 8 months, I’ve worked on several projects at Arizona State University’s Blockchain research Lab. In March 2018, I worked on building a PoC for a Carbon credits trading ecosystem using blockchain for Lightworks at ASU. The system enables various players of the market to control carbon emissions while maintaining sustainable growth by incentivizing carbon capturing actors. A brief description of this project can be found here. Currently, we’re working with the Center for Negative Emission of Carbon to design a way to verify capture as well as emission of carbon with minimal human intervention. My current research focus is developing a data sharing protocol that enables edge to edge communication in IoT devices. I’ve also been working on building the CSE 598: Engineering Blockchain Applications course on Coursera for Arizona State University.

What project in Hyperledger are you working on? Any new developments to share? Can you sum up your experience with Hyperledger?

Primarily, I’ve been working with Hyperledger Fabric and Hyperledger Composer and overall, the experience has been really good. Hyperledger Fabric has a good set of tools to build the infrastructure for a distributed ledger solution. The certification authority is a high-quality tool that helps us with cryptographic validation and dynamically assigning certificates for actors being added to the network. Once a person is familiar with the documentation, it’s really simple to go about building applications. Hyperledger Composer is a tool that excited me the most over the last 8 month because it runs on top of Hyperledger Fabric and it can help a blockchain novice on how to build a distributed application. Both frameworks have really good tutorial sections that help developers get familiar with the technology.

As Hyperledger’s incubated projects start maturing and hit 1.0s and beyond, what are the most interesting technologies, apps, or use cases coming out as a result from your perspective?

In my opinion, supply chains and blockchain technology were always meant to go hand in hand. The most interesting app or use case that I’ve been recently come across is Everledger. Everledger rewires trust and confidence in a previously broken market by building consortiums of actors participating and maintaining provenance by supplementing blockchain technology with various other verification techniques. In the near future, I see other products also adopt such an architecture to avoid counterfeiting and adultery.


What’s the one issue or problem you hope blockchain can solve?

One issue I’m hoping to be solved using blockchain technology is verification of identity through digitization of personal documents. Verification of documents using hash-based fingerprinting assigning ownership of this digital record to the person rather than a centralized authority can help a lot in maintaining the privacy of data as well as avoiding frauds through detection of counterfeit documents being used.


Where do you hope to see Hyperledger and/or blockchain in 5 years?

In 5 years, I expect blockchain technology to move over the crypto-hype and focus on the real applications and use cases that it be integrated with. For Hyperledger, the most interesting upcoming project, in my opinion, is Hyperledger Quilt which aims to achieve interoperability in blockchains. I’d also like to see a solution within the Hyperledger project that enables seamless integration of blockchain application with the existing infrastructure.

Hyperledger Iroha Security Audit Results

By | 网志, Hyperledger Iroha

Introduction

The time has come again for another Hyperledger project to begin their version 1.0 release process. Hyperledger Iroha is getting close to a 1.0 release and as part of that, Hyperledger hired an outside security auditing firm to review the code and audit it for security vulnerabilities. Nettitude conducted a review of the code this past fall and reported their findings to the Hyperledger security team and the Iroha developers.

The Iroha audit found four security issues, including one that was critical enough to require us to issue our first Common Vulnerabilities and Exposure (CVE) notice. All four issues were tracked using our JIRA and resolved shortly after the audit concluded.

I want to highlight the details of two of the security issues that the audit discovered because they show how easy it is to make bad assumptions about cryptography that results in a critical failure. Crypto code is always difficult to get right and as you will see, knowing good coding practices isn’t always enough. A developer must also be aware of algorithm and implementation details and the guarantees offered by a cryptographic primitive.

Blockchain Review

Before digging into the error, let us review the way things are supposed to work in a permissioned blockchain network. Figure 1 shows the normal process of transaction proposal and verification. In the diagram, Node 1 proposes the transaction by signing it and forwarding it to Node 2. Node 2 verifies the validity of the transaction as well as the validity of Node 1’s digital signature endorsement. Node 2 then endorses the transaction and forwards it to Node 3. Node 3 does the same checks as Node 2 except that it is also careful to ensure that the endorsements from Node 1 and Node 2 are both valid and unique. If everything passes the checks, Node 3 endorses the transaction and forwards it to Node 4. Node 4 now repeats the checks of Node 2 and Node 3 and sees that the transaction has enough valid and unique endorsements to be accepted into the next block of the blockchain. Node 4 transmits the fully endorsed and accepted transaction to all other nodes in preparation of the block construction and consensus steps. It is important to point out that not only is the validity of each digital signature important, but that a transaction also has enough unique endorsements before it will be accepted.

Figure 1—How a transaction is endorsed and validated.

Signature Schemes

Hyperledger Iroha uses the Twisted Edwards Curves based elliptic curve digital signature scheme more commonly known as Ed25519 or EdDSA. Unlike almost every other elliptic curve digital signature scheme, Ed25519 doesn’t take random data as one of its inputs. Most digital signature schemes generate a random number used only once—also known as a nonce (Number used ONCE)1—when calculating a digital signature of a message. The reason for this is because a digital signature is just a message digest encrypted using a public key encryption algorithm. Public key encryption algorithms are trivial to break if there is no nonce or a nonce gets reused, with the same secret key, to encrypt multiple messages.2 This is called a “chosen plaintext attack”.3 Figure 2 shows how a random nonce is used when encrypting the message digest to create the digital signature. By including a nonce, repeated use of the secret key over different messages does not compromise the encryption. Digital signatures using this method are different even though the same secret key and message are used.

Figure 2—Digital signature calculation with random nonce.

The Ed25519 signature scheme used by Iroha is different in that it generates the nonce by processing the inputs to the signing algorithm and thus repeated signatures of the same data with the same key result in the same encrypted data.4 This doesn’t compromise the key because the nonce is still different for different inputs. Figure 3 illustrates how the nonce for an Ed25519 digital signature is calculated from the input message and are therefore deterministic rather than generated randomly. Digital signatures using this method are the same when the same secret key and message are given.

Figure 3—Digital signature calculated with deterministic nonce.

The Bug

The flaw in Iroha was that the developers wrote the signature checking code to assume that signing the same data with the same key would always result in the same encrypted data. When determining if a transaction has enough different signatures to be valid, the code was comparing the public key bytes as well as the digital signature bytes when testing to see if two signatures were different. Figure 4 shows how the public key bytes and the digital signature bytes were combined when checking to see if two endorsements were different.

Figure 4—Flawed endorsement check that includes digital signature bytes.

The auditors at Nettitude created a modified version of the Ed25519 signature library so that it instead used random nonces, thus creating different encrypted data for the same secret key and message data. Figure 5 shows how the comparison of endorsements fails when random nonces are used. The resulting endorsements are not the same even though the message and secret key used to sign the message are the same.

Figure 5—Random nonces produce different signatures from the same inputs.

The result is that other nodes in the Iroha network—nodes running unmodified Ed25519 libraries—correctly validate the signatures because the public key correctly decrypts the digital signatures but the code for testing the uniqueness of the signatures is fooled. Each validating node sees different signatures for the same data and the same secret key and assumes they are unique endorsements and that the transaction is properly endorsed. Figure 6 shows how the Nettitude engineers were able to fully bypass this check with their single malicious node. It resulted in a bypass of the Byzantine guarantees of the system.

Figure 6—A malicious node bypassing the Byzantine checks.

The Fix

The correction for this security bug is to change the transaction and block signature validation code to first check that all signatures are valid and then check only the public keys for uniqueness when determining if there are enough valid and unique signatures on a transaction or block. Figure 7 shows how the scenario in Figure 6 plays out with the fixed code. Again a malicious node with a modified Ed25519 implementation signs a transaction multiple times with the same key. The signature bytes are unique, but the keys are not. When the other nodes in the network check the transaction, they see three valid signatures but the keys are not different. Each nodes determines that there is only one unique and valid signature and rejects the transaction.

Figure 7—A malicious node unable to bypass the Byzantine checks.

Two bugs were filed, one for transaction validation and one for block validation to address this flaw. The first bug is titled “multi-signature transactions can potentially be authorised by single user”5 The second bug is titled “vote early, vote often”6 Both flaws were fixed shortly after the report was given to us from Nettitude and the current version of Iroha has been fixed.

Conclusion

It is very important for developers to understand the subtleties of cryptography and applying it to engineering problems. Careful study and consideration of the guarantees and assumptions is required as well as multiple reviews from other engineers with similar knowledge and attention to detail. The “many eyeballs” theory of open source software development does work. This audit proved it.

The management and technical reports from the audit can be found on the Hyperledger wiki.

Links

  1. https://en.wikipedia.org/wiki/Cryptographic_nonce
  2. https://rdist.root.org/2010/11/19/dsa-requirements-for-random-k-value/
  3. https://en.wikipedia.org/wiki/Chosen-plaintext_attack
  4. https://ed25519.cr.yp.to/ed25519-20110926.pdf
  5. https://jira.hyperledger.org/browse/IR-2
  6. https://jira.hyperledger.org/browse/IR-3

Hyperledger Kicks Off the New Year with Eight New Members

By | Announcements, Hyperledger Fabric, Hyperledger Grid, Hyperledger Ursa


Growing community, new project developments and accelerating pace of deployments mark start of 2019

SAN FRANCISCO (January 30, 2019) Hyperledger, an open source collaborative effort created to advance cross-industry blockchain technologies, begins 2019 by announcing it has added eight new members to the consortium. In addition, Hyperledger has delivered some key technology updates and now has a total of 12 projects.

Hyperledger is a multi-venture, multi-stakeholder effort that includes various enterprise blockchain and distributed ledger technologies. Recent project updates include the release of Fabric v1.4 LTS, the first long term support version of the framework, as well as the addition of two new projects Hyperledger Ursa and Hyperledger Grid. Grid uses shared, reusable tools to accelerate the development of ledger-based solutions for cross-industry supply chain applications. Additionally, a detailed case study on Circulor’s Hyperledger Fabric-based production system for tracing tantalum mining in Rwanda adds to growing list of resources for guiding enterprise blockchain adoption.

“We wrapped up 2018 with a successful and exciting Hyperledger Global Forum,” said Brian Behlendorf, Executive Director, Hyperledger. “This first worldwide meeting of the Hyperledger community underscored the growing pace of development and deployment of blockchain in general and our tools and technologies in particular. We are seeing more signs of this accelerating pace of maturation and adoption here in early 2019. We welcome these newest members and look forward to their help in driving this growth.”

Hyperledger allows organizations to create solid, industry-specific applications, platforms and hardware systems to support their individual business transactions by offering enterprise-grade, open source distributed ledger frameworks and code bases. The latest general members to join the community are BTS Digital LLP, Exactpro Systems Limited, Jitsuin, Lares Blockchain, Myndshft, Omnigate, Poste Italiane and Wrapious Marketing Co Ltd.

New member quotes:

BTS Digital LLP

“We are an emerging company aiming at creating a national digital ecosystem in Kazakhstan that will facilitate the basic processes of human life and provide equal access to resources,” said Eugene Volkov, Chief Digital Officer, BTS Digital LLP. “As we see accelerated growth of transactions and actors in today’s life, we acknowledge the growing need to build a trustworthy society where all the participants can act with consensus, immutability, equality and transparency. Building such an environment requires trust. Our trust in Hyperledger’s expertise is a primary reason why we choose to become a member. We believe this community will guide us in finding technological solutions in achieving our goals.”

Exactpro Systems Limited

“Being a firm strategically focused on providing the highest level QA services for mission-critical market infrastructures, Exactpro understands the important role of this new technology and strives to enhance our expertise in this area through collaboration with leading blockchain consortia such as Hyperledger,” said Maxim Rudovsky, CTO, Exactpro. “We firmly believe our Hyperledger and The Linux Foundation memberships will provide Exactpro with access to community resources that will help us deliver more profound testing of DLT-based software systems to our clients.”

Jitsuin

“One of the founding decisions we made at Jitsuin was to become a Hyperledger member,” said Jon Geater, Chief Technology Officer, Jitsuin. “As part of our mission to unlock the value of data in the Internet of Things, we focus on Industrial IoT device lifecycle assurance where security, price, reliability and shared responsibility are all crucial. Keeping IoT in a known, good state is a team sport and is exactly where distributed ledger technologies work best. I am also delighted to continue serving the Governing Board and Hyperledger community to help ensure it remains the unrivaled home of advanced cross-industry business blockchain technologies.”

Lares Blockchain

“Lares Blockchain Security is delighted to join the Hyperledger community,” said Chris McGarrigle, CEO, Lares Blockchain Security. “Hyperledger’s fundamental strengths of performance, scalability and security resonate with our core values at Lares Blockchain Security. As our blockchain products and technologies continue to gain momentum in the medical, biotech, mining and financial industries, we see our partnership with Hyperledger as critical to further establishing ourselves in the enterprise.”

Myndshft

“Blockchain presents an enormous opportunity for healthcare to simplify and unify claims management, prior authorizations and other administrative functions, helping payers and providers reduce costs and improve timeliness and quality of care,” said Ron Wince, CEO, Myndshft Technologies. “That is why Myndshft is thrilled to join Hyperledger and collaborate with blockchain leaders and innovators across industries to find ways to leverage the technology to increase efficiency of healthcare operations, improve the patient experience and optimize financial performance in the value-based care era.”

Omnigate

“Omnigate Systems is delighted to join Hyperledger and to leverage blockchain technologies to drive interoperability in finance. Omnigate provides enterprise-grade, universal ledger software with extensive integrations. Our mission is to empower businesses of any size to rapidly build production-grade transactional systems for both traditional assets and emerging digital assets,” said Raphael Carrier, CEO, Omnigate. “We consider the integration of the Interledger protocol (via Hyperledger Quilt) into our product to be a key milestone. We believe this is an important initiative which will advance interoperability and accessibility to the ‘Internet of Value.'”

Poste Italiane

“Blockchain is not just a buzzword or a myth anymore, but is becoming the foundation for establishing a distributed, transparent and cross-industry interoperable ecosystem,” said Mirko Mischiatti, Chief Information Officer, Poste Italiane. “Poste Italiane wants to actively participate in this new and exciting community by becoming a member of Hyperledger in order to continue its path for the innovation and modernization of financial, logistic and insurance industries. We really look forward to working with other members and making our effort to contribute for the enhancement of blockchain technology.”

Wrapious Marketing Co Ltd

“It is our honor to become a member of the Hyperledger community,” said Tommy Wong, Chief Operating Officer, Wrapious Marketing Co Ltd. “Joining Hyperledger provides us with more opportunity to explore more within the blockchain space and to contribute to project developments. Our vision is to create a virtual world that provides equal access to everyone regardless of their status or social class in the community. We believe being part of Hyperledger will add to our ability to achieve this vision.”

About Hyperledger

Hyperledger is an open source collaborative effort created to advance cross-industry blockchain technologies. It is a global collaboration including leaders in finance, banking, Internet of Things, supply chains, manufacturing and Technology. The Linux Foundation hosts Hyperledger under the foundation. To learn more, visit: https://www.hyperledger.org/.