Cordite’s Metering Notaries, a Deeper Dive

Cordite’s Metering Notaries, a Deeper Dive

In Richard Crook’s medium article, he takes us on a blockchain incentives journey from Proof of Work, Proof of Stake and then to Metering in Cordite.

In this article I am going to walk through the current metering implementation in Cordite, the design choices that we made and ultimately how it works.

I should emphasise that, at the moment, the the current implementation of metering is one of many possibilities and we are not limited to single approach to metering — we are not opinionated!

A short version is provided here, for those of you in a hurry

‘Metering why, what, how, in 9 bullet points’

Why?

1. The purpose of metering is to incentivise organisations to run notaries and to ensure that cost of transactions is fairly distributed among high and low volume users, without reverting to environmentally expensive algorithms.

What?

2. Metering is a cost recovery model, where the notary counts all the transactions processed by a business node and then sends it an invoice for those transactions. This is very similar to how you’re are billed for your electricity usage.

How?

3. Metering currently runs as a Corda Service on a single node validating or non validating notary.

4. Transaction Cost, Token Type and Allocation of fees are held in a DAO State called ‘meteringModelData’.

5. Metering Invoices are set to the business nodes originating transactions using the standard Corda Collect Sigs/Finality Flow.

6. Business nodes can pay or dispute Invoices.

7. Metering Payments are sent to the DAO node which then disperses fees according to the DAO rules.

8. To integrate Metering into an existing Cordapp, it needs to be notary ‘aware’ and have the Cordite jars.

9. Unlike existing PoW and PoS protocols, the metering design is not constrained to a single reward model.

Still here? — The Deeper Dive

The following two sections are recaps of the whys and whats of metering in my own words — if you feel comfortable with both of these then feel free to move onto the design considerations section.

Why Metering?

‘Is it fair if Alice and Bob pay the same price for different transaction usage?’

Imagine for a second that you decide the concept of metering is complete nonsense and you start your own notary business called ‘Rock Solid Notaries Inc’.

You buy a production worthy cloud server with 16 cores and 64 GB of memory, that costs you 70 pence an hour , that’s roughly £500 a month if it runs at full tilt.

You have two customers already and decide to charge them £250 each a month for using your notary; you are secretly hoping that you will profit from the server not being fully utilised in the first few months of your business.

You first customer is Alice, she works for Mega corp and they push 10,000 transactions in the first month. Alice is delighted as her transaction cost is 2.5 pence a transaction.

You second customer is Bob who has just started up his own alt coin brokerage business. He is really excited about new customers using his service, so he pays the £250 notary subscription. Unfortunately Bob has a bad month, only makes two deals, does two transactions resulting in his transaction cost being £125 a transaction.

At the end of the month, you, Bob and Alice go out to celebrate your new business.

Alice and Bob get talking. Bob’s face drops when he hears that Alice is paying 2.5p a transaction. Bob looks sad, so you decide to buy him a beer from your profits, but when you look at your bank account the balance is zero as your server was 100% utilised.

You decide that metering might actually be a good idea after all if you want a fairer business model.

What is metering?

‘Utility company billing is a good model on which to based cost recovery’

Metering follows the same model as the utility bill in that you use some resources, be it gas, electricity or mobile phone data and at some point in the future you get a bill for your usage; and at some point further in the future you pay the bill!

In the case of metering notaries, the transactions are the resources and ‘metering’ invoices are sent to the creators of those transactions.

One of the advantages of adopting a metering and bill model is that we have the opportunity to create a number of different business models that mirror how current utility models work; these enable companies to control and plan their cash flow. Here’s a few examples:

• Pay as You Go — only pay for what you use.
• Freemium — the first 1000 transactions are free, then you start paying.
• Upfront Discount — you pay for a block of ‘cheaper’ transactions up front and then pay the normal rate once you have used up your block.
• Delay Repay — You have 90 days to pay your metering fees.

None of the above models are particularly original, but if you compare this flexibility to the single currency and single transaction cost models of bitcoin, Ethereum and some of the nascent Proof of Stake models, then I hope you’ll agree that there are opportunities to create cost models to suit all shapes and sizes of businesses.

Design Considerations

`slapping a cost state into each Corda transaction might not be such a good idea`

When we were designing the cost recovery model for Cordite we had many discussions around the business implications and technical implications. At the time we had no idea that we would end up with metering but we were ultimately driven towards it, by asking some key questions.

Should we embed a transaction cost on each Corda Transaction?

Most of today’s crypto currencies work in this way; in order for you to get your bitcoin transaction processed, you must embed what you are prepared to pay in the transaction.

For someone who has built a Cordapp and who then wanted to use this model, they would have to change all of their flows to embed some kind of ‘token’ state in every one of their transactions; this would immediately create an expensive barrier to adoption.

We thought it better if you could magically point your Cordapp at a metering notary and cost recovery would just work — we are very close to that.

Should we change the Core Corda Codebase ?

Another option would have been to bury the transaction cost model into the Corda Core code so that everyone gets it no matter what.

Our general principle with Cordite was that it would be built on top of Corda and we’d only add more functionality to Corda Core if absolutely necessary.

Being agile in that we can choose our own release cycles was important as much as not turning Corda into a ‘God’ object that does too many things.

Following David Rutter’s analogy of Corda being the Operating system, then Cordite is the ‘Office Suite’ providing a set of key tools to enable your business.

`Why limit yourself to a single economic model?`

What currency/token should we use for cost recovery?

Having a single token and with it a single issuance and hence economic model would work for some business networks, but probably not for all.

Our view was that we needed to have some kind of token to pay for transactions , but we could not predict how businesses in future would like to exchange value; some might only wish to exchange fiat backed tokens where as other business may wish to have their own utility tokens, reward tokens or enterprise tokens.

We decided that we would let the business networks and notaries owners decide which tokens they use.

`paying a transaction cost to raise an invoice could leave you in chicken egg paradox`

What if users don’t have any money to pay for their transactions?

Imagine the paradoxical scenario where in order to get some money for your business you need to send an invoice; but in order to send an invoice you need to pay a transaction cost?

It is often the case that new businesses don’t have a lot cash floating about and need to pay their bills later — most invoices these days have a 90 day payback period.

We carefully considered how we could manage this in Cordite and came up with the met
ering concept base on existing utility services model. The utility company meter your transactions and then send you a bill.

`empower users to protest against poor service`

What if node owners have received a poor service?

Once we have an ‘invoicing’ system, we now need to handle cases where notaries have provided a poor service or nodes have not paid their bills on time.We decided to implement ‘dispute’ functionality so that we can govern poor service and inaccurate charging.

How do we govern metering costs and distribute funds?

By this point we had already fleshed out the concept of the Cordite DAO; we decided to use this to govern the metering rules for a business network such as the token type, transaction cost and allocation of rewards to the metering ‘parties’. This effectively enables a level of control over pricing and fee distribution; think of it like ‘ofgem’ for DLT. (Note: these are the regulators for water and power in the UK).

`who polices the police?`

Should Metering notaries be able to notarise their own invoice transactions?

We took a pessimistic view that metering notaries might be malicious, and we should not trust them to notarise their own metering invoices.

Therefore we needed to introduce another notary, originally called the ‘police’ notary, but later changed it to the more friendly ‘Guardian’ Notary. This had nothing to do with seeing Guardians of the Galaxy II at the cinema the previous week.

In asking these questions we have hopefully come up with a cost recovery model that can be adapted to suit the requirements of many different types of business networks.

`Metering is Cordite eating its own dog food`

How does metering work?

Before I go into great detail about how metering works, it’s worth mentioning that metering utilises the other two components of Cordite, the DGL and DAO. If you are not familiar with these, then you can read about them here:

Metering is built from a set of Corda collect signature/finality flows and two Corda services that run on the Metering Notary and DAO Node(s).

If you are not familiar with Corda Services, then you can get the lowdown on these in a great article from Cordite contributor Dan Newton here.

This diagram shows a view of the key actors in metering and how they orchestrate the metering flow.

There are four isolated ‘transactions’ at play in this diagram. This isolation prevents metering affecting the overall performance of the initial business transaction and enables less friction in the overall operation.

The transactions are numbered 1–4 and do the following:

1. A business transaction occurs between Node A and Node B; this is notarised by the metering notary.

2. The Metering Service retrieves the transaction from the Notary’s vault and issues a Metering Invoice to Node A that created the transaction; this is notarised by the Guardian notary.

3. Node A pays the Metering invoice using the specified DGL token and sends this payment to a DAO Node; this is also notarised by the Guardian notary.

4. The Fee Dispersal Service splits the metering fee according to the Metering rules held in the DAO and pays the Metering Notary, Guardian Notary and DAO Foundation Fund.

`Corda services are Metering’s best friends`

The key drivers in Metering are the Metering Service and the Fee Dispersal Service.

How does the metering service work?

For the metering service to start on a notary, you need a small config file. Here it is in all it’s glory.

{
“meteringRefreshInterval” : 2000,
“meteringServiceType” : “SingleNodeMeter
er”,
“daoPartyName” : “Cordite Committee”,
“meteringNotaryAccountId”: “metering-account-acc1”,
“meteringPartyName”: “Cordite Metering Notary”,
“daoName”:”Cordite Committee”
}

It is worth talking through each parameter as this will flesh out the metering story.

• meteringRefreshInterval

Currently the metering service runs on poll of 2 seconds. During each poll it checks it configuration and then reads the committed transactions in the notary’s vault.

For single node notaries, the transactions are stored in the following table:

NODE_NOTARY_COMMIT_LOG

In your own implementation of metering you may want to do this during quiet times, at scheduled times or even daily; each of these options will suit your own use case.

• MeteringServiceType

We are anticipating a number of different types of notaries and metering types in the future; this parameter drives the type of ‘Meterer’ which is inherited from an ‘Abstract’ meterer class.

• daoPartyName

Metering invoices are paid from the Business node to the DAO. The metering notary puts this parameter on each metering invoice, so the Business node knows where to send the payment.

• meteringNotaryAccountId

This is the Name off the Cordite DGL account which the metering fees are paid into; again, these are stamped on each invoice. As an aside to this, in the business world today, many invoices are issued without actually stating to which account the payment should be made — this is quite shocking if you think about it!

• Metering Party Name

The metering service checks the metering notary’s node Organisation name against this parameter. If they don’t match then the metering service will exit; this acts as additional belts and braces against metering running on the wrong node by accident.

• daoName

The rest of the metering configuration is held in a specific DAO state. Note: there can be many DAO states in the network. For our test network our DAO state is simply called ‘Cordite Committee’

The Metering service checks for the DAO state on each poll to ensure that it is up to date with the latest metering charges and configuration.

If it does not find a DAO State containing metering parameters, then nothing gets metered!

`Your business network members ultimately control the metering rules`

The rest of the metering configuration is stored in a class called ‘meteringModelData’ which is stored inside the DAO State.

This is what it looks like in json with some edits for brevity

{
meteringTransactionCost:
{ meteringTransactionCost: 10,
  meteringTransactionTokenDescriptor:{ symbol: ‘XTS’ }
},

meteringNotaryMembers:{

‘OU=Cordite Foundation, O=Cordite Metering Notary, L=London, C=GB’:{ accountId: ‘metering-notary-account1’,
description: ‘I am a cool metering notary’,
meteringNotaryType: ‘METERER’ },

‘OU=Cordite Foundation, O=Cordite Guardian Notary, L=London, C=GB’:{ accountId: ‘guardian-notary-account1’,
description: ‘I am a guardian of the galaxy’,
meteringNotaryType: ‘GUARDIAN’ }
},

meteringFeeAllocation:{
daoHoldingAccountId: ‘dao-holding-account’,
daoFoundationAllocation: 50,
meterNotaryAllocation: 40,
guardianNotaryAllocation: 10,
daoFoundationAccount: ‘dao-foundation-account’ }
}

The config is split into three main parts.

MeteringTransaction Cost:

This determines the type of DGL token that is used to pay the metering fees and how much of that unit should be paid.

Metering Notary Members:

This defines a list of all notaries ‘approved’ by the DAO. The metering Notary picks its Guardian notary from this list. With Corda 4 we will be able to enforce the use of this list using Roger Willis’s marvellous reference states implementation.

Metering Fee Allocation:

This determines how the metering fees are split between the Metering notary, Guardian notary and your business network fund.

All of the above parameters are controlled by the DAO managed by your business network through the voting and proposal mechanism; this enables governance of metering by the DAO members.

In the current implementation, the Metering and Guardian notaries are voting members of the DAO so they can vote against any proposals for ‘free’ transactions !!!

This takes us nicely to the fee dispersal service.

`rewards are dispersed according to rules set by the business network`

How does the fee dispersal service work?

You’ve probably guessed by this point that the Fee Dispersal Service needs a configuration file, so here it is:

{
“feeDispersalRefreshInterval”:2000,
“feeDispersalServicePartyName”:”Cordite Committee”,
“daoName”: “Cordite Committee”
}

Like the metering service the fee dispersal service runs on a 2 second poll at the moment and then extracts the ‘meteringModelData’ from the DAO state.

The service runs on the designated DAO node that collects the metering fees.

Metering fees are paid into Cordite DGL account called the ‘holding account’.

The Fee Dispersal Service checks its vault for ‘Paid’ metering invoices and when it finds a Paid metering invoice it reads the metering fee allocation structure and divides the tokens in the holding account according to the Metering fee allocation table.

It then creates a ‘MeteringInvoiceSplit’ for each of these allocations and sends these to the Metering and Guardian notaries as well as putting some of the funds in the ‘Dao foundation account’ to be used to fund further development of your Business network.

All of these metering transactions are notarised by the Guardian notary, which leaves with a slight issue in that we need to pay the Guardian notary for it’s valiant efforts. Therefore we have to do a notary transfer to the Metering notary before we can pay the Guardian notary.

At this point, you might say, ‘surely the metering notary will then meter this transaction and create an infinite metering loop’ — well it did until I fixed the code to not charge for transactions from the Guardian notary!

`No Cordapp is of any use without a robust set of flows’

The metering flows

No Cordapp is a true Cordapp without a set of robust flows and its worth talking about these to round things off and fill in a few extra details.

The Metering Invoice goes through a life cycle of states as it moves from issuance to payment, to dispersal. This is a pretty bog standard state machine approach to managing a business flow. These states are defined in the ‘MeteringState’ enum — I will mention these states as I explain the flows.

‘All metering transactions can be done in bulk’

IssueMeteringInvoicesFlow:

As it says on the tin, this issues metering invoice from the Metering Notary to the Business node that instigated the transaction. As with all the metering flows, this is notarised by the Guardian notary.

You may have noticed that the flow name is pluralised; this is a because all the metering flows can handle multiple invoices and payments. The metering invoice contract also goes to great pains to validate that there is no skullduggery when submitting multiple states into the transaction.

At the receiver end of the this flow, the business node checks that it did in fact create the transaction and it was notarised by the metering notary. This again reflects my view that we should distrust everything from where ever it came.

At this point the metering invoice is in the ‘ISSUED’ state and is owned by the node owner.

‘The Powerful concept of paying for stuff in an atomic transaction`

PayMeteringInvoicesFlow:

This is instigated by the Business node which essentially means that the node owner is in control of when she pays her metering bill.

The parameters to the flow are a list of metering transaction Ids and the Cordite DGL account Id from which the fees will be paid.

In this simple flow there is a very powerful concept which I call the ‘pay for stuff’ transaction.

Up to this point in technology solutions, paying an invoice result in the data moving onto separate rails, one for the payment (BACS, SWIFT etc), and one for the Invoice (email, fax, post! ?).

When we can combine the payment and the paid invoice in a single atomic operation, it is hugely powerful — accounts clerks across the global will no longer spend endless hours matching invoice to payments; and the fraudster who sends random invoice to small companies will head for the hills.

This flow uses the Cordite DGL to do that and in doing so we have ensured that you can easily integrate DGL token states in your own flows to pay for digital items such as invoices, digital media, loans, m
ortgages and crypto hamsters.

At this point the metering invoice is in the ‘PAID’ state and is owned by the DAO dispersal node.

‘Splitting the invoice with the DGL token chosen by the business network`

DisperseMeteringInvoicesFlow:

I covered dispersal above, so the only thing to add is that the metering invoice is split into three ‘metering invoice splits’ and sent with the corresponding payments in a DGL token of the DAO’s choice — again atomic operations reducing the need for reconciliation.

If all goes well then the Metering invoice will end up in a ‘FUNDS_DISPERSED state.

In addition the three Metering Invoice split states will end up in a ’SPLIT_DISPERSED’ state

‘On ledger digital disputes’

DisputeMeteringInvoicesFlow:

If the business node owners thinks the metering invoices are unfair she can send them back to the metering notary using this flow. Its simple and to the point!

At this point the metering invoice is in an ‘IN_DISPUTE’ state and owned by the Metering notary’

‘Uniqueness and Resolving Digital Disputes`

ReIssueMeteringInvoiceFlow:

The metering notary ‘owner’ can reissue the metering invoice, but has limited power over which fields can change in the invoice — currently the amount, token type and target account.

The dispute/re-issue cycle can happen many times as it can do in current business models. This highlights another issue in that we need to be very careful to preserve uniqueness in our metering invoices to prevent the creation of duplicate invoices.

We do this by enforcing a unique index on the metering invoice in the vault as well as checking the ‘reissueCount’ carefully in the metering invoice contract.

‘Open Source, Open Thought’

In conclusion

Thank you for getting this far and I hope that this article has given you a good insight into Cordite Metering, how it consumes the other Cordite components and ultimately how you can build fairly complex Cordapps using Corda flows and services.

If you want to set up cordite metering for your Cordapp then it’s worth looking at our end to end metering integration test which runs through the whole thing. Check out this class in the cordite repo:

MeteringIntegrationTest

Here is a preview of the test:

private fun runEndToEndTest() {
  createDaoAndMembers()
  setUpDaoMeteringParameters()
  createAndLoadAccountsForMeteringFunds()
  createAccountsAndDoSomeTransfers()
  checkForMeteringInvoicesAndPayThem()
  checkFinalBalances()
}

Cordite is shared as open source and this article has been our attempt to also share our thought processes — the ‘why’ is just as import as the ‘how’

There is an increasing number of contributors to cordite and we welcome all size of commits and design thoughts; to find out more, start here!