Hybrid transaction engines

Combining blockchains and SQL-DB?

Lucas Huber

If you are not familiar with blockchain technology have a look at this pretty crypto ledger crash-course


Transaction engines (software for banking or payment systems) of any kind keeping their data either in a distributed ledger (blockchain) or on a SQL Database. The latter is a standard or basic technology since many decades in computer technology. On the other hand blockchains where introduced by Bitcoin and other crypto-currencies less then 10 years ago and are only seen as an other type of database since some month.

Both have their pro's and con's and a recently intensified discussion did rise about the usability of the blockchain at all for large scale payment systems. So it may help to list the mayor pro's and con's of this two type of database technologies to get an idea how combining them could be a promising way to go.


Item

SQL

Blockchain

Scalability (Nr. of concurrent users)

good (needs large server infrastructure)

limited

Scalability (speed)

good (needs large server infrastructure)

limited (depends on kind of blockchain)

Resistance against malicious attacks from outside

Possible (needs a lot of effort to defeat such threats)

Depending on the type of consensus mechanism, easy to challenging

Resistance against malicious attacks from inside

Possible (needs a lot of effort to defeat such threats)

Much easier to build a fault tolerant system

Privacy (keeping data in-house)

Theoretically easy, but in practice difficult to achieve

Standard design is transparent but doesn’t keep a lot of metadata, with permissioned ledgers better privacy can be reached

Privacy (keeping data over time)

Theoretically easy to delete older data, but in practice difficult to achieve because of compliance rules

Until now impossible to delete older data in distributed ledger system

Costs

High if high security standards have to be met

low if higher speed is not a primary goal

 

This table is of course neither complete nor valid for any type of specific database technologies, but it shows that the pro's and con's are normally contrary or lets say complementary.
So why not combining the two technologies to build a transaction engine that can use many of the positive attributes of both type of databases?
Lets define a use-case that is straightforward and not to complex; community currencies are a good example because they need quite sophisticated functionalities, as loan aso. and also a high transaction speed when it comes to point of sales payments. Most of them are using today SQL based transaction engines as Cyclos but that may change in the future. Also it should be noticed, that in banking – even in community banking – often different types of accounts are in use. There are personal accounts, which are normally the ones used for the everyday payments and there are deposit or savings accounts to keep larger sums. For such type of accounts as deposit and savings a distributed ledger database is a perfect choice because speed is not important here and the long-time (ethernal?) data storage is from other aspects (eg. money laundering) a positive one. So a hybrid transaction engine for those does not make a lot of sense. But for personal accounts with a high volume and frequency of transactions, it could be a perfect solution, especially when we take into account that coins and notes are not in use for a particulate type of “community” currency. Or who loves the idea that a distributed database is tracking constantly every single payment that I did and keeps these records forever? I personally would prefer that, if I pay my personal things with coins, only my fingerprints on coin leave a trace of my identity. And that should also be the case if my coins are digital ones.

If I would start designing a hybrid transaction engine for everyday payments the core goal in the design would be consequently to achieve a short memory effect of the payments. In other world the ledger must contain a technology that allows to forget to whom I did spend my money after a while (a month, a year?), but also should meet the compliance and data integrity needs.

Out of this the database, that will contain all the data about the payment is the SQL database. I imagine that the transaction engine will then push on a daily basis the balance of each account to a distributed ledger that serves as a kind of backup instance of the SQL DB. Maybe this push information can contain some other information, as how many transactions that have been performed during the day. That's up to detailed specification and the use -case of such a hybrid system. Also it sounds reasonable not to push every single account every day, when noting happened on this particular account in the last 24 hrs.

What we are achieving in using a distributed ledger as described, is to have a kind of distributed backup of the most important part of a ledger, the balances of each account. And by doing this we get the freedom to delete or to consolidate the transaction records of the past, without to increase the risk do get into data integrity problems. Further advantages are, double check of data integrity; if the over all balance of the SQL DB and ledger are different it is obvious, there must be something wrong. If it wasn't an error in the software, eventually a SQL injection or another kind of attack could have been happen.

To avoid troubles with the regulatory it would be wise to dump, before deleting any old record in the SQL Database, the data into some backup file and prepare some easy restoring process. Just in the case that the information in the distributed ledger will not be detailed enough for certain circumstance.

Of course to create such hybrid transaction engine (HTE) is not an easy task and certainly some hassles would coming up during a future development. One challenge would be to synchronize a timeline between the two database instances. If this is not done very carefully it will be hard to perform a balance check between the two. Another important issue is how to handle the keys; a transaction in a blockchain does always require a valid key pair for the senders account. But in the case of HTE the daily transaction of pushing the balance of an account to the blockchain is not executed through the account holder, but is triggered by the transaction engine itself or from a smart-contract running somewhere on this engine.
So using a permissioned ledger as blockchain instance would normally mean that there exists a key pair for an administrator access (name SQL-push?) with the appropriate access rights to the ledger.
Of course other access identities would be useful too, such as for monitoring or for the regulators. That again is depending on a particular use-case.
But until we will have some use-cases a lot of additional research in this field is appropriate.


If I would start designing a hybrid transaction engine for everyday payments the core goal in the design would be consequently to achieve a short memory effect of the payments. In other world the ledger must contain a technology that allows to forget to whom I did spend my money after a while (a month, a year?), but also should meet the compliance and data integrity needs.

Out of this the database, that will contain all the data about the payment is the SQL database. I imagine that the transaction engine will then push on a daily basis the balance of each account to a distributed ledger that serves as a kind of backup instance of the SQL DB. Maybe this push information can contain some other information, as how many transactions that have been performed during the day. That's up to detailed specification and the use -case of such a hybrid system. Also it sounds reasonable not to push every single account every day, when noting happened on this particular account in the last 24 hrs.

What we are achieving in using a distributed ledger as described, is to have a kind of distributed backup of the most important part of a ledger, the balances of each account. And by doing this we get the freedom to delete or to consolidate the transaction records of the past, without to increase the risk do get into data integrity problems. Further advantages are, double check of data integrity; if the over all balance of the SQL DB and ledger are different it is obvious, there must be something wrong. If it wasn't an error in the software, eventually a SQL injection or another kind of attack could have been happen.

To avoid troubles with the regulatory it would be wise to dump, before deleting any old record in the SQL Database, the data into some backup file and prepare some easy restoring process. Just in the case that the information in the distributed ledger will not be detailed enough for certain circumstance.

Of course to create such hybrid transaction engine (HTE) is not an easy task and certainly some hassles would coming up during a future development. One challenge would be to synchronize a timeline between the two database instances. If this is not done very carefully it will be hard to perform a balance check between the two. Another important issue is how to handle the keys; a transaction in a blockchain does always require a valid key pair for the senders account. But in the case of HTE the daily transaction of pushing the balance of an account to the blockchain is not executed through the account holder, but is triggered by the transaction engine itself or from a smart-contract running somewhere on this engine.
So using a permissioned ledger as blockchain instance would normally mean that there exists a key pair for an administrator access (name SQL-push?) with the appropriate access rights to the ledger.
Of course other access identities would be useful too, such as for monitoring or for the regulators. That again is depending on a particular use-case.

There are other solutions to provide anonymus metal coin like payments, one are kinds of prechargeable wallets, or more protocol http://opentransactions.org or ledger/blockchain based Zerocash ones. But they are maybe not yet the best solution and hopefully we get one day something as a standard. Other  chaum implementation are https://taler.net and http://opencoin.org.

A similar approach to the hybrid model named "double decker blockchain" has been desribed from Gideon Greenspan in the blog of Multichain.

So it may be worth to do additional research in this field.



Leave a comment

You must be logged in to post a comment.