bitcoin engineering security

Developer diary: building a peer to peer bitcoin marketplace

So you want to build a bitcoin startup?

Building any reasonably sized web app is complex but building a bitcoin app requires an additional layer of code to interface with the blockchain and the expertise to secure it. A peer to peer marketplace includes additional challenges on top of that. The book can come later but for now here is a short developers brain dump for those crazy enough to go down this road. Have a cup of joe ready.

This article will be simple for developers who understand a  little bit of how bitcoin’s blockchain technology works – unconfirmed transaction, block, blockchain, confirmations etc… If you know about them, then you will swim through it 🙂

 

The Paxful example

Paxful took around 6-8 months to build a fully working bitcoin peer to peer marketplace.

This was for the MVP that has all the functionality that other the marketplaces already have. All of us have at least 5 years of software development experience including financial software for the biggest banks in Europe. This gave us some advantages in that the most common development bad practices were avoided and security was baked into the spine of the product rather than an annoying after thought as with most web apps.

Of course initially we planned to complete it in 2-3 months but as it usually is with software estimations to get realistic estimations you have to multiply it with Tambov constant 🙂

lunch-atop-skyscraper-new-york-construction-workers-crossbeam

Bitcoin technology stack basics

Regardless of your project being bitcoin 1.0 or 2.0 you still need to “talk” to blockchain network and execute operations. In talking with blockchain you could either directly deploy your full node and execute RPC commands against your full node’s Bitcoin Core (formerly BitcoinD) or use 3rd party providers like BlockCypherChain.com or others. Compared to couple of years ago todays 3rd party API providers are more reliable than some older ones out there who may have downtimes and funny bugs.

If you want to move with maximum speed then integrate with 3rd party API providers. In our case we didn’t want to rely on them because we had bad experiences with these old 3rd parties so we built our own simple bitcoin API server which separates the API server and application layer while still running full node Bitcoin Core. The more full nodes running Bitcoin Core the better!

We recommend this API server over some of the other implementations out there and not just because we wrote it. It does all the basics, has a very simple code structure and REST API endpoints and is built on the best supported PHP framework Laravel. Much more than a wrapper it does full database logging and can help track balance integrity. We also documented installing it step by step so even linux newbies can get it going and even included a security guide. Give it a shot.

 

Developing a simple and secure bitcoin web wallet

Maybe you think that it’s not what you planned to develop, but if you are doing something in bitcoin 1.0 you probably have to deal with users balances and therefore must have a wallet. This is a lot more work than it may seem because security must be of primary focus and is the foundation of this application.

Database Layer

The non bitcoin blockchain part in web wallet is internally managing user balances. Make sure you are familiar with SQL transaction isolation levels to avoid hackers trying to race condition you on withdrawals. Race conditions are like database layer “double spends”. They do two withdraws at the same time hoping that the balance will only be deducted once. Of course there are more solutions to avoid race conditions like queueing withdrawals or using synchronized methods if your technology stack allows that. Properly locking down your app on the SQL level makes it really bullet-proof. Use an ACID compliant full relational database like Postgres or other popular databases. Avoid NoSQL databases for financial applications in general, having mutexes isn’t 100% protection with databases that provide eventual consistency. Also have very strict validation rules and database constraints. Put database constraints like balance column(s) can’t be less than 0 value and if you have there any other business logic, for example a POS solution where user can set a tax amount, validate that the tax amount can’t be negative, otherwise malicious user can wreak havoc in your system.

onshore---construction-materials-engineering-and-testing

Blockchain Layer

Lets jump into the bitcoin/blockchain part. Important components here are:

  1. Generate bitcoin addresses for users,
  2. Receive payments,
  3. Send out bitcoins
  4. Confirm incoming transactions after n confirmations

 

Generating bitcoin address

That means you are basically generating a private key that will let you spend from that bitcoin address and also you can derive a public key from private key. The public key is the bitcoin address or “account number”. In case of a 3rd party API they can generate private key and bitcoin address for you so you would need to store them securely or at least encrypt them in the database. In case of using Bitcoin Core, use RPC commands to generate new bitcoin addresses. Bitcoin Core will handle pool of private keys for you. By using our simple API server, we are saving generated bitcoin addresses in a database which allows for faster lookup and match them with users.

Receive payments aka “HELLO WORLD”

This part relates to the bitcoin/blockchain and is fairly trivial yet the extreme lack of documentation on the topic made this almost like alchemy in the early days of bitcoin development. Getting this working is the equivalent of a “hello world” for blockchain engineering.

Low-rise building construction
The Blockchain is the future database structure of the entire internet, better start the “hello world”s now.

You are just listening on blockchain if there is an unconfirmed transaction coming to that address. With a 3rd party API you are providing them with your webhook/callback URL where the notification of a new unconfirmed transaction will be sent with all data related to it – amount of bitcoins, transaction hash and all the other data of a bitcoin transaction. A webhook/callback URL is the URL endpoint in your web app that a notification hits with data and therefore notifies that you that you received a transaction. In case of using Bitcoin Core or Paxful’s simple API server, you would need to edit  Bitcoin Core’s config file wallet notify callback URL too which is exactly same approach as with 3rd party providers. You can find more about setting up walletnotify in our documentation.

After your web app got hit with new incoming payment do your business logic for wallet. That feeling of accomplishment is great when you actually got your first walletnotify 🙂 It’s not recommended to confirm that amount as received, because unconfirmed bitcoin transaction in the network could still be double spent. Depending on your business risk factor you should decide on how many block confirmations you should set transaction as fully received and cleared in the network. Some exchanges require 6 confirmations, some require 3. We require 3 as well.

Confirm incoming transactions

How do you know if a transaction received 1 confirmation or 2 confirmations and so on? In a 3rd party providers case once you set up the web hook you can set on how many confirmations the notification will shoot. In case of Bitcoin Core, walletnotify shoots on 0 confirmation (once the transaction hits the mempool) and on the 1st confirmation. To track the  2nd, 3rd and so on you have to set up a callback URL for blocknotify. That is well documented in our API server. As for schema design you would probably keep unconfirmed transactions in one table and then move all confirmed transactions to a cleared transactions table.

When going the Bitcoin Core way you have an unconfirmed transaction and want to get the new number of confirmations, wait for blocknotify callback and when you get it, do an RPC call gettransaction for every unconfirmed transaction and you’ll have its new confirmations number.

IMPORTANT! Here is a risk for a race condition. In the bitcoin network 2 blocks can be found literally at the same time which means you will receive blocknotify twice right away and during your business logic operations some unknown consequences may happen. To avoid it your callback URL defined method should be executed synchronously, for example in a queue.

Send out bitcoins

Now you are ready to send out payments. Send out functionality is rather different when using 3rd party providers and Bitcoin Core. Each 3rd party has their own approach to send out funds. It’s because you have to provide them private keys for the addresses where you want to spend from but in Bitcoin Core’s case it’s internally managing all the private keys and automatically chooses the most optimal outputs to send from. When sending out bitcoins update your user balance accordingly. Be sure to use transactions and an ACID compliant relational database. Which database is best is a spicy topic and worth of it’s own paper or book.

We have excluded developing any kind of phone wallets, multisig solutions or bitcoin 2.0 projects as above described and only talked about bitcoin/blockchain specific functionality. Be aware that this blockchain relevant code base  is very tiny compared to the whole code base you would have to write for your app. The rest is just your industry specific business logic  and the code can be as complex as needed.

 

The peer to peer marketplace part

Here talking to bitcoin/blockchain network is not necessary and everything happens on the application level.

effectively_market_your_company_online_to_build_your_online_presence

Create offers/ads

Markets connect buyers and sellers and offers are the central objects that let them discover each other. Set your rules here. These decisions will affect everything and can dynamically change the marketplace culture and economy.

  1. Are payment methods hard-coded?
  2. Will you allow users to dynamically create new payment methods? Drop down or Stackoverflow tagging system approach?
  3. Set payment window time? I.e 90 minutes for buyer to complete payment or 3 days for a check?
  4. BTC price premium? I.e 10% over the average price across all exchanges?
  5. Which users can see which offers? I.e only logged in users with at least 10 completed trades, etc…

Additionally have all the CRUD (create, read, update, delete) operations for the offer and all the business logic like updating price every 10 minutes, activating, deactivating and so on.

The trading part

When users decide to accept an offer and begin a trade they are essentially in a live instant messaging session that has a process flow. With every point reached there are different rules and logic.

  • Start trade
  • Mark trade as paid
  • Complete trade
  • Auto-cancel trade
  • Cancel trade by user
  • Dispute trade
  • Leave feedback

For all these mentioned trade actions the business logic has to be extremely well thought through and code written well with all edge cases taken into account. Otherwise you can end up with adding balance twice, deleting accidentally records or simply break database schema. Not to mention that the entire process must MAKE SENSE to a first time buyer and guide them through step by step. This process cost us quite a few hairs on our heads. Good luck!

Trade live chat

This is a lot deeper than it may seem and depends how fancy you want to get. Are you going to use primitive AJAX long polling or 3rd party plugins like Pusher, Firebase or your own solution with web sockets, XMPP and OpenFire or abstractions like Faye or Converse.js. Consider also that you need moderator messages in case of dispute and anything else you see there.

File Upload and video

Media is an integral part of the trade process. Sellers want ID, receipts pictures and sometimes even a live video feed. Building these is tough but securing them doubles the complexity. We suggest going with the leading third parties here to avoid keeping any such media on your servers. Security first!

Feedback system

This is another critical part to keep your platform fair, safe and scammer free. The rules here have an EXTREME impact on market dynamics as reputation is the beating heart of any free market. What decisions shall you make? Leave feedback for every trade or per trade partner? Our approach is to leave feedback to every trade and a reply can be left too. In case of dispute a loser can’t leave feedback. We listened to our traders and built the feedback system for them. Win Win Win!

Reporting capabilities for any historical data/actions

That is what your users would require from you. Some of the power sellers will be doing alot of trades and need to report their profits, trades to their government and tax authorities. Make data generation and exporting very convinient for them. For historical data users want to see their completed trades, activity ledger, transactions ledger, generated bitcoin addresses and any other historical data there might be. This is a huge project in and of itself. A whole startup could be built helping pro traders track profits.

Backend admin for website health.

Because you are probably in custody of users funds you surely want to know at any moment how much bitcoin is kept in your system and what is total bitcoin balance, users balances in database, active trades, history and anything you can think of and you want to see it in real time.

Cold Storage

Lets not forget cold storage! Set watch addresses for your cold storage and think of procedures that you have to follow to refill hot wallet/move funds to cold storage. This is another deep topic and worthy of a long guide.

 

Conclusion

All the points above were focused on building a bitcoin peer to peer marketplace. We didn’t get into enterprise software and secure website development, unit tests, scalability, caching, replication, disaster recovery, build automation, backups, validations, ddos protection and defense against the most common hacks. Depending on your rush for MVP and how much of infrastructure or code organization/refactoring you are willing to do this could be a the beginning of a new age of financial freedom for humanity or another bitcoin disaster waiting to happen.

For secure web application development we have some tips shared in another blog post https://blog.paxful.com/7-security-tips-bitcoin-startups-must-not-ignore/ We will be sharing our security tips in an ongoing series. The more we develop the eco system and share knowledge the better the bitcoin world will be and that will lead to a better more paxful world.

Paxful means “peace” in Latin.

Facebooktwittergoogle_plusredditpinterestlinkedinmail

Comments

comments