SmartDec Beam Security Audit Jan2019
SmartDec Beam Security Audit Jan2019
SmartDec Beam Security Audit Jan2019
Disclaimer
The audit does not give any warranties on the security of the code. One audit cannot be
considered enough. We always recommend proceeding with several independent audits and
a public bug bounty program to ensure the security of the project. Besides, security audit is
not an investment advice.
Summary
The audit indicated that the source code of the project is of high quality, which attests to high
proficiency and experience of the developers. The development team was very responsive to
requests and suggestions; the issues were resolved promptly. All the findings were discussed
with the team and either fixed or justified.
General recommendations
The project is still under development, so we recommend performing regular code audit to
ensure correctness and security of newly added code.
Additionally, we recommend creating source code documentation. In some cases the overall
structure and readability of the code can be improved by refactoring.
The text below is for technical use only; it details the statements made
in Summary and General recommendations.
The project employs the latest advances in cryptography to ensure integrity and confidentiality
of the data, contained within. Based on Pedersen commitment idea, MimbleWimble protocol is
used to hide payment amounts as well as senders and receivers identities as the blockchain
stores no wallet addresses. Besides, Beam declares built-in support for time-locked
transactions, escrow transactions, atomic swaps and some other features, that can only be
implemented in other blockchains on the smart-contract level. Additionally, Beam uses a
number of techniques for compact blockchain to achieve a very moderate blockchain size as
compared to other blockchain implementations.
The original plan of the audit was to perform a traditional application security audit manually
and using tools with subsequent code review for the blockchain protocol, data storage and
processing. Actually, several stages of manual code review are done; each audit stage uses
the current version of the source code from GIT. The source code is written using C++ as the
primary programming language with some supporting libs in plain C. The source code has
total 190407 LOC ANSI C that is 71.74% of all the code and 73849 LOC in C++ that is 27.82%
of all the code. Beam in-house code has approximately 40261 (91.59%) LOC written in C++
and 3693 (8.40%) LOC written in ANSI C.
In the first stage of the audit the project was compiled. There were some unaccounted
dependencies and hardware-specific options, all of them were revealed. Boost was not
checked by cmake; -mavx gcc compiler option was available for x86_64 only. Compiler
warnings had to be analysed and fixed:
In the next stages analysis of the papers describing math foundations and algorithms was
performed. We searched and identified MimbleWimble and Bulletproof implementations.
Beam has its own Bulletproof implementation by the author (B. Bunz), which is available as a
fork (with a pull request) of libsecp256-zkp. During performing the Beam protocol
analysis we recovered the Beam message types and associated values, which were sent and
received by cross-referencing for each message type.
Procedure
The audit was performed in three stages, each of which was accompanied by checking the
latest version of the code. Repository link and exact commit hash are provided in the
corresponding section of the report.
The source code description was generated automatically with the use of Doxygen tool. The
description contains the inheritance graph and cross-references and is convenient for the
source code exploration.
It should be underlined that the yas library uses the header files of the boost library.
Utility directory of the project contains both the third-party code and the code of the project itself.
Recommendation: separate the third-party code from the project code by placing it, for
example, in the 3rdparty directory.
Recommendation: add the checking for the presence of boost library in the
CMakeLists.txt file.
Recommendation: use the -mavx option only for the x86_64 architecture.
After removal of the considered option from CMakeLists.txt, the compilation was finished
successfully.
utility/logger_checkpoints.cpp: In destructor
‘beam::Checkpoint::~Checkpoint()’:
utility/logger_checkpoints.cpp:82:32:
warning: ‘bool std::uncaught_exception()’ is deprecated
[-Wdeprecated-declarations]
if (_maxItems != 0 && std::uncaught_exception()) {
Moreover, the compiler generated warnings about comparison of signed and unsigned
numbers in the file pow/impl/crypto/equihash_impl.cpp.
Unused code
The beam/bss directory contains an obsolete component that is no longer used.
No IPv6 support
There is a definition of the StreamId structure with an anonymous union in the
p2p/types.hheader file :
union {
uint64_t u64;
struct {
uint32_t ip;
uint16_t port;
uint16_t flags;
} fields;
};
Recommendation: add the IPv6 support. This can be achieved by reserving 16 bytes for
the address. In the foreseeable future IPv4 will be definitely the major protocol version on
the Internet, but IPV6 support should not be discarded.
Recommendation: do not reset and copy objects with the memset function. One of the
possible ways to re-initialize an object is given below:
class A
{
A() { ... }
void reset()
{
*this = A();
}
};
void func()
{
PROC(10)
}
It is a correct syntax, however, it does not correspond to usual practices of macros usage in
C/C++.
Syntactically, this macro acts like an operator requiring a semicolon after it, so the code
becomes more uniform:
void func()
{
PROC(10);
}
Unsigned overflow
block_crypt.cpp:
return (uint32_t)duration_cast<milliseconds>(system_clock::n
ow().time_since_epoch()).count();
Cryptographic hash-function
• Signing transactions and/or messages. First, the hash function of the message is
calculated. This calculated value is used as an input value in the asymmetric
cryptography method for electronic signature.
• Proving that a transaction is stored in the blockchain without downloading the entire
blockchain. In order to accomplish this, transactions in each block are organized into a
Merkle Tree data structure. This is a binary tree with transactions stored in its leaves. In
the nodes of the tree there is a hash function from the sons of each node.
• This means that if the tree height equals to h, then the transaction location in the tree is
defined by a chain of hashes, starting from the leaf, where the transaction is stored, and
ending with the tree root.
• In some blockchains the hash function is used to link blocks into a chain. For example,
in the Bitcoin network each block contains a hash function from the previous block.
The Beam blockchain mainly uses SHA256 hash function. This is a second generation hash
function and at the present moment the use of it is considered to be completely safe, since
no practical collision finding algorithms have been found for it yet.
The BLAKE2 hash function is used in the Proof-of-Work scheme. The BLAKE2 version
implemented a 256-bit hash in the Beam outputs as a result.
At the present moment BLAKE2 is considered to be safe for use. There are no working
methods that allow significantly reducing the brute force to find a collision.
Methods for significantly reducing brute force to find the preimage are also unknown.
Asymmetric cryptography
The Beam project uses asymmetric cryptography on elliptic curves provided by the
secp256k1 standard library.
The source code of the project (core/ecc.cpp and others) uses constants (for instance, the
order of a group of points on the elliptic curve) that coincide with the standard values.
Consensus
The Beam project uses the Proof-of-Work concept to obtain consensus. That means the
network nodes must perform computationally complex tasks to add a new block to the
blockchain (mine block).
The Beam project exploits the Equihash Proof-of-Work scheme. The authors of the scheme
assumed that the schemes with a large number of RAM would be economically unprofitable.
The Equihash scheme is based on the problem of the generalized birthday paradox.
To solve the problem with maximum performance, a large amount (approx. 1GB) of RAM is
required. Performance falls exponentially with the decreasing RAM requirements.
However, recently there have been reports, that Equihash Proof-of-Work scheme is “hacked”
in terms of ASIC-realisation resistance since ASIC chip about 100 times more efficient than
the software implementations has been developed.
Information hiding
Transactional blockchain, by their nature, reveal all the information about transactions: sender
address, receiver address, and amounts. This allows tracking the fund movements. Although
only addresses take part in transactions and the owner of the address does not appear in
transactions explicitly, this is just a quasi-anonymity, since if the owner of the address is
established from the other source, anonymity is lost.
The Beam project implements the MimbleWimble protocol that allows hiding amounts
transferred in transactions. However, as before, anyone can make sure all the transactions
are valid. MimbleWimble is based on a scheme named Pedersen Commitment. In the original
Pedersen Commitment scheme it is possible to specify huge amount causing integer overflow,
effectively producing money out of thin air. To prevent this attack we limit the transferred
amount to some reasonable max value. A Zero Knowledge Proof (ZKP) is attached to each
commitment, so anyone can ensure that the amount is valid without disclosing it. Therefore, a
Zero Knowledge Proof that the amount in the commitment satisfies the constraints is attached
to each commitment (Rangeproof).
Prover is the side that wants to prove that it has some information without revealing this
information. For example, the prover wants to prove to everybody that the number x lies in the
interval [a;b) without actually revealing x.
Verifier is the other side of the proof. The verifier wants to make sure that the other party
actually has the information.
ZKPs are usually formulated as interactive proofs, that is, the verifier asks some “questions” to
which the prover answers. In considered process we are interested in minimizing the number
of rounds and the amount of information transmitted between the parties, both in one round
and total.
The general scheme of Bulletproof Zero Knowledge Proofs, which is used in the Beam
blockchain, has the following advantages:
• Bulletproof requires significantly less memory compared to previous schemes. Proof of
belonging to a 64-bit range requires 675 bytes.
• When transitioning from a single number to a vector, the amount of required memory
grows not linearly, but logarithmically. That is, the proof of belonging of two numbers to a
64-bit range requires 739 bytes, four – 803 bytes.
• This means that different range proofs can be merged into one vector range proof in
order to decrease the amount of required memory for the proof.
• When range proof is used in the blockchain with the UTXO model, it is possible to
remove range proof for transaction outputs wasted in other transactions.
• It is required to store only 32 bytes of range proof in wasted transaction.
The idea of ZKP is in transition from scalar commitments to vector ones and in calculation of
the scalar product of vector commitments.
Interactive proof can be converted to a noninteractive one by using Fiat-Shamir heuristic. For
this heuristic interactive proof should have a public coin property, i.e. requests from the verifier
to the prover must be generated randomly and uniformly from the predefined set of requests.
Instead of the verifier, who makes a request to the prover, the prover refers to the “random
oracle” model. It applies a cryptographic hash function from some state, which possibly
includes the previous response from the prover to the verifier. In this way a request to the
prover is formed. It is computationally difficult for the prover to form a request that is
advantageous for himself, since this would require finding the preimage of some value of
cryptographic hash function.
Beam realization
Most of the cryptographic primitives are implemented in the secp256k1-zkp library
(https://github.com/ElementsProject/secp256k1-zkp). This library is based on secp256k1
from Bitcoin (https://github.com/bitcoin-core/secp256k1). The Beam project exploits
cryptographic primitives on the elliptic curves, SHA256 hashing.
The BLAKE2 hash function used in the Proof-of-Work scheme is located in the utility/crypto/
blake directory. The code in the considered directory is similar to the official BLAKE2 code
(https://github.com/BLAKE2/BLAKE2) in the ref and sse directories.
In the core directory there is a realization of the AES symmetric cipher (aes.h, aes.cpp).
The author of the realization is Christophe Devine.
There is almost identical code in other repositories (for example, in
https://github.com/eddelbuettel/digest). Methods implemented in C++ and written in the Beam
project are provided at the end of the file.
https://www.apache.org/licenses/GPL-compatibility.html:
The Proof-of-Work Equihash scheme is located in the beam/pow directory. These files are
apparently taken from the ZCash project (https://github.com/zcash/zcash).
The implementation (within the Beam project from scratch) of the range proof according to the
Bulletproof scheme is given in the beam/core/ecc_Bulletproof.cpp file.
The author’s implementation of the Bulletproof and MimbleWimble protocols is available on github
(https://github.com/apoelstra/secp256k1-mw).
Moreover, in the official secp256k1-zkp repository there is a merge request from the author of
Bulletproof.
Other found errors and notes on the source code are presented below:
1. _countof macro is used incorrectly in the ecc.cpp file. _countof("STR") is always
equal to sizeof(void*) since "STR" is casted to the const char * type. The
following usage is the correct one:
_countof(str);
2. In different source files in different namespaces the uintBig type is defined with a
different number of bits in the integer value. In the core/block_crypt.h file in the
AmountBig structure it has the size of 128 bits and in the core/ecc.h file in the
ECC namespace it has the size of 256 bits.
3. In the ecc.cpp file the memory for ECC::Context singletone is allocated in the
following way:
That means the size of the allocated array is a multiple of eight and sufficient for storing
ECC::Context.
The use of low-level memory manipulation appears to be unjustified. The get method
for obtaining an object reference can be implemented in the following way:
Thus, according to C++11 standard, the constructor of the cntx object will be called
with the first call of the get method and in thread-safe manner. Accordingly, the
functionality implemented in the InitializeContext should be moved to the
constructor.
0x00, Config
Transferred data:
• ECC::Hash::Value CfgChecksum;
• bool SpreadingTransactions;
• bool Bbs;
• bool SendPeers;
Message sent at:
• wallet/wallet_network.cpp
WalletNetworkIO::WalletNodeConnection::OnConnectedSecure()– commented out
• core/proto.cpp
FlyClient::NetworkStd::Connection::OnConnectedSecure()
• node/node.cpp
Node::Peer::OnConnectedSecure()
0x01, Bye
Transferred data:
• uint8_t Reason;
Message sent at:
0x02, Ping
Transferred data: none
Message sent at:
• core/proto.cpp
FlyClient::NetworkStd::Connection::SendRequest
Message processed at:
• node/node.cpp
Node::Peer::OnMsg – just answers with Pong
0x03, Pong
Transferred data: none
Message sent at:
• node/node.cpp
Message processed at: none
0x04, SChannelInitiate
Transferred data:
• ECC::uintBig NoncePub
Message sent at:
• core/proto.cpp
NodeConnection::SecureConnect
Message processed at:
• core/proto.cpp
NodeConnection::OnMsg
0x05, SChannelReady
Transferred data: none
Message sent at:
• core/proto.cpp
NodeConnection::OnMsg
0x06, Authentication
Transferred data:
• PeerID ID
• uint8_t IDType
• ECC::Signature Sig
Message sent at:
• core/proto.cpp
NodeConnection::ProveID(ECC::Scalar::Native& sk, uint8_t nIDType)
Message processed at:
• wallet/wallet_network.cpp – the whole logic is commented out
• node/node.cpp
Node::Peer::OnMsg(proto::Authentication&& msg)
• core/proto.cpp
NodeConnection::OnMsg(Authentication&& msg)
0x07, PeerInfoSelf
Transferred data:
• uint16_t Port
Message sent at:
• node/node.cpp
Node::Peer::OnConnectedSecure()
Message processed at:
• node/node.cpp
Node::Peer::OnMsg(proto::PeerInfoSelf&& msg)
0x08, PeerInfo
Transferred data:
• PeerID ID
• io::Address LastAddr
Message sent at:
• node/node.cpp
Node::Peer::OnResendPeers()
Obtained from PeerMan::PeerInfo m_This.mPeerMan
0x09, GetExternalAddr
Transferred data: none
Message sent at: none
• processed at:
• node/node.cpp
Node::Peer::OnMsg(proto::GetExternalAddr&& msg)
0x0a, ExternalAddr
Transferred data:
• uint32_t Value
Message sent at:
• node/node.cpp
Node::Peer::OnMsg(proto::GetExternalAddr&& msg)
Message processed at: none
0x0b, GetTime
Transferred data: none
Message sent at: none
Message processed at:
• node/node.cpp
Node::Peer::OnMsg(proto::GetTime&& msg)
0x0c, Time
Transferred data: none
Message sent at:
• node/node.cpp
Node::Peer::OnMsg(proto::GetTime&& msg)
Message processed at:
• node/unittest/node_test.cpp – test code
MyClient::OnMsg(proto::Time&& msg) – empty function body
0x0e, Boolean
Transferred data:
• bool Value;
Message sent at: none
Message processed at:
• wallet/wallet_network.cpp – commented out
0x10, NewTip
Transferred data:
• Block::SystemState::Full Description
Message sent at:
• node/node.cpp
Node::Peer::OnConnectedSecure()
Node::Processor::OnNewState()
Message processed at:
• wallet/wallet_network.cpp – commented out
• core/proto.cpp
FlyClient::NetworkStd::Connection::OnMsg(proto::NewTip&&
msg)– simple context-free state validation
• node/node.cpp
• Node::Peer::OnMsg(proto::NewTip&& msg) – new state is received and validated in
Node::Processor::OnState. Updates sender’s rating if the state is valid and calls the
Node::RefreshCongestions() function.
0x12, Hdr
Transferred data:
• Block::SystemState::Full Description
Message sent at:
• node/node.cpp
Node::Peer::OnMsg(proto::GetHdr&& msg)
Message processed at:
• node/node.cpp
Node::Peer::OnMsg(proto::Hdr&& msg)
0x13, GetHdrPack
Transferred data:
• Block::SystemState::ID Top
• uint32_t Count
Message sent at:
• node/node.cpp
Node::TryAssignTask(Task& t, Peer&, p)
Message processed at:
• node/node.cpp
Node::Peer::OnMsg(proto::GetHdrPack&& msg)
0x14, HdrPack
Transferred data:
• Block::SystemState::Sequence::Prefix, Prefix
std::vector<Block::SystemState::Sequence::Element>,vElements
Message sent at:
• node/node.cpp
Node::Peer::OnMsg(proto::GetHdrPack&& msg)
0x15, GetBody
Transferred data:
• Block::SystemState::ID ID
Message sent at:
• node/node.cpp
Node::Peer::TryAssignTask(Task& t, Peer& p)
Message processed at:
• node/node.cpp
Node::Peer::OnMsg(proto::GetBody&& msg)
0x16, Body
Transferred data:
• ByteBuffer Perishable;
• ByteBuffer Ethernal;
Message sent at:
• node/node.cpp
Node::Peer::OnMsg(proto::GetBody&& msg)
Message processed at:
• node/node.cpp
Node::Peer::OnMsg(proto::Body&& msg)
0x17, GetProofState
Transferred data:
• Height Height
Message sent at:
• node/unittests/node_test.cpp – test code
MyClient::OnMsg(proto::NewTip&& msg)
Message processed at:
• node/node.cpp
Node::Peer::OnMsg(proto::GetProofState&& msg)
0x19, GetProofKernel
Transferred data:
• Merkle::Hash ID;
Message sent at:
• node/node.cpp
Node::Peer::TryAssignTask(Task& t, Peer& p)
Message processed at:
• node/node.cpp
Node::Peer::OnMsg(proto::GetProofKernel&& msg) – one might send empty
proto::ProofKernel
0x1a, ProofKernel
Transferred data:
• TxKernel::LongProof Proof
Message sent at:
• node/node.cpp
Node::Peer::OnMsg(proto::GetProofKernel&& msg)– one might send empty
proto::ProofKernel
• wallet/unittests/wallet_test.cpp – test code
Client::OnMsg(proto::GetProofKernel&& msg)
Message processed at:
• wallet/wallet_network.cpp
WalletNetworkIO::WalletNodeConnection::OnMsg2
(proto::ProofKernel&& msg) – commented out
• node/unittests/node_test.cpp – test code
MyClient::OnMsg(proto::ProofKernel&& msg)
0x1c, ProofUtxo
Transferred data:
• std::vector<Input::Proof> Proofs
Message sent at:
• node/node.cpp
Node::Peer::OnMsg(proto::GetProofUtxo&& msg)
• wallet/unittests/wallet_test.cpp – test code
TestBlockchain::GetProof(const proto::GetProofUtxo& data,
proto::ProofUtxo& msgOut)
Client::OnMsg(proto::GetProofUtxo&& msg)
Message processed at:
• node/unittests/node_test.cpp – test code
MyClient::OnMsg(proto::ProofUtxo&& msg)
0x1d, GetProofChainWork
Transferred data:
• Difficulty::Raw LowerBound
Message sent at:
• core/proto.cpp
FlyClient::NetworkStd::Connection::RequestChainworkProof()
• node/unittests/node_test.cpp – test code
MyClient::OnMsg(proto::NewTip&& msg)
Message processed at:
0x1e, ProofChainWork
Transferred data:
• Block::ChainWorkProof Proof
Message sent at:
• node/node.cpp
Node::Peer::OnMsg(proto::GetProofChainWork&& msg)
• wallet/unittests/wallet_test.cpp – test code
Client::OnMsg(proto::GetProofChainWork&& msg)
Message processed at:
• node/node.cpp
Node::Peer::OnMsg(proto::ProofChainWork&& msg)
• core/proto.cpp
FlyClient::NetworkStd::Connection::OnMsg(proto::ProofChainWork&& m
sg)
• node/unittests/node_test.cpp – test code
MyClient::OnMsg(proto::ProofChainWork&& msg)
0x20, MacroblockGet
Transferred data:
• Block::SystemState::ID ID
• uint8_t Data
• uint64_t Offset
Message sent at:
• node/node.cpp
Node::SyncCycle(Peer& p)
Message processed at:
• node/node.cpp
Node::Peer::OnMsg(proto::MacroblockGet&& msg)
0x21, Macroblock
Transferred data:
• Block::SystemState::ID ID
0x22, GetCommonState
Transferred data:
• std::vector<Block::SystemState::ID> IDs
Message sent at:
• node/unittests/node_test.cpp – test code
• MyClient::OnMsg(proto::NewTip&& msg)
Message processed at:
• node/node.cpp
Node::Peer::OnMsg(proto::GetCommonState&& msg)
Comment: message processing throws exception when any of the received states has
a lower height than the genesis block. As a result, one can form incorrect message to
crash the node.
0x23, ProofCommonState
Transferred data:
• uint32_t iState
• Merkle::HardProof Proof
Message sent at:
• node/node.cpp
Node::Peer::OnMsg(proto::GetCommonState&& msg)
Message processed at: none
0x24, GetProofKernel2
Transferred data:
• Merkle::Hash ID
• bool Fetch
Message sent at:
• node/unittests/node_test.cpp – test code
MyClient::OnMsg(proto::NewTip&& msg)
0x25, ProofKernel2
Transferred data:
• Merkle::Proof Proof
• Height Height
• TxKernel::Ptr Kernel
Message sent at:
• node/node.cpp
Node::Peer::OnMsg(proto::GetProofKernel2&& msg)
Message processed at:
• node/node.cpp
Node::Peer::OnMsg(proto::ProofKernel2&& msg)
0x28, GetMined
Transferred data:
• Height HeightMin
Message sent at:
• node/unittests/node_test.cpp – test code
MyClient::OnMsg(proto::NewTip&& msg)
Message processed at:
• node/node.cpp
Node::Peer::OnMsg(proto::GetMined&& msg)
0x29, Mined
Transferred data:
• std::vector<PerMined> Entries
Message sent at:
• node/node.cpp
Node::Peer::OnMsg(proto::GetMined&& msg)
Message processed at: none
0x2a, Recover
Transferred data:
0x2b, Recovered
Transferred data:
• std::vector<Key::IDV> Private
• std::vector<Key::IDV> Public
Message sent at:
• node/node.cpp
Node::Peer::OnMsg(proto::Recover&& msg)
Message processed at:
• node/unittests/node_test.cpp – test code
MyClient::OnMsg(proto::Recovered&& msg)
0x30, NewTransaction
Transferred data:
• Transaction::Ptr Transaction
• bool Fluff
Message sent at:
• node/node.cpp
Node::Peer::SendTx(Transaction::Ptr& ptx, bool bFluff)
Message processed at:
• node/node.cpp
Node::Peer::OnMsg(proto::NewTransaction&& msg)
0x31, HaveTransaction
Transferred data:
• Transaction::KeyType ID
Message sent at:
• node/node.cpp
Node::Peer::OnMsg(proto::Config&& msg)
0x32, GetTransaction
Transferred data:
• Transaction::KeyType ID
Message sent at:
• node/node.cpp
Node::WantedTx::OnExpired(const KeyType& key)
Node::Peer::OnMsg(proto::HaveTransaction&& msg)
Message processed at:
• node/node.cpp
Node::Peer::OnMsg(proto::GetTransaction&& msg)
0x38, BbsMsg
Transferred data:
• BbsChannel Channel
• Timestamp TimePosted
• ByteBuffer Message
Message sent at:
• node/node.cpp
Node::Peer::SendBbsMsg(const NodeDB::WalkerBbs::Data& d)
Message processed at:
• core/proto.cpp
FlyClient::NetworkStd::Connection::OnMsg(proto::BbsMsg&& msg)
• node/node.cpp
Node::Peer::OnMsg(proto::BbsMsg&& msg)
• wallet/wallet_network.cpp
WalletNetworkViaBbs::OnMsg(const proto::BbsMsg& msg)
WalletNetworkViaBbs::BbsSentEvt::OnMsg(proto::BbsMsg&& msg)– calls
parent’s function OnMsg
0x39, BbsHaveMsg
Transferred data:
0x3a, BbsGetMsg
Transferred data:
• BbsMsgID Key
Message sent at:
• node/node.cpp
Node::Peer::OnMsg(proto::BbsHaveMsg&& msg)
Node::Bbs::WantedMsg::OnExpired(const KeyType& key)
Message processed at:
• node/node.cpp
Node::Peer::OnMsg(proto::BbsGetMsg&& msg)
0x3b, BbsSubscribe
Transferred data:
• BbsChannel Channel
• Timestamp TimeFrom
• bool On
Message sent at:
• core/proto.cpp
FlyClient::NetworkStd::BbsSubscribe(BbsChannel ch, Timestamp ts,
IBbsReceiver* p)
FlyClient::NetworkStd::Connection::OnMsg(proto::Config&& msg)
Message processed at:
• node/node.cpp
Node::Peer::OnMsg(proto::BbsSubscribe&& msg)
0x3c, BbsPickChannel
Transferred data: none
Message sent at: none
Message processed at:
0x3d, BbsPickChannelRes
Transferred data:
• BbsChannel Channel
Message sent at:
• node/node.cpp
Node::Peer::OnMsg(proto::BbsPickChannel&& msg)
Message processed at: none
struct BbsSubscribe
{
static const uint8_t s_Code = 0x3b;
BbsChannel m_Channel;
Timestamp m_TimeFrom;
bool m_On;
BbsSubscribe(Zero_ = Zero)
{
ZeroInit(m_Channel);
ZeroInit(m_TimeFrom);
ZeroInit(m_On);
}
BbsSubscribe(Uninitialized_)
{
}
BbsSubscribe(BeamNodeMsg_BbsSubscribe(
};
struct BbsSubscribe_NoInit :public BbsSubscribe
{
BbsSubscribe_NoInit() : BbsSubscribe(Uninitialized) {}
};
The last argument of the constructor (Unused) is required to bypass syntax limitations of C
macros (one must place comma after each parameter).
Template InitArg method is used to initialize fields. This method is additionally specialized
to correctly process unique_ptr.
Structure
This module includes the libuv folder with implementation of this library.
Headers:
• address.h
• bufferchain.h
• coarsetimer.h
• fragment_writer.h
• mempool.h
• sslio.h
• sslstream.h
• tcpstream.h
• asyncevent.h
• buffer.h
• errorhandling.h
• libuv.h
• reactor.h
• sslserver.h
• tcpserver.h
• timer.h
Implementation:
• address.cpp
• bufferchain.cpp
• coarsetimer.cpp
• fragment_writer.cpp
• sslio.cpp
• sslstream.cpp
• tcpstream.cpp
• asyncevent.cpp
• buffer.cpp
• errorhandling.cpp
class Reactor
reactor.h
The class wraps the libuv library. It includes private fields:
• _loop – handler for Eventloop in libuv
• _stopEvent – stop event EventLoop
• _handlePool– handlers pool in EventLoop
• bool _creatingInternalObjects = false
• std::unique_ptr<PendingWrites> _pendingWrites
• std::unique_ptr<TcpConnectors> _tcpConnectors
• std::unique_ptr<TcpShutdowns> _tcpShutdowns
reactor.h includes private functions that use internal class Object to wrap commonly used
libuv function. Object has libuv handler and pointer to Reactor. The code that interacts
with Reactor allocates a lot of internal class objects. It stores pointers to these new objects in
EventLoop handlers. Reactor’s destructor calls uv_close() for every handler.
Reactor.cpp
This file includes TcpConnectors, TcpConnectors, and PendingWrites classes.
Reactor uses these classes to store TCP connections and data awaiting for sending.
class Address
address.h
The class wraps IP address and port. It can be converted to sockaddr_in.
Functions:
• SharedBuffer normalize(const SerializedMsg& msg, bool
makeUnique=false)– normalizes data to SharedBuffer. It makes buffer unique if it
is necessary.
• textttSharedBuffer map_file_read_only(const char* fileName) – makes memory map of
the file to the buffer
class BufferChain
bufferchain.h
The class stores a chain of buffers (vectors of the IOVec classes) and the pointer to the
empty IOVec.
timer.h
Functions:
• Create – stores raw pointer to the object in the EventLoop
• Start – sets a callback and starts the timer by calling restart function
• Restart – directly interacts with Reactor and starts timer at the EventLoop level,
setting callback in the same indirect way as AsyncEvent
• Cancel – cancels the timer if it is in the same thread
class MemPool
mempool.h
The class is used for storing handlers in the EventLoop. In the destructor it frees memory
pointed to by the stored pointers. The data field in the structure stores the pointer to the
Reactor::Object.
coarsetimer.h
This class allows setting/resetting several timers with different times, but with the same
callback. Only one timer can work at the same time, since the Timer object is changed by
every call of the set_timer and cancel functions, which are responsible for timers in
EventLoop. However, the class emulates the work of several timers simultaneously. It stores a
queue of timers with the time when they should start. When an internal timer of the
Timer object goes off the internal callback function checks if there are timers in the queue that
should go off. If so, the external callback function is called and the timer id is passed to it.
TIMER_ACCURACY constant allows setting the tolerance for the go off time.
class MultipleTimers
coarsetimer.h
The class implements the logic for multiple timers with different times and callbacks via
CoarseTime class. It stores a set of pointers to callbacks for each timer id. It uses internal
callback for choosing the corresponding external callback. Timer work logic is similar to the
logic in the CoarseTimer class.
fragment_writer.h
The class writes data to a fragment (instance of the SharedMem class). When the entire
fragment is filled the writer passes it to the external callback via move semantics. The
constructor defines the fragment size, header size, and callback.
Public functions:
• void* write(const void *ptr, size_t size)
• void finalize()
class TcpStream
tcpstream.h
The interface is well documented and its logic is clear. There are functions for reading/writing
from/to stream, retrieving information about connection, turning on/off reading from the
stream, and closing the stream.
The _reactor field is not initialized anywhere, though this field is referenced in the code. The
Reactor::init_tcpstream and uv_tcp_init functions are not used. Stream in the
EventLoop is not initialized.
The close function from the interface is not implemented. It was apparently replaced with
the shutdown function.
class SSLContext
sslio.h
The class implements context for the SSL connection and context for the server (with the
certificate and private key) and for the client.
sslio.h
The class implements interface for SSL data encryption/decryption. The
bool SSLIO::new_data_from_stream function is implemented instead of
Result SSLIO::on_encrypted_data_from_stream.
The implementation uses the client variable that is not defined (probably, it is defined in
openssl, but it does not appear that there is such a variable). Perhaps, the authors wanted to
use SSL* _ssl.
class SslStream
sslstream.h
The class inherits from TcpStream which causes its problems with initialization. It is not
initialized in the EventLoop.
It overrides the functions for writing into stream via SSLIO only.
The class does not implement functions for reading encrypted data from the stream.
The on_decrypted_data function calls TcpStream::on_read. However, no functions are
passed to the EventLoop (read_cb is not overridden) for encrypted input data processing.
class TcpServer
tcpserver.h
This class implements SSL server with the callback function that will be called on every new
connection with the server. The considered class is initialized in Reactor in the same way as
the other Reactor::Object inheritors described above.
The only interesting function is the server creation function:
class SslServer
sslserver.h
This class implements SSL server with the callback function that will be called on every new
connection with the server. The considered class inherits TcpServer and is initialized in
Reactor in the same way as the other Reactor::Object inheritors described above.
db.h / db.cpp
The NodeDB class allows the node to work with the sqlite3 database. The database is
opened via the NodeDB::open function. The DB structure is created by calling the
NodeDB::create function.
Constant variables are defined at the beginning of the db.cpp file. The variables contain the
names of the DB tables/fields that will be used in the queries.
Low-level creating/processing requests to the database is made with the use of the
NodeDB::Recordset class. The NodeDB::Recordset::Reset(NodeDB&,
Query::Enum, const char*) function sends a request to the database. The second
parameter determines the cell of the NodeDB::m_pPrep array in which the request handler,
defined by the sqlite3_prepare_v2, will be saved.
In the Query structure the enum is specified. It is used to separate queries. For example,
setting Recordset for getting all the peers will be done with the following command:
In the NodeDB class there are functions for working with various entities necessary for node
operation (states, peers, events, blocks, transaction cores, etc.). The
NodeDB::Transaction class implements the minimal functionality for working with
database transactions.
txpool.h / txpool.cpp
It implements the TxPool transaction pool class.
The internal class TxPool::Profit describes the profit from the transaction. It stores
transaction fee and its size. Operator < is implemented for the profits comparison.
In both classes Element moved the pointer from beam/core to Transaction and does not
allocate free memory for it in a heap. Therefore, the transaction should live more than the
TxPool instance.
Fluff monitors the memory allocation/release for Fluff::Element in the following way: a
new element is created in a heap and initialized in the AddValidTx function. After that
pointers to subclasses are written in multisets, the address of the parent element is calculated
in any function that removes the subclass from the multiset and the corresponding memory in a
heap is released.
Stem does not create new Stem::Element instances. Pointer to Stem::Element is passed
to the functions that add subclasses. Probably, it is implied that the ownership of the object is
transferred to the Stem instance at that moment, since when the element is deleted, the
memory is released.
Stem has the functionality to control timers. In the callback for the timers (Stem::OnTimer)
only the next timer is activated, with timeout taken from multiset that stores time for Element.
When the timer is created, Element records the response time of the timer. No other use of
the imers or their interaction has been detected.
processor.h / processor.cpp
These files contain realisation of the NodeProcessor class. This class is an interlayer
between the node, which communicates with the network, and the internal database. The class
takes prepared data (states, blocks, transactions, inputs, and outputs) as the input, validates
them with the cryptography from beam/core, if needed, and writes them to the database.
It also gets data from the database and forms the corresponding classes, implements the
NodeProcessor::Cursor class that stores current blockchain processor state.
NodeProcessor leaves event handlers (such as OnNewState) empty. They are also
overridden in Node::Processor. In addition, the VerifyBlock function complementing the
block checking procedure is overridden. NodeProcessor implements only transaction context
validation (assuming that the validation of the transaction itself is done) and height of the block,
where the transaction is written.
The NodeProcessor::Rollback() function rolls back one block, invokes block check on
the current cursor state, and calls the OnCorrupted() function if the block is invalid.
The OnState and OnBlock functions write new objects to the database and call the
OnStateData and OnBlockData callbacks respectively.
The IBlockWalker class is implemented for walking the blocks. Child classes must
implement the OnBlock handler.
The NodeProcessor::EnumBlocks function walks through all the blocks till the last
macroblock inclusive and passes them to the IBlockWalker::OnBlock functon. It is used
by the classes described above as well as by the InitializeFromBlocks function.
Validation
Validation is performed in the following functions:
• HandleBlockElement
• OnStateInternal
• VerifyBlock
• ApproveState
• ImportMacroBlockInternal
• ValidateTxContext
• ValidateTxWrtHeight
• HandleBlock
2. If bFwd == false
a) bool bCreate = true
b) Corresponding UTXO is searched:
p = m_Utxos.Find(cu, key, bCreate)
the key has the same commitment and maturity as the input
c) If bCreate == true
i. Count of the found UTXO is set to 1
d) Otherwise
i. Count of the found UTXO is incremented
ii. The tree search cursor is invalidated
NodeProcessor::HandleBlockElement(const Output& v,
Height h, const Height* pHMax, bool bFwd)
The function processes input :
1. UtxoTree::Key::Data is created
a) UtxoTree::Key::Data = d
4. When pHMax != NULL, it is checked that maturity d < maturity v; maturity d is set to
maturity
7. If bFwd == true
a) If bCreate == true
i. count UTXO is set to 1
b) Otherwise
i. Input::Count nCountInc is set to count UTXO plus 1
ii. If nCountInc == 0 (i.e. the overflow is present), then the function returns
false
iii. count UTXO is set to nCountInc
8. If bFwd == false
a) If count UTXO is 1, then the UTXO is removed from the tree
b) Otherwise count is decremented
TODO. The flag logic is unclear. What does the bFwd flag really mean and why are
diametrically opposed actions taken with the found UTXO for its different values?
NodeProcessor::OnStateInternal(const
Block::SystemState::Full& s, Block::SystemState::ID&
id)
The function validates the received state s:
1. It receives state id and writes it by reference id.
3. The function checks if the state timestamp is higher than the internal one by no more
than Rules::get().TimestampAheadThreshold_s. Otherwise, it returns
DataStatus::Invalid
5. The function checks if the state height is not lower than the lowest cursor horizon.
Otherwise, it returns DataStatus::Unreachable
bool NodeProcessor::VerifyBlock(const
Block::BodyBase& block, TxBase::IReader&& r, const
HeightRange& hr)
The function calls Block::BodyBase::IsValid function from the
beam/core/block_crypt module.
Node::Processor::VerifyBlock(const Block::BodyBase&
block, TxBase::IReader&& r, const HeightRange& hr)
The function validates the block:
1. It receives the number of verification threads
3. If the number is greater than zero, then the function works with the following object: v =
m_Verifier (Node::Processor::Verifier)
6. The function creates a block verification task and waits until it finishes
Verification in Node::Processor::Verifier
The thread function makes the checks in the thread:
1. It creates new std::unique_ptr<Verifier::MyBatch> p(new
Verifier::MyBatch)
3. The function holds mutex m_Mutex and waits for a new task
7. It executes
10. If the check in the item 6 is valid and no other thread sets the value of shared variable
m_bFail to true, then the current context is merged with the general context
12. If bValid is false, then it sets the shared variable m_bFail to false. That means the
part of the context is invalid. Thus the whole task is marked as invalid
NodeProcessor::ImportMacroBlockInternal(
Block::BodyBase::IMacroReader& r)
The function imports and verifies the macroblock. It does not add the macroblock to the DB.
Node class in Node::ImportMacroblock function does it after successfull call of the
current function.
1. The function checks if the height of the macroblock beginning is greater than the height
of the cursor by one
2. It checks if the hash of the state is equal to the hash of the block previous to the current
macroblock
3. It checks if the maturity of the initial state’s kernel is not greater than the height of the
cursor
5. If the cursor height is greater than the height of the genesis block, it fills cmmr with the
data from the DB
6. The function traverses all the headers and the prefixes in the macroblock and fills cmmr,
cmmrKrn (currently disordered), increases difficulty, validates every state and adds it
to the DB (it returns false if at least one state is invalid), checks if the received root
ccmrKrn corresponds to the state’s kernel at every step
9. The function compares the definition of the resulting state with the one gained from
cmmr
10. If definitions are different, it calls the HandleValidatedBlock function again with the
bFwd flag set to false
13. For each prefix it looks for the corresponding state in the DB
14. It removes block corresponding to the state (apparently, if the usual block was loaded
instead of the macroblock for this state)
15. The function places the kernel of the final state in the DB
16. It recognizes the UTXO of the owner using the RecognizeUtxos function
NodeProcessor::ValidateTxWrtHeight(const
Transaction& tx,
Height h)
The function checks if the height of each transaction’s kernel is in the allowed range using
the function from beam/core/block_crypt.
NodeProcessor::ValidateTxContext(const Transaction&
tx)
The function validates the context of the transaction:
1. The function checks the transactions height relative to the cursor using the
NodeProcessor::ValidateTxWrtHeight function
3. The traveller t is created; it supports the counter that will be used for the search of
every input’s UTXO
4. For the every input it is checked if there is no input with the same commitment in the
transaction
5. t is set in such a way that only UTXO with the commitment, same as the input’s one, is
searched among all the UTXO’s (with maturity set from 0 to the height of the
cursor + 1)
7. If all the transaction inputs pass all the checks, the function returns true
4. Hashed of all the kernels are extracted from the block body
7. The block is validated by the HandleValidatedBlock function and the result is written
to bOk
9. if bOk == true
a) If bFwd == true all the block kernels are added to the DB, otherwise, all the
kernels are removed from the DB
b) If bFwd == true all the block UTXO are recognised by the RecognizeUtxos
function, otherwise, all the events below the current height are removed
All the functions generating blocks imply that the block context with a transaction pool and
some block body will be passed to them (except for the one that creates a block with an empty
body). The data will be added to the block body and context. It is assumed that the class
calling this function will take the block header, filled block body and, thus, get an entire block.
size_t NodeProcessor::GenerateNewBlock(BlockContext&
bc, Block::Body& res, Height h)
The function fills the required block fields:
1. The function adds coinbase emission to res.subsidy (the total amount generated
by the block)
2. If the subsidy is open in the block context, the res.m_SubsidyClosing flag is set
to false
4. offset = res.m_Offset
5. UTXO from coinbase is generated and added with the key from bc (sk is rewritten)
6. sk = -sk
7. offset += sk
9. sk = -sk
10. offset += sk
11. The block is serialised and its size is checked to be consistent with the Rules; later
the block will be updated with UTXO fees and its size will be checked again
13. If the transaction does not pass at least one check, it is eliminated from the transactions
pool
15. The transaction and the fee are written in the block (only UTXO, without kernel)
16. The state of the cursor is set to the hash of previous block or 0 if the cursor is in
null position
17. If the subsidy closing flag is true then the ToggleSubsidyOpened function is called
19. The block body is normalised (its kernels are sorted) the fmmr object – the instance of
the Merkle::FlyMmr class – is filled by the kernels
20. The kernels’ hash in the block header is gained from the fmmr object
21. Item 17 is repeated (Why? The block body is only modified by normalization which does
not affect the subsidy flag)
23. The new difficulty taken from the cursor is set in the block header
25. The function increasing the difficulty based on ChainWork from the context and the
cursor are called (Why is the NodeProcessor::get_NextDifficulty function
required? Should it be called before generation of the block?)
26. The block timestamp is increased as it is required for timestamp to be not less than the
moving median ( the NodeProcessor::get_NextDifficulty function)
4. The previous function generating the block is called and the returned block size is saved
9. Cast::Down<Block::BodyBase>(res) and
Cast::Down<TxVectors::Perishable>(res) are serialized and the size is written
to bc.m_BodyP
Subsidy mechanism
The enabled subsidy flag allows one to issue more than stated in coinbase emission. Initially,
this flag is set to true in the NodeProcessor::Extra function. If the block with the
m_SubsidyClosing flag is processed, then the ToggleSubsidyOpened function is called,
which sets the subsidy flag to the correct value in the NodeProcessor::Extra function.
2. The m_Utxos.Find function sets the bCreate flag (probably it shows if the
corresponding UTXO was created)
4. If the subsidy is closed, the count of found UTXO is increased by 1, otherwise, this
UTXO is eliminated from the tree
In the HadleValidatedBlock function it is checked if the current flag is consistent with the
block and the passed argument. The function does not process the block if
body.m_SubsidyClosing && (m_Extra.m_SubsidyOpen != bFwd) (bFwd is false
only for the block with zero height).
According to the code calling the block handlers the open subsidy is available only for a
number of the initial blocks. As soon as a block with the m_SubsidyClosing flag is
generated, the subsidy must be switched off and not be able to become switched on again.
The switching of subsidy (i.e. the call of the ToggleSubsidyOpened function) is present
only in the HandleValidatedBlock and GenerateNewBlock functions.
Next, a comparison between the restored Bulletproof and the Bulletproof described in the
article was made. It has been concluded that its implementation in the Beam source code
conforms to the algorithm as specified in the paper.
sk = gamma, random
? + vec(H)*( vec(z) +
P = hµ ¨ g l ¨ (h1 )r
vec(z2̂*2n̂*y-̂n) )
Notes:
1. The second equality check was not found during the audit
// calculate delta(y,z) = (z -
δ(y, z) = (z´z 2 )¨x1n , y n y´z 3 x1n , 2n y P Zp
z^2) * sumY - z^3 * sum2
?
t̂ = x1, ry P Zp
// S = G*ro + vec(sL)*vec(G) +
S = hρ g SL hSR P G
vec(sR)*vec(H)
$
α ÐÝ Zp // A = G*alpha
+ vec(aL)*vec(G) +
A = hα g al haR P G
vec(aR)*vec(H)
Bulletproof interface
The prover calls the following function:
CoSign is called inside of Create (the variables are calculated and commits are made here).
Commits are saved in the RangeProof::Confidential structure that has the following
fields:
• structure m_Part1
1. m_A //A
2. m_S //S
• structure m_Part2
1. m_T1 //T1
2. m_T2 //T2
1. m_TauX //τx
2. m_Mu //μ
3. m_tDot //t̂