Ethereum Tests¶
Note
See Contribute to Docs if you want to help improve this documentation.
Blockchain Tests¶
The blockchain tests aim is to test the basic verification of a blockchain.
| Location | /BlockchainTests |
| Supported Hardforks | Byzantium | Constantinople | EIP150 | EIP158 | Frontier | Homestead |
| Status | Actively supported |
A blockchain test is based around the notion of executing a list of single blocks,
described by the blocks portion of the test. The first block is the modified
genesis block as described by the genesisBlockHeader portion of the test.
A set of pre-existing accounts are detailed in the pre portion and form the
world state of the genesis block.
Of special notice is the /BlockchainTests/GeneralStateTests folder within the blockchain tests folder structure, which contains a copy of the General State Tests but executes them within the logic of the blockchain tests.
Test Implementation¶
It is generally expected that the test implementer will read genesisBlockHeader
and pre and build the corresponding blockchain in the client. Then the new blocks,
described by its RLP found in the rlp object of the blocks (RLP of a complete block,
not the block header only), is read. If the client concludes that the block is valid,
it should execute the block and verify the parameters given in blockHeader
(block header of the new block), transactions (transaction list) and uncleHeaders
(list of uncle headers). If the client concludes that the block is invalid, it should verify
that no blockHeader, transactions or uncleHeaders object is present in the test.
The client is expected to iterate through the list of blocks and ignore invalid blocks.
Test Structure¶
For a formal structure definition see also the related JSON Schema in the repo.
{
"TESTNAME_Byzantium": {
"blocks" : [
{
"blockHeader": { ... },
"rlp": { ... },
"transactions": { ... },
"uncleHeaders": { ... }
},
{
"blockHeader": { ... },
"rlp": { ... },
"transactions": { ... },
"uncleHeaders": { ... }
},
{ ... }
],
"genesisBlockHeader": { ... },
"genesisRLP": " ... ",
"lastblockhash": " ... ",
"network": "Byzantium",
"postState": { ... },
"pre": { ... },
"sealEngine": [ "NoProof" | "Ethash" ]
},
"TESTNAME_EIP150": {
...
}
...
}
The Blocks Section¶
The blocks section is a list of block objects, which have the following format:
rlpsection contains the complete rlp of the new block as described in the yellow paper in section 4.3.3.blockHeadersection describes the block header of the new block in the same format as described in genesisBlockHeader.transactionssection is a list of transactions which have the same format as in Transaction Tests.uncleHeaderssection is a list of block headers which have the same format as descibed in genesisBlockHeader.
The genesisBlockHeader Section¶
coinbase:- The 160-bit address to which all fees collected from the successful mining of this block be transferred, as returned by the COINBASE instruction.
difficulty:- A scalar value corresponding to the difficulty level of this block. This can be calculated from the previous block’s difficulty level and the timestamp, as returned by the DIFFICULTY instruction.
gasLimit:- A scalar value equal to the current limit of gas expenditure per block, as returned by the GASLIMIT instruction.
number:- A scalar value equal to the number of ancestor blocks. The genesis block has a number of zero.
timestamp:- A scalar value equal to the reasonable output of Unix’s time() at this block’s inception, as returned by the TIMESTAMP instruction.
parentHash:- The Keccak 256-bit hash of the parent block’s header, in its entirety
bloom:- The Bloom filter composed from indexable information (logger address and log topics) contained in each log entry from the receipt of each transaction in the transactions list.
extraData:- An arbitrary byte array containing data relevant to this block. This must be 1024 bytes or fewer.
gasUsed:- A scalar value equal to the total gas used in transactions in this block.
nonce:- A 256-bit hash which proves that a sufficient amount of computation has been carried out on this block.
receiptTrie:- The Keccak 256-bit hash of the root node of the trie structure populated with the receipts of each transaction in the transactions list portion of the block.
stateRoot:- The Keccak 256-bit hash of the root node of the state trie, after all transactions are executed and finalisations applied.
transactionsTrie:- The Keccak 256-bit hash of the root node of the trie structure populated with each transaction in the transactions list portion of the block.
uncleHash:- The Keccak 256-bit hash of the uncles list portion of this block
Pre and postState Sections¶
presection: as described in General State Tests.postStatesection: as described in General State Tests (section - post).
Seal Engine¶
The sealEngine parameter (values: NoProof | Ethash) defines the seal engine the
test is generated with. For tests with a value NoProof you can skip block validation
which will speed up test execution. Note that this also means that you cannot rely on
PoW specific block header values (mixHash, nonce) for tests labelled this way.
Currently this field is optional and there are still tests with no sealEngine parameter
with the default here being the NoProof setting. So make sure to first check on parameter
existence in your implementation.
Optional BlockHeader Information¶
"blocknumber" = "int" is section which defines what is the order of this block.
It is used to define a situation when you have 3 blocks already imported but then it comes new version of the block 2 and 3 and thus you might have new best blockchain with blocks 1 2’ 3’ instead previous. If blocknumber is undefined then it is assumed that blocks are imported one by one. When running test, this field could be used for information purpose only.
"chainname" = "string" This is used for defining forks in the same test. You could mine blocks to chain “A”: 1, 2, 3 then to chain “B”: 1, 2, 3, 4 (chainB becomes primary). Then again to chain “A”: 4, 5, 6 (chainA becomes primary) and so on. chainname could also be defined in uncle header section. If defined in uncle header it tells on which chain’s block uncle header would be populated from. When running test, this field could be used for information purpose only.
"chainnetwork" = "string" Defines on which network rules this block was mined. (see the difference https://github.com/ethereum/EIPs/blob/master/EIPS/eip-2.mediawiki). When running test, this field could be used for information purpose only.
General State Tests¶
The state tests aim is to test the basic workings of the state in isolation.
| Location | /GeneralStateTests |
| Supported Hardforks | Byzantium | Constantinople | EIP150 | EIP158 | Frontier | Homestead |
| Status | Actively supported |
A state test is based around the notion of executing a single transaction, described
by the transaction portion of the test. The overarching environment
in which it is executed is described by the env portion of the test and
includes attributes of the current and previous blocks. A set of pre-existing accounts
are detailed in the pre portion and form the world state prior to execution.
Similarly, a set of accounts are detailed in the post portion to specify the
end world state. Since the data of the blockchain is not given, the opcode BLOCKHASH
could not return the hashes of the corresponding blocks. Therefore we define the hash of
block number n to be SHA256("n").
The log entries (logs) as well as any output returned from the code (output) is also detailed.
Test Implementation¶
It is generally expected that the test implementer will read env, transaction
and pre then check their results against logs, out, and post.
Note
The structure of state tests was reworked lately, see the associated discussion here.
Test Structure¶
{
"testname" : {
"env" : {
"currentCoinbase" : "address",
"currentDifficulty" : "0x020000", //minimum difficulty for mining on blockchain
"currentGasLimit" : "u64", //not larger then maxGasLimit = 0x7fffffffffffffff
"currentNumber" : "0x01", //Irrelevant to hardfork parameters!
"currentTimestamp" : "1000", //for blockchain version
"previousHash" : "h256"
},
"post" : {
"EIP150" : [
{
"hash" : "3e6dacc1575c6a8c76422255eca03529bbf4c0dda75dfc110b22d6dc4152396f",
"indexes" : { "data" : 0, "gas" : 0, "value" : 0 }
},
{
"hash" : "99a450d8ce5b987a71346d8a0a1203711f770745c7ef326912e46761f14cd764",
"indexes" : { "data" : 0, "gas" : 0, "value" : 1 }
},
...
],
"EIP158" : [
{
"hash" : "3e6dacc1575c6a8c76422255eca03529bbf4c0dda75dfc110b22d6dc4152396f",
"indexes" : { "data" : 0, "gas" : 0, "value" : 0 }
},
{
"hash" : "99a450d8ce5b987a71346d8a0a1203711f770745c7ef326912e46761f14cd764",
"indexes" : { "data" : 0, "gas" : 0, "value" : 1 }
},
...
],
"Frontier" : [
...
],
"Homestead" : [
...
]
},
"pre" : {
//same as for StateTests
},
"transaction" : {
"data" : [ "" ],
"gasLimit" : [ "285000", "100000", "6000" ],
"gasPrice" : "0x01",
"nonce" : "0x00",
"secretKey" : "45a915e4d060149eb4365960e6a7a45f334393093061116b197e3240065ff2d8",
"to" : "095e7baea6a6c7c4c2dfeb977efac326af552d87",
"value" : [ "10", "0" ]
}
}
}
The env Section¶
currentCoinbasecurrentDifficultycurrentGasLimitcurrentNumbercurrentTimestamppreviousHashThe transaction Section¶
datagasLimitgasPricenonceaddresssecretKeytovalueThe post Section¶
Indexes section describes which values from given array to set for transaction
before it’s execution on a pre state. Transaction now has data, value, and gasLimit as arrays.
post section now has array of implemented forks. For each fork it has another array
of execution results on that fork rules with post state root hash and transaction parameters.
The pre Section¶
The pre section have the format of a mapping between addresses and accounts.
Each account has the format:
balancenoncecodestorage"1200" or "0x04B0" For values used $DATA_ARRAY.RLP Tests¶
Describes an RLP (https://github.com/ethereum/wiki/wiki/RLP) encoding using the .json file.
| Location | /RLPTests |
| Supported Hardforks | Hardfork-independent |
| Status | Actively supported |
Test Implementation¶
The client should read the rlp byte stream, decode and check whether the contents match its json representation. Then it should try do it reverse - encode json rlp representation into rlp byte stream and check whether it matches the given rlp byte stream.
If it is an invalid RLP byte stream in the test, then ‘in’ field would contain string INVALID.
Some RLP byte streams are expected to be generated by fuzz test suite. For those
examples ‘in’ field would contain string VALID as it means that rlp should be easily decoded.
Note that RLP tests are testing a single RLP object encoding and not a stream of RLP objects in one array.
Test Structure¶
{
"rlpTest": {
"in": "dog",
"out": "83646f67"
},
"multilist": {
"in": [ "zw", [ 4 ], 1 ],
"out": "c6827a77c10401"
},
"validRLP": {
"in": "VALID",
"out": "c7c0c1c0c3c0c1c0"
},
"invalidRLP": {
"in": "INVALID",
"out": "bf0f000000000000021111"
},
...
}
Sections¶
in- json object (array, int, string) representation of the rlp byte stream (*except valuesVALIDandINVALID)out- string of rlp bytes stream
Difficulty Tests¶
These tests are designed to just check the difficulty formula of a block.
| Location | BasicTests (difficulty*.json) |
| Supported Hardforks | Test Networks | Frontier | Homestead |
| Status | Outdated |
difficulty = DIFFICULTY(currentBlockNumber, currentTimestamp, parentTimestamp, parentDifficulty)
described at EIP2 point 4 with homestead changes.
So basically this .json tests are just to check how this function is calculated on different function parameters (parentDifficulty, currentNumber) in its extremum points.
There are several test files:
difficulty.json- Normal Frontier/Homestead chain difficulty tests defined manually
difficultyFrontier.json- Same as above, but auto-generated tests
difficultyMorden.json- Tests for testnetwork difficulty. (it has different homestead transition block)
difficultyOlimpic.json- Olympic network. (no homestead)
difficultyHomestead.json- Tests for homestead difficulty (regardless of the block number)
difficultyCustomHomestead.json- Tests for homestead difficulty (regardless of the block number)
Test Structure¶
{
"difficultyTest" : {
"parentTimestamp" : "42",
"parentDifficulty" : "1000000",
"currentTimestamp" : "43",
"currentBlockNumber" : "42",
"currentDifficulty" : "1000488"
}
}
Sections¶
parentTimestamp- indicates the timestamp of a previous blockparentDifficulty- indicates the difficulty of a previous blockcurrentTimestamp- indicates the timestamp of a current blockcurrentBlockNumber- indicates the number of a current block (previous block number = currentBlockNumber - 1)currentDifficulty- indicates the difficulty of a current block
Transaction Tests¶
Describes a complete transaction and its RLP representation using the .json file.
| Location | /TransactionTests |
| Supported Hardforks | Constantinople | EIP158 | Frontier | Homestead |
| Status | Actively supported |
Test Implementation¶
The client should read the rlp and check whether the transaction is valid, has the correct sender and corresponds to the transaction parameters. If it is an invalid transaction, the transaction and the sender object will be missing.
Test Structure¶
{
"transactionTest1": {
"rlp" : "bytearray",
"sender" : "address",
"blocknumber" : "1000000"
"transaction" : {
"nonce" : "int",
"gasPrice" : "int",
"gasLimit" : "int",
"to" : "address",
"value" : "int",
"v" : "byte",
"r" : "256 bit unsigned int",
"s" : "256 bit unsigned int",
"data" : "byte array"
}
},
"invalidTransactionTest": {
"rlp" : "bytearray",
},
...
}
Sections¶
rlp- RLP encoded data of this transactiontransaction- transaction described by fieldsnonce- A scalar value equal to the number of transactions sent by the sender.gasPrice- A scalar value equal to the number of wei to be paid per unit of gas.gasLimit- A scalar value equal to the maximum amount of gas that should be used in executing this transaction.to- The 160-bit address of the message call’s recipient or empty for a contract creation transaction.value- A scalar value equal to the number of wei to be transferred to the message call’s recipient or, in the case of contract creation, as an endowment to the newly created account.v, r, s- Values corresponding to the signature of the transaction and used to determine the sender of the transaction.sender- the address of the sender, derived from the v,r,s values.blocknumber- indicates network rules for the transaction. Since blocknumber = 1000000 Homestead rules are applied to transaction. (see https://github.com/ethereum/EIPs/blob/master/EIPS/eip-2.mediawiki)
VM Tests¶
The VM tests aim is to test the basic workings of the VM in isolation.
| Location | /VMTests |
| Supported Hardforks | Currently only Homestead |
| Status | Actively supported |
This is specifically not meant to cover transaction, creation or call processing, or management of the state trie. Indeed at least one implementation tests the VM without calling into any Trie code at all.
A VM test is based around the notion of executing a single piece of code as part of a transaction,
described by the exec portion of the test. The overarching environment in which it is
executed is described by the env portion of the test and includes attributes
of the current and previous blocks. A set of pre-existing accounts are detailed
in the pre portion and form the world state prior to execution. Similarly, a set
of accounts are detailed in the post portion to specify the end world state.
The gas remaining (gas), the log entries (logs) as well as any output returned
from the code (out) is also detailed.
Test Implementation¶
It is generally expected that the test implementer will read env, exec and pre
then check their results against gas, logs, out, post and callcreates.
If an exception is expected, then latter sections are absent in the test. Since the
reverting of the state is not part of the VM tests.
Because the data of the blockchain is not given, the opcode BLOCKHASH could not return the hashes of the corresponding blocks. Therefore we define the hash of block number n to be SHA3-256(“n”).
Since these tests are meant only as a basic test of VM operation, the CALL and
CREATE instructions are not actually executed. To provide the possibility of
testing to guarantee they were actually run at all, a separate portion callcreates
details each CALL or CREATE operation in the order they would have been executed.
Furthermore, gas required is simply that of the VM execution: the gas cost for
transaction processing is excluded.
Test Structure¶
{
"test name 1": {
"env": { ... },
"pre": { ... },
"exec": { ... },
"gas": { ... },
"logs": { ... },
"out": { ... },
"post": { ... },
"callcreates": { ... }
},
"test name 2": {
"env": { ... },
"pre": { ... },
"exec": { ... },
"gas": { ... },
"logs": { ... },
"out": { ... },
"post": { ... },
"callcreates": { ... }
},
...
}
The env Section¶
currentCoinbase: The current block’s coinbase address, to be returned by theCOINBASEinstruction.currentDifficulty: The current block’s difficulty, to be returned by theDIFFICULTYinstruction.currentGasLimit: The current block’s gas limit.currentNumber: The current block’s number.currentTimestamp: The current block’s timestamp.previousHash: The previous block’s hash.
The exec Section¶
address: The address of the account under which the code is executing, to be returned by theADDRESSinstruction.origin: The address of the execution’s origin, to be returned by theORIGINinstruction.caller: The address of the execution’s caller, to be returned by theCALLERinstruction.value: The value of the call (or the endowment of the create), to be returned by theCALLVALUEinstruction.data: The input data passed to the execution, as used by theCALLDATA… instructions. Given as an array of byte values. See $DATA_ARRAY.code: The actual code that should be executed on the VM (not the one stored in the state(address)) . See $DATA_ARRAY.gasPrice: The price of gas for the transaction, as used by theGASPRICEinstruction.gas: The total amount of gas available for the execution, as would be returned by theGASinstruction were it be executed first.
The pre and post Section¶
The pre and post sections each have the same format of a mapping between addresses and accounts. Each account has the format:
balance: The balance of the account.nonce: The nonce of the account.code: The body code of the account, given as an array of byte values. See $DATA_ARRAY.storage: The account’s storage, given as a mapping of keys to values. For key used notion of string as digital or hex number e.g:"1200"or"0x04B0"For values used $DATA_ARRAY.
The callcreates Section¶
The callcreates section details each CALL or CREATE instruction that has been executed. It is an array of maps with keys:
data: An array of bytes specifying the data with which theCALLorCREATEoperation was made. In the case ofCREATE, this would be the (initialisation) code. See $DATA_ARRAY.destination: The receipt address to which theCALLwas made, or the null address ("0000...") if the corresponding operation wasCREATE.gasLimit: The amount of gas with which the operation was made.value: The value or endowment with which the operation was made.
The logs Section¶
The logs sections contains the hex encoded hash of the rlp encoded log entries, reducing the overall size of the test files while still verifying that all of the data is accurate (at the cost of being able to read what the data should be).
Each logentry has the format:
keccak(rlp.encode(log_entries))
The gas and output Keys¶
Finally, there are two simple keys, gas and out:
gas: The amount of gas remaining after execution.out: The data, given as an array of bytes, returned from the execution (using theRETURNinstruction). See $DATA_ARRAY.
- $DATA_ARRAY - type that intended to contain raw byte data
- and for convenient of the users is populated with three types of numbers, all of them should be converted and concatenated to a byte array for VM execution.
The types are: 1. number - (unsigned 64bit) 2. “longnumber” - (any long number) 3. “0xhex_num” - (hex format number)
e.g:
````[1, 2, 10000, "0xabc345dFF", "199999999999999999999999999999999999999"]````
Test Creation¶
Instructions on how to create tests and how to use the cpp-client testeth tool
can be found in the c++ documentation
test generation chapter.
Since docker images pointed to in the test generation docs are often outdated, you will probably have to rely on your own source build of the Aleth C++ Ethereum client for generating tests (start this early on since this is taking some time).
Contribute to Docs¶
This documentation has been build using the Python Sphinx documentation tool.
Since the Ethereum tests repository is very large to clone locally, a convenient way to contribute to the documentation is to make a fork of the test repo, add the changes online with the GitHub reStructuredText editor and then open a PR.
If you want to clone to your desk you might want to make use of git clone --depth 1
for faster download.
You can build the documentation by running make html from the docs directory
in the tests repository.