The Definitive Guide to Testing Smart Contracts on Stellar

Automated testing is one of the most powerful tools for creating more reliable, safer software.

Good tests:

• Confirm that the software does what was intended the first time it is built;
• Provide confidence to refactor code knowing that functionality has not changed;
• Help developers understand and prove the scope of a change;
• Are a multiplier on teams because they give others the confidence to change code that they have no prior experience with.

Good testing tools and strategies make our skepticism much more rigorous than we alone are capable of.

Unit tests are small tests that test one piece of functionality within a contract. They’re a great place to get started and in many ecosystems, they are the simplest test to write. Unit tests are helpful when the code to be tested is a narrow part of the program without few or no dependencies.

Below is an example of a unit test that tests the functionality of an increment contract.

#![cfg(test)]
use soroban_sdk::Env;
use crate::{IncrementContract, IncrementContractClient};

#[test]
fn test() {
   let env = Env::default();
   let contract_id = env.register(IncrementContract, ());
   let client = IncrementContractClient::new(&env, &contract_id);

   assert_eq!(client.increment(), 1);
   assert_eq!(client.increment(), 2);
   assert_eq!(client.increment(), 3);
}

To learn more about the example above and writing unit tests, see the how-to guide on Unit Tests.

Integration tests that test with Mainnet contracts can also pull in data from Mainnet so that the integration test is as similar as possible to a transaction submitted to Mainnet.

To integration test against contracts and their data from Mainnet or Testnet, use the stellar-cli to take a snapshot of the contract and its data, then load that snapshot into the test.

$ stellar snapshot create --address C... --output json --out snapshot.json

Below is an example of an integration test that tests the functionality of an increment contract along with a dependency, the pause contract that was snapshotted along with its data and loaded into the environment in the test.

use soroban_sdk::Env;
use crate::{IncrementContract, IncrementContractClient};

#[test]
fn test() {
   let env = Env::from_ledger_snapshot_file("snapshot.json");

   let pause_id = Address::from_str(&env, "C...");
   let contract_id = env.register(
       IncrementContract,
       IncrementContractArgs::__constructor(&pause_id),
   );
   let client = IncrementContractClient::new(&env, &contract_id);

   assert_eq!(client.increment(), 1);
}

Differential testing is the testing of two things to discover differences in their behavior. The goal is to prove that the two things behave consistently and that they do not diverge in behavior.

This strategy is effective when building something new that should behave like something that already exists. That could be a new version of a contract, or it could be the same contract built with a new version of an SDK or other dependency, or it could be a refactor that expects no functional changes.

When differential testing against a deployed contract, use the stellar-cli to fetch the contract from Mainnet or Testnet.

$ stellar contract fetch --id C... –-out-file pause.wasm

Below is an example of a differential test that tests that a deployed version of a contract and a new version of the contract behave the same and emit the same events.

#![cfg(test)]
use crate::{IncrementContract, IncrementContractClient};
use soroban_sdk::{testutils::Events as _, Env};

mod deployed {
   soroban_sdk::contractimport!(file = "contract.wasm");
}

#[test]
fn differential_test() {
   let env = Env::default();
   assert_eq!(
       // Baseline – the deployed contract
       {
           let contract_id = env.register(deployed::WASM, ());
           let client = IncrementContractClient::new(&env, &contract_id);
           (
               // Return Values
               (
                   client.increment(),
                   client.increment(),
                   client.increment(),
               ),
               // Events
               env.events.all(),
           )
       },
       // Local – the changed or refactored contract
       {
           let contract_id = env.register(IncrementContract, ());
           let client = IncrementContractClient::new(&env, &contract_id);
           (
               // Return Values
               (
                   client.increment(),
                   client.increment(),
                   client.increment(),
               ),
               // Events
               env.events.all(),
           )
       },
   );
}

To learn more about writing differential tests, see the how-to guide on Differential Tests.

Measuring code coverage uses tools to identify lines of code that are and aren't executed by tests. Code coverage stats can give us an idea of how much of a contract is actually tested by its tests.

One way to identify code not tested is to use code coverage tools to count and identify lines of code that are not executed by tests.

In unit and integration tests the Rust tool cargo-llvm-cov will report code coverage stats and identify lines. See Code Coverage.

In fuzz tests, the Rust tool cargo-fuzz contains coverage functionality. See Fuzzing.

Another way to identify code not tested is with mutation testing. Mutation testing is making changes to a program to identify changes that can be made that don't get caught by tests.

The cargo-mutants Rust tool can be used to mutation test contracts. See Mutation Testing.