Giter Site home page Giter Site logo

upgrade-scripts's Introduction

Upgrade Scripts (WIP)

Scripts to automate keeping track of active deployments and upgrades. Allows for:

  • automatic contract deployments and proxy upgrades if the source has changed
  • keeping track of all latest deployments and having one set-up for unit-tests, deployments and interactions
  • storage layout compatibility checks on upgrades

These scripts use ERC1967Proxy (the relevant functions can be overridden, see deploying custom proxies).

Example SetUp Script

This example is from ExampleSetupScript.

contract ExampleSetupScript is UpgradeScripts {
    ExampleNFT nft;

    function setUpContracts() internal {
        address implementation = setUpContract("ExampleNFT");

        bytes memory initCall = abi.encodeCall(ExampleNFT.init, ("My NFT", "NFTX"));
        address proxy = setUpProxy(implementation, initCall);

        nft = ExampleNFT(proxy);
    }
}

Running this script on a live network will deploy the implementation contract and the proxy contract once. Re-running this script without the implementation having changed won't do anything. Re-running this script with a new implementation will detect the change and deploy a new implementation contract. It will perform a storage layout compatibility check and update your existing proxy to point to it. All current deployments are updated in deployments/{chainid}/deploy-latest.json.

SetUpContract / SetUpProxy

This will make sure that MyContract is deployed and kept up-to-date. If the .creationCode of MyContract ever changes, it will re-deploy the contract. The hash of .creationCode is compared instead of addr.codehash, because this would not allow for reliable checks for contracts that use immutable variables that change for each implementation (such as using address(this) in EIP-2612's DOMAIN_SEPARATOR).

string memory contractName = "MyContract"; // name of the contract to be deployed
bytes memory constructorArgs = abi.encode(arg1, arg2); // abi-encoded args (optional)
string memory key = "MyContractImplementation"; // identifier/key to be used for json (optional, defaults to `contractName`)
bool attachOnly = false; // don't deploy, only read from latest-deployment and "attach" (optional, defaults to `false`)

address implementation = setUpContract(contractName, constructorArgs, key, attachOnly);

The key is used for display in the console and as an identifier in deployments/{chainid}/deploy-latest.json. Setting up multiple contracts/proxies of the same type requires different keys to be set.

Similarly, a proxy can be deployed and kept up-to-date via setUpProxy.

bytes memory initCall = abi.encodeCall(MyContract.init, ()); // data to pass to proxy for making an initial call during deployment (optional)
string memory key = "MyContractProxy"; // identifier/key to be used for json (optional, defaults to `${contractNameImplementation}Proxy`)
bool attachOnly = false; // (optional, defaults to `false`)

address proxy = setUpProxy(implementationAddress, initCall, key, attachOnly);

Storage layout mappings are stored for each proxy implementation. These are used for storage layout compatibility checks when running upgrades. This requires the implementation contract to be set up using setUpContract for the script to know what storage layout to store for the proxy. It is best to run through a complete example to understand when/how this is done.

Example Tutorial using Anvil

First, make sure Foundry is installed.

  1. Clone the repository:
git clone https://github.com/0xPhaze/upgrade-scripts
  1. Navigate to the example directory and install the dependencies
cd upgrade-scripts/example
forge install
  1. Spin up a local anvil node in a second terminal.
anvil

Read through deploy.s.sol before running random scripts from the internet using --ffi.

  1. In the example project root, run
UPGRADE_SCRIPTS_DRY_RUN=true forge script deploy --rpc-url http://127.0.0.1:8545 --private-key 0xac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80 -vvvv --ffi

to go through a "dry-run" of the deploy scripts. This connects to your running anvil node using the default account's private key.

  1. Add --broadcast to the command to actually broadcast the transactions on-chain.
forge script deploy --rpc-url http://127.0.0.1:8545 --private-key 0xac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80 -vvvv --broadcast --ffi

After a successful run, it should have created the file ./example/deployments/31337/deploy-latest.json which keeps track of your up-to-date deployments. It also saves the contracts creation code hash and its storage layout.

  1. Try running the command again. It will detect that no implementation has changed and thus not create any new transactions.

Upgrading a Proxy Implementation

If any registered contracts' implementation changes, this should be detected and the corresponding proxies should automatically get updated on another call. Try changing the implementation by, for example, uncommenting the line in tokenURI() in ExampleNFT.sol and re-running the script.

contract ExampleNFT {
    ...
    function tokenURI(uint256 id) public view override returns (string memory uri) {
        // uri = "abcd";
    }
}

After a successful upgrade, running the script once more will not broadcast any additional transactions.

Detecting Storage Layout Changes

A main security-feature of these scripts is to detect storage-layout changes. Try uncommenting the following line in ExampleNFT.sol.

contract ExampleNFT is UUPSUpgrade, ERC721UDS, OwnableUDS {
    // uint256 public contractId = 1;
    ...
}

This adds an extra variable contractId to the storage of ExampleNFT. If the script is run again (note that --ffi needs to be enabled), it should notify that a storage layout change has been detected:

  Storage layout compatibility check [0x9fE46736679d2D9a65F0992F2272dE9f3c7fa6e0 <-> 0xDc64a140Aa3E981100a9becA4E685f962f0cF6C9]: fail
  
Diff:
  [...]

  
If you believe the storage layout is compatible, add the following to the beginning of `run()` in your deploy script.
`
isUpgradeSafe[31337][0x9fE46736679d2D9a65F0992F2272dE9f3c7fa6e0][0xDc64a140Aa3E981100a9becA4E685f962f0cF6C9] = true;
`

Note that, this can easily lead to false-positives, for example, when any variable is renamed or when, like in this case, a variable is appended correctly to the end of existing storage. Thus any positive detection here requires manually review.

Another peculiarity to account for is that, since dry-run uses vm.prank instead of vm.broadcast, there might be some differences when calculating the addresses of newly deployed contracts. Thus, sometimes, the scripts need to be run without a dry-run to get the correct address to be marked as "upgrade-safe".

Since we know it is safe, we can add the line

isUpgradeSafe[31337][0x9fE46736679d2D9a65F0992F2272dE9f3c7fa6e0][0xDc64a140Aa3E981100a9becA4E685f962f0cF6C9] = true;

to the start of run() in deploy.s.sol. If we re-run the script now, it will deploy a new implementation, perform the upgrade for our proxy and update the contract addresses in deploy-latest.json.

Extra Notes

Environment Variables

These variables can be set in before running a script, by overriding setUpUpgradeScripts() or by passing them in with the command line. They can also be abbreviated (US_RESET=true forge script ...).

bool UPGRADE_SCRIPTS_RESET; // re-deploys all contracts
bool UPGRADE_SCRIPTS_BYPASS; // deploys contracts without any checks whatsoever
bool UPGRADE_SCRIPTS_DRY_RUN; // doesn't overwrite new deployments in deploy-latest.json
bool UPGRADE_SCRIPTS_ATTACH_ONLY; // doesn't deploy contracts, just attaches with checks
bool UPGRADE_SCRIPTS_BYPASS_SAFETY; // bypass all upgrade safety checks

Accessing Deployments from other Chains

Deployed addresses from other chains can be accessed via loadLatestDeployedAddress(key, chainId):

address latestFxRootTunnel = loadLatestDeployedAddress("RootTunnelProxy", rootChainId); // will be address(0) if not found

Additional init Scripts

if (isFirstTimeDeployed(addr)) {
    // ... do stuff when the proxy is deployed for the first time
}

Deploying Custom Proxies

All functions in UpgradeScripts can be overridden. These functions in particular might be of interest to override.

 function getDeployProxyCode(address implementation, bytes memory initCall) internal virtual returns (bytes memory) {
     // ...
 }

 function upgradeProxy(address proxy, address newImplementation) internal virtual {
     // ...
 }

 function deployCode(bytes memory code) internal virtual returns (address addr) {
     // ...
 }

See exampleOZ/ExampleSetupScript.sol for a complete example using OpenZeppelin's upgradeable contracts.

Running on Mainnet

If not running on a testnet, adding a confirmation through the current timestamp will be necessary, i.e. adding mainnetConfirmation = 1667499028;. This is an additional safety measure.

Testing with Upgrade Scripts

In order to keep the deployment as close to the testing environment, it is generally helpful to share the same contract set-up scripts.

To disable any additional checks or logs that are not necessary when running forge test, the function setUpUpgradeScripts() can be overridden to include UPGRADE_SCRIPTS_BYPASS = true;. This can be seen in ExampleNFT.t.sol. This bypasses all checks and simply deploys the contracts.

Interacting with Deployed Contracts

To be able to interact with deployed contracts, the existing contracts can be "attached" to the current environment (instead of re-deploying). An example of how this can be done in order to mint an NFT from a deployed address is shown in mint.s.sol. This requires the previous steps to be completed.

The script can then be run via:

forge script mint --rpc-url http://127.0.0.1:8545 --private-key 0xac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80 -vvvv --broadcast

What is '/data/'? Should this be committed?

The files in '/data/' are there 1) to tell whether a deployment's contract code has changed and needs to be re-deployed and 2) to determine whether an upgrade is safe.

  1. '*.creation-code-hash' stores the hash of the complete creation code which is used for detecting any code changes. If the the scripts can't find the relevant '.creation-code-hash' file, it will just assume that a new deployment is necessary.
  2. '*.storage-layout' keeps track of the storage layout files tied to the specific deployment addresses. This is used to ensure that the storage layout's between the old and the new implementation contracts are compatible. If the relevant '.storage-layout' file is not found for an address, the script will complain. This means the user needs to manually approve the upgrade.

Contract Storage Layout Incompatible Example

Here is an example of what a incompatible contract storage layout change could look like:

"label": "districts",                                          |   "label": "sharesRegistered",
"type": "t_mapping(t_uint256,t_struct(District)40351_storage)" |   "type": "t_mapping(t_uint256,t_bool)"
"astId": 40369,                                                |   "astId": 40531,
"label": "gangsters",                                          |   "label": "districts",
"type": "t_mapping(t_uint256,t_struct(Gangster)40314_storage)" |   "type": "t_mapping(t_uint256,t_struct(District)40514_storage)"
"astId": 40373,                                                |   "astId": 40536,
"label": "itemCost",                                           |   "label": "gangsters",
                                                               >   "type": "t_mapping(t_uint256,t_struct(Gangster)40477_storage)"
                                                               > },
                                                               > {
                                                               >   "astId": 40540,
                                                               >   "contract": "src/GangWar.sol:GangWar",
                                                               >   "label": "itemCost",
                                                               >   "offset": 0,
                                                               >   "slot": "7",
"astId": 40377,                                                |   "astId": 40544,
"slot": "7",                                                   |   "slot": "8",

Here, an additional mapping(uint256 => bool) sharesRegistered (right side) was inserted in a storage slot where previously another mapping existed, shifting the slots of the other variables. The variable itemCost, previously slot 7 (left side) is now located at slot 8. Running an upgrade with this change would lead to storage layout conflicts.

Using some diff-tool viewer (such as vs-code's right-click > compare selected) can often paint a clearer picture. image

Notes and disclaimers

These scripts do not replace manual review and caution must be taken when upgrading contracts in any case. Make sure you understand what the scripts are doing. I am not responsible for any damages created.

Note that, it currently is not possible to detect whether --broadcast is enabled. Thus the script can't reliably detect whether the transactions are only simulated or sent on-chain. For that reason, when --broadcast is not set, UPGRADE_SCRIPT_DRY_RUN=true must ALWAYS passed in. Otherwise this will update deploy-latest.json with addresses that haven't actually been deployed yet and will complain on the next run.

When deploy-latest.json was updated with incorrect addresses for this reason, just delete the file and the incorrect previously created deploy-{latestTimestamp}.json (containing the highest latest timestamp) and copy the correct .json (second highest timestamp) to deploy-latest.json.

If anvil is restarted, these deployments will also be invalid. Simply delete the corresponding folder rm -rf deployments/31337 in this case.

upgrade-scripts's People

Contributors

0xphaze avatar kalloc avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

upgrade-scripts's Issues

Error when testing or deploying a contract containing a library function call.

Hi.

Thank you very much for your work.
But, I found a mistake while using it.
If the contract under test contains a library function call, the testing fails.
I made a repo with an error https://github.com/8gen/0xPhaze-upgrade-script-bug, so you may repeat.

How I call function:

..
import {Lib} from "./library/Lib.sol";
..
number = newNumber + Lib.x();
..
$ forge test

[⠆] Compiling...
No files changed, compilation skipped
2023-03-15T12:59:19.471056Z ERROR forge::runner: setUp failed reason="failed to read from \"../out/Counter/Counter.json\": No such file or directory (os error 2)" contract=0x7fa9385be102ac3eac297483dd6233d62b3e1496

Running 1 test for test/Counter.t.sol:CounterTest
[FAIL. Reason: Setup failed: failed to read from "../out/Counter/Counter.json": No such file or directory (os error 2)] setUp() (gas: 0)
Test result: FAILED. 0 passed; 1 failed; finished in 965.23µs

Failing tests:
Encountered 1 failing test in test/Counter.t.sol:CounterTest
[FAIL. Reason: Setup failed: failed to read from "../out/Counter/Counter.json": No such file or directory (os error 2)] setUp() (gas: 0)

Encountered a total of 1 failing tests, 0 tests succeeded

Running without --broadcast?

Hey @0xPhaze ! Good stuff!

I ran a deploy script without the US_DRY_RUN=true option, but also without the --broadcast, which gave a simulation of the transactions. However it did create a deployment json file, even though no contracts were actually deployed. If I wanted to go ahead and make a deployment, I'd need to delete some json files before going ahead.

This seems like an in-between step between dry-run and actual execution. Some questions:

  • Should this "simulation" be the actual dry run? I.e. Remove the US_DRY_RUN flag and just rely on whether the broadcast flag is present?
  • Regardless whether we keep the US_DRY_RUN flag, is it possible to detect the absence of the --broadcast flag and not output the deployments .json file?

If you're busy but have an opinion, I/we could make the change.

Feature request: having a new flag that disable console.log outputs

While console.log is great and appreciated. During testing would be great to be able to turn them off once we are confidence that our proxies are behaving as we expect so we don't clutter the output. Ideally, do something similar to the existing levels of verbosity.

For example:
Screenshot 2023-02-19 at 11 44 16 AM

Happy to help with this new feature if you found it valuable. Let me know what do you think.

Not upgrading if bytecode and creation code have not changed

Hey!

I was wondering if there is a way to prevent re-deployment of code that basically has not changed (e.g. reformatting, adding comments, or changing function order). Currently those can cause false-positive storage incompatibilities with isUpgradeSafe. However, implementation address is the thing that is actually checked for a difference to know whether something is changed or not. There is no control over this, and a contract will get redeployed in this case.

Is there a way to see if implementation bytecode is identical (even if solidity code has slight cosmetic changes), and opt out of upgrading a contract in that case? I think solidity outputs different bytecode if the source changes (a section towards the end of the bytecode), I'm assuming for verification purposes. Any way to check only the implementation bytecode and no anything that depends on the source?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.