Different behaviour: `encode_single` vs `encode`

[Uxio0]

Different behaviour: `encode_single` vs `encode`

[Pai-Sho]

I see the problem is on my side. I should not be treating the values as a tuple in encode. v4.0.0 yields the correct result

[fselmo]

@Uxio0 what version of eth-abi are you using? There was an encoding bug that was also fixed for dynamic encoding. The most recent version of eth-abi lines up with Solidity encoding of empty bytes. Neither of what you posted is what Solidity returns. My suggestion is to try to update your library to use the latest eth-abi. If you need some time to transition to the latest version then, yeah, we need to figure out how / why encode is differing from encode_single within the same version.

I'm also assuming that you used the different methods since both of your code snippets are using encode and neither use encode_single (though the second imports it). I assumed that was just a typo. (I already confirmed this discrepancy in version 3.0.1 below)

Solidity encoding used for reference:

contract Contract {
    function encodeEmptyBytes() public pure returns (bytes memory) {
        bytes memory _b;
        return abi.encode(_b);
    }
}

which returns (consistent with the latest version of eth-abi):

b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00 \x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'

[fselmo]

Hmm I do see the discrepancy in 3.0.1 which is definitely a bug as they should yield the same results. I think this is a result of the bug that was in the encoding. We can and should definitely put up a patch version for 3.0.1 with this bugfix but, again, neither of these are correct compared to what Solidity returns. I suggest upgrading to the latest version of eth-abi as soon as you can.

Thanks for submitting this.

[fselmo]

On second thought, and looking at this again, I don't see any easy way to fix this since

1. The way encode (previously encode_abi) was written, it is inherently doing something different than encode_single and results in a different result.

2. Modifying encode() in this version to return encode_single() if the length is 1 breaks encode()'s behavior (breaking change). This is because if you used encode_abi() in the past, with a single item, it would produce a different result. This is the most unfortunate part of this change.

3. They are really both wrong and this bug is fixed in the latest eth-abi version.

The only answer I see here really is unfortunately just skipping this version altogether and updating the docs to reflect this and encourage users to only use the latest version. Thankfully we've moved away from decoupling encoding a single item from many. This just leads to more surface + room for errors like these.

Is there any other option I missed that would be better for you as you are transitioning to the new eth-abi version? @Uxio0

[Uxio0]

Hi @fselmo and thanks for the detailed answer:

I was using v2.2.0 and v3.0.1. Indeed:

len(eth_abi.encode(['bytes'], [b'']))
96
len(eth_abi.encode_single('bytes', b''))
64

but with 4.0.0

len(eth_abi.encode(['bytes'], [b'']))
64

You are right, I wanted to make sure everything was working before migrating to v4.0.0 and indeed it was fixed in the last version. I will update to v4.0.0 and then do the refactor in the same step.

Thank you very much

[Pai-Sho]

@fselmo I am still observing this behavior with v4.0.0, and it's causing problems upgrading

Example:

with v2.2.0

encode_single(
    '(address,uint256,uint256,uint256,bytes)',
    ["0xC02aaA39b223FE8D0A0e5C4F27eAD9083C756Cc2", 1000, 1, 0, b'']
)

with v4.0.0

encode(
    ['(address,uint256,uint256,uint256,bytes)'],
    [("0xC02aaA39b223FE8D0A0e5C4F27eAD9083C756Cc2", 1000, 1, 0, b'')]
)

These should return the same result, but they do not.

[fselmo]

@fselmo I am still observing this behavior with v4.0.0, and it's causing problems upgrading

Example:

with v2.2.0

encode_single(
    '(address,uint256,uint256,uint256,bytes)',
    ["0xC02aaA39b223FE8D0A0e5C4F27eAD9083C756Cc2", 1000, 1, 0, b'']
)

with v4.0.0

encode(
    ['(address,uint256,uint256,uint256,bytes)'],
    [("0xC02aaA39b223FE8D0A0e5C4F27eAD9083C756Cc2", 1000, 1, 0, b'')]
)

These should return the same result, but they do not.

2.2.0 had a bug. Do not compare anything to those results. Instead compare the latest encode to Solidity.

[Pai-Sho]

@fselmo Hmm, using v2.2.0, v4.0.0 and solidity I'm getting 3 different answers

Using the above examples, I get

0x0000000000000000000000000000000000000000000000000000000000000020000000000000000000000000c02aaa39b223fe8d0a0e5c4f27ead9083c756cc200000000000000000000000000000000000000000000000000000000000003e80000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000a00000000000000000000000000000000000000000000000000000000000000000
for v4.0.0

0x000000000000000000000000c02aaa39b223fe8d0a0e5c4f27ead9083c756cc200000000000000000000000000000000000000000000000000000000000003e80000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000a000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
for v2.2.0

and
0x000000000000000000000000c02aaa39b223fe8d0a0e5c4f27ead9083c756cc200000000000000000000000000000000000000000000000000000000000003e80000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000a00000000000000000000000000000000000000000000000000000000000000000
with solidity abi.encode(address(0xC02aaA39b223FE8D0A0e5C4F27eAD9083C756Cc2), uint256(1000), uint256(1), uint256(0), bytes(""));

[fselmo]

closing as resolved / completed / stale