Hello, I think I may have found a possible bug in the ruby openssl code for encryption.
If initialization vector is set before setting the encryption key when using one of the AES-*-GCM algorithms, the encryption does not take the IV into account at all and two different IVs (with the same key) produce the same encrypted ciphertext. If IV is set after the key, everything behaves perfectly OK. This issue does not affect other algorithms, only the AES-GCM ones.
For more context about how I came to this conclusion, please see this stack overflow question and this pull request in the encryptor gem.
Let me first show a simple test in ruby, to prove this issue:
# test_gcm.rb
require 'openssl'
def hex_enc(bytes)
bytes.bytes.map { |b| sprintf("%02x", b) }.join(" ")
end
def test_aes_encr(n, cipher, data, key, iv, iv_before_key = true)
cipher = OpenSSL::Cipher.new(cipher)
cipher.encrypt
if iv_before_key
cipher.iv = iv
cipher.key = key
else
cipher.key = key
cipher.iv = iv
end
if cipher.name.downcase.end_with?("gcm")
cipher.auth_data = ""
end
result = cipher.update(data)
result << cipher.final
puts "#{n} #{cipher.name}, iv #{iv_before_key ? "BEFORE" : "AFTER "} key: " +
"iv=#{iv}, result=#{hex_enc(result)}"
end
data = "something private"
key = "This is a key that is 256 bits!!"
# control tests using AES-256-CBC
test_aes_encr(1, "aes-256-cbc", data, key, "aaaabbbbccccdddd", true)
test_aes_encr(2, "aes-256-cbc", data, key, "eeeeffffgggghhhh", true)
test_aes_encr(3, "aes-256-cbc", data, key, "aaaabbbbccccdddd", false)
test_aes_encr(4, "aes-256-cbc", data, key, "eeeeffffgggghhhh", false)
# failing tests using AES-256-GCM
test_aes_encr(5, "aes-256-gcm", data, key, "aaaabbbbcccc", true)
test_aes_encr(6, "aes-256-gcm", data, key, "eeeeffffgggg", true)
test_aes_encr(7, "aes-256-gcm", data, key, "aaaabbbbcccc", false)
test_aes_encr(8, "aes-256-gcm", data, key, "eeeeffffgggg", false)When you run this test file, you'll get these results:
1 AES-256-CBC, iv BEFORE key: iv=aaaabbbbccccdddd, result=e0 80 06 71 ac d1 98 45 08 44 31 37 66 91 20 a1 2d 0d 9a 6d 7f fa 7a dd e5 54 f6 fd 76 9b d1 63
2 AES-256-CBC, iv BEFORE key: iv=eeeeffffgggghhhh, result=4f bb a6 d9 68 1b da fc 35 af 8b ab c8 5d f1 9c 17 aa f8 aa 33 b9 eb 63 28 62 2d 7c d2 ae ac 61
3 AES-256-CBC, iv AFTER key: iv=aaaabbbbccccdddd, result=e0 80 06 71 ac d1 98 45 08 44 31 37 66 91 20 a1 2d 0d 9a 6d 7f fa 7a dd e5 54 f6 fd 76 9b d1 63
4 AES-256-CBC, iv AFTER key: iv=eeeeffffgggghhhh, result=4f bb a6 d9 68 1b da fc 35 af 8b ab c8 5d f1 9c 17 aa f8 aa 33 b9 eb 63 28 62 2d 7c d2 ae ac 61
5 id-aes256-GCM, iv BEFORE key: iv=aaaabbbbcccc, result=4e 5f c7 7e 45 a9 c2 80 72 79 84 73 e8 cc f8 c8 8a
6 id-aes256-GCM, iv BEFORE key: iv=eeeeffffgggg, result=4e 5f c7 7e 45 a9 c2 80 72 79 84 73 e8 cc f8 c8 8a
7 id-aes256-GCM, iv AFTER key: iv=aaaabbbbcccc, result=fb 82 32 9f b4 52 0c a8 a6 4d 08 b4 4b 78 27 e7 c1
8 id-aes256-GCM, iv AFTER key: iv=eeeeffffgggg, result=de 6f 6e 10 3c 9b f5 e8 75 44 3d c2 b8 e0 a6 73 9d
The critical lines are lines 5 and 6. They show that when the IV is set before the encryption key, the IV is not taken into account. Lines 1-4 show that it this behavior is not present in the CBC encryption mode.
I tried to do further tests and they suggest that this behavior is caused by the pre-initialization of the encryption key in ossl_cipher.c. In the following test, I tried to closely mimic the C calls that ruby-openssl makes when doing a very simple encryption task:
/* aes_gcm.c */
#include <stdio.h>
#include <stdbool.h>
#include <openssl/bio.h>
#include <openssl/evp.h>
# define EVP_CTRL_AEAD_SET_IVLEN 0x9
static const unsigned char gcm_key[] = {
'T', 'h', 'i', 's', ' ', 'i', 's', ' ', 'a', ' ', 'k', 'e', 'y', ' ',
't', 'h', 'a', 't', ' ', 'i', 's', ' ', '2', '5', '6', ' ', 'b', 'i', 't', 's', '!', '!'
};
static const unsigned char gcm_iv[] = {
'a', 'a', 'a', 'a', 'b', 'b', 'b', 'b', 'c', 'c', 'c', 'c'
};
static const unsigned char gcm_pt[] = {
's', 'o', 'm', 'e', 't', 'h', 'i', 'n', 'g', ' ', 'p', 'r', 'i', 'v', 'a', 't', 'e'
};
static const unsigned char gcm_aad[] = { };
void aes_gcm_encrypt(bool iv_before_key, bool initialize_key) {
int outlen;
unsigned char outbuf[1024];
EVP_CIPHER_CTX *ctx;
const EVP_CIPHER *cipher;
unsigned char null_key[EVP_MAX_KEY_LENGTH];
/* initialize context */
ctx = EVP_CIPHER_CTX_new();
EVP_CIPHER_CTX_init(ctx);
/* configure cipher */
cipher = EVP_aes_256_gcm();
if (initialize_key) {
printf("init key, ");
memset(null_key, 0, EVP_MAX_KEY_LENGTH);
EVP_CipherInit_ex(ctx, cipher, NULL, null_key, NULL, -1);
} else {
printf("no init key, ");
EVP_CipherInit_ex(ctx, cipher, NULL, NULL, NULL, -1);
}
/* encrypt */
EVP_CipherInit_ex(ctx, NULL, NULL, NULL, NULL, 1);
/* set key and IV */
if (iv_before_key) {
printf("iv before key:\n");
EVP_CipherInit_ex(ctx, NULL, NULL, NULL, gcm_iv, -1);
EVP_CipherInit_ex(ctx, NULL, NULL, gcm_key, NULL, -1);
} else {
printf("iv after key:\n");
EVP_CipherInit_ex(ctx, NULL, NULL, gcm_key, NULL, -1);
EVP_CipherInit_ex(ctx, NULL, NULL, NULL, gcm_iv, -1);
}
/* set the authenticated data */
EVP_CipherUpdate(ctx, NULL, &outlen, gcm_aad, sizeof(gcm_aad));
/* encrypt! */
EVP_CipherUpdate(ctx, outbuf, &outlen, gcm_pt, sizeof(gcm_pt));
/* print result */
BIO_dump_fp(stdout, outbuf, outlen);
EVP_EncryptFinal_ex(ctx, outbuf, &outlen);
EVP_CIPHER_CTX_free(ctx);
}
int main(int argc, char **argv) {
aes_gcm_encrypt(true, true);
aes_gcm_encrypt(true, false);
aes_gcm_encrypt(false, true);
aes_gcm_encrypt(false, false);
}The test tries to encrypt the same data as in the ruby test above, with IV set before / after the key and with or without the pre-initialization of the key. Compiling and running the test reveals the following:
init key, iv before key:
0000 - 4e 5f c7 7e 45 a9 c2 80-72 79 84 73 e8 cc f8 c8 N_.~E...ry.s....
0010 - 8a .
no init key, iv before key:
0000 - fb 82 32 9f b4 52 0c a8-a6 4d 08 b4 4b 78 27 e7 ..2..R...M..Kx'.
0010 - c1 .
init key, iv after key:
0000 - fb 82 32 9f b4 52 0c a8-a6 4d 08 b4 4b 78 27 e7 ..2..R...M..Kx'.
0010 - c1 .
no init key, iv after key:
0000 - fb 82 32 9f b4 52 0c a8-a6 4d 08 b4 4b 78 27 e7 ..2..R...M..Kx'.
0010 - c1
The test shows that when the key pre-initialization (i.e. setting the key to all zeroes when configuring the cipher) is skipped, the IV, even if set before the encryption key, is correctly taken into account. On the other hand, when the pre-initialization takes place, the IV must be set after the key for the data to be encrypted correctly. I have not experienced any seg-faults when not preinitializing the key (a warning about this is present in the comment above the preinitialization code). I compiled and tested the code above against master branch of openssl as well as openssl-1.0.1f with the same results.
Overall, this behavior seems to me like a bug. Nowhere in the ruby-openssl documentation I have found any mention about the order of setting IV vs key being relevant for the encryption process. I believe this should perhaps be more explicitly documented, because accidental setting IVs before keys with GCM algorithms would lead to a severe weakening of the whole encryption, without the user being warned in any way.
What do you think? Let me know if you need further info and thanks!
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent