squiddev/cipher-crackers

Crackers for National Cipher Challenge

Cipher-Crackers

This is a collection of utilities for solving the National Cipher Challenge.

Ciphers Implemented

• Caesar shift: Attempts every possible key and scores using quadgram statistics
• Substitution: Uses a hill-climbing algorithm.
• Vigenere: Guesses the key length from repeated sub-strings, then solves individual shifts using monogram statistics.
• Columnar Transposition: Tries every key, scores using quadgrams
• Rail-fence: Tries every key, scores using quadgrams
• Hill Cipher: Finds the modular matrix inverse using cribs. Requires several cribs as not every crib is usable (the determinant of the matrix must be co-prime with 26).

Analysis Tools

• Auto-spacer: Attempts to add spaces between words
• N-Gram statistics: Occurrences of words of a specific length.
• Cipher guesser: Uses some of the methods described here to try to identify a cipher.

Usage

Whilst this is MIT licensed, and so you're free to use this I do request that you try to avoid using this in competitions. It's far more satisfying putting the tools together yourself. If you are interested in writing your own crackers, I recommend the Practical Cryptography website, a resource that proved invaluable when writing this. This project is intended as an educational resource, not just for easy wins.

cipher-crackers's Issues

NGrams ignores last letter

Ciphers to Implement

• Substitution
• Caeser shift
• Vigenere
• Columnar Transposition
• Railfence
• Hill cipher
• Playfair
• Bifid

Playfair is too slow

IndexOf is far too slow.

How to solve this I'm not sure though. One idea would be to use a dictionary for lookups as changes to the key happen once a tick cycle rather than once every second.

Anything else here

Document everything

It would be nice if we could document all the code in the main Ciphers project.

Origin of wordguesser data files?

I'm interested in writing an auto-spacer like the one included here, and I was wondering if:

https://raw.githubusercontent.com/SquidDev/Cipher-Crackers/master/src/Cipher/Analysis/AutoSpace/2WordScores.txt

and

https://raw.githubusercontent.com/SquidDev/Cipher-Crackers/master/src/Cipher/Analysis/AutoSpace/1WordScores.txt

Are under the license in the readme or if they have some other origin which means that I am allowed to use them. (I use my cipher cracker for the national cipher challenge).

Data driven testing

It would be nice to have some data driven testing for ciphers using XML or something.

This question gives some ideas.

Multlthread more code

We want to Multithread anything that takes more than 100ms. However this may result in having to multithread everything...

Times and currently mutlithreaded:

• Auto spacer (400ms)
• Cipher analysis (100ms)
• Substitution crack (270ms)
• Transposition crack (340ms)

Crack methods should be multithreaded and cipher analysis to.

We can probably get rid of abort as it doesn't take that long...

CShell Integration

This isn't a priority but it would be fun to implement...

Multithreading

There are probably some crackers that could benefit from multi-threading:

• Caeser: Each iteration is just a decode and then score: There is no point.
• Transposition: Each possible key length could be run in a separate thread, or I could look into splitting the permutations into separate threads.
• Hill Brute: Each row could be run in a separate thread
• Rail fence: No point as this is pretty fast.
• Substitution: Each of the 5 iterations could be run in a separate thread.
• Vigenere: Each Caesar could be run in a separate thread, however this is pretty quick, so there is little point.

Cipher Guesser does not work

The cipher guesser's results are wrong. The text statistics work so the problem is probably somewhere in calculating the standard deviation. It might be a good thing to check that all cipher statistics are correct.

The test cases do not pass.

AutoSpacer does not work with strings less than 20 characters

Nothing happens if the text is less than MAX_WORD_LENGTH](https://github.com/SquidDev/Cipher-Crackers/blob/master/C%23/Cipher/Analysis/AutoSpace/WordGuesser.cs#L87) characters.

Not dissimilar to #1 as it does not happen in the original python implementation.

Word guesser trims last words

0df9903 works very well but the last MAX_WORD_LENGTH characters are not spaced at all. I can't see the hole in the logic yet...

WordGuesser Preprocessing

It takes ~4 seconds to load the stats for the word guesser. We should put most of the processing in Python and just load raw dictionaries.

I've started here: https://github.com/SquidDev/Cipher-Crackers/tree/wordguesser-converter