cameronlonsdale/lantern

Cryptanalysis library for breaking classical ciphers

lantern's Issues

Things to think about because im disorganised

LanguageFrequency and LanguageNgrams is the same thing, why do I need 2. Need to refactor into the same bit of code but the nicest name that its understandible how It can be used.

need to think about the range of values that can result from a fitness function, specifically in relation to the corpus problem

V0.1 Roadmap

• Make dynamicdict have appropriate behaviour for its name. not even a dict right now

• Update the vigenere example text to showcase the diversity better. Investigate that bug why the hackers manifesto with punctuation didnt decrypt the first paragraph properly.
  (Fixed! It was due to digits not being removed. Its a messy fix, but it works for now until I can build nicer abstractions in later updates)

• Investigate speed, see if its possible to speed anything up
  I think the speed is reasonable. 2 seconds for simple sub / vigenere. If I wanted speed I wouldnt use python. I made some small changes to free up some bottlenecks but Im pushing the limits of the language. I might investigate rewriting several key functions in a compiled language in the future

• Investigate usability. See if alternating subsitution cipher code can be simplified and made intuitive

• Update README to show python3.4 or newer is required (same for docs)

• Review documentation to make sure its configured properly

• Setup readthedocs for stable and development releases

• Update version to 0.1 and upload to pypi

LanguageFrequency refactor

LanguageFrequency and LanguageNgrams is the same thing, why do I need 2. Need to refactor into the same bit of code but the nicest name that its understandable how It can be used.

Chi_squared is not flexible enough

There is an issue with chi_squared where by you can be penalised easily for silly mistakes. For instance trying to score the text "abc" against english.unigrams wont work because the unigrams are capital letters. I need to find a way that users arent penalised for this, currently they are. So RIP.

Similar issue to this is the zero division error in chi squared if source_len is 0. Which can happen when you source freq map has no similar characters to the target. In which case, an Error should probably be thrown.

Add type hints for the codebase

This is a big job, it will be done piecewise.

Better Documentation

The readme needs updating + I want docs which show how to use the library and example code.

Simple Substitution working for ciphertext < 250 characters

Using ngram analysis simple substitution works well for ciphertexts of length 250 characters or more. Current thought is to try a corpus along side ngram to check for word existance however the current implementation of corpus might need some modification to deal with non whitespace hint

Rethink documentation for fitness functions

Its not essential that init and call are documented, It might be better to just have examples instead

Write code to guess which cipher was used to encrypt ciphertext based on index of coincidence

Fix Vigenere Bug

Not sure why, but this does not get decrypted properly

N VVWJN ENB A PWINRG ITNNG WE VEEKRWAEG VW FRCQJL WICJN. GPR OUAKGROAA BO A AQASA VVPUUQMQ NSCQBWATM, QNCRXGROA, IAM SHZCAIFM NCTNKXB. TUMVA CBDRAT ZMGQOQA BO WNOVWG VZENGHTNA WNZSJRR ERAE QMRVEQ LVBHBVBAAOTR JNQ JRWENBU CHR PBWOE WS CHR ANVUEIV. CHBCTQ SUQAXBV XEXPRZ, NB SCMPRAYTL CRNQAND FXVNS NVQ VEEKRWAEQRB, ACXRJRRL VW TUM 15GQ CRVGDRL LHAIAO GQE FMAPOXC CNRVWQ (15CH–17GP PNNGCEREF), IACEPMQNNGA ZJY UIIN EKQFCEQ IF NAETL JS GPR 12CH PMACUEG."

Also, you should get better error checking for when ChiSquared does not match the text thats provided.

Infer whitespace in text

quipquip.com puts whitespace in the resulting plaintext for you even if there are no whitespace hints. I'd like to add this feature.

Import issue for ngrams

Currently I predefine several ngram classes so you can import and go, but the issue is that when you import one, every one of them has to load, which slows down the program.

I need to re write this code so that when you want quadgrams, you only get quadgrams

Add example of byte shifting an image

Id like to be able to use the shift decryptor to automate crack a byte shifted image by checking for the correct header bytes at the start of the image.

Support adaptive scoring functions

Currently scoring functions get run in sequence, however for certain functions, this process will take too long.

I would like to support an option to only run a subsequent function if there is not a clear correct decryption. Ways to do this could be to look at the scoring distribution for the decryptions and running more scoring rounds if the values are too close together.

iterate_ngrams is very slow

This function is used very often when decrypting vigenere, and its slowness has impacted the overall speed of vigenere. Need to look into speeding it up, perhaps with the help of itertools, or even writing a snippet of compiled C/D/Rust/whatever

Add an encrypt function for shift

While its not necessarily needed for cryptanalysis, for test purposes and building challenges it would be nice to be able to encrypt within the same framework aswell.

Return the key with every decryption

The key should be returned with each decryption. Rather than using a tuple and be coupled to the order, we should make a lightweight class to group a decryption attributes

Substitution Cipher speed up

Substitution cipher's current implementation is quite slow, I would like this to be sped up as much as is possible in python

Determine best number of swaps and trials for Simple Substitution Cipher

The current hardcoded solution takes too long for examples which can be cracked with only 2 trials. I need to come up with a way to estimate what the best settings are for a given ciphertext. This might be a job for torched (or whatever I'm calling the CLI) I could look at ciphertext length for this metric.