aosabook/500lines

500 Lines or Less

500 Lines or Less

"What I cannot create, I do not understand."

-- Richard Feynman

This is the source for the book 500 Lines or Less, the fourth in the Architecture of Open Source Applications series. As with other books in the series, all written material will be covered by the Creative Commons - Attribution license, and all code by the MIT License: please see the license description for details. In addition, all royalties from paid-for versions will all go to Amnesty International.

The production of this book has been made possible by the financial support of PagerDuty.

Mission

Every architect studies family homes, apartments, schools, and other common types of buildings during her training. Equally, every programmer ought to know how a compiler turns text into instructions, how a spreadsheet updates cells, and how a database efficiently persists data.

Previous books in the AOSA series have done this by describing the high-level architecture of several mature open-source projects. While the lessons learned from those stories are valuable, they are sometimes difficult to absorb for programmers who have not yet had to build anything at that scale.

"500 Lines or Less" focuses on the design decisions and tradeoffs that experienced programmers make when they are writing code:

• Why divide the application into these particular modules with these particular interfaces?
• Why use inheritance here and composition there?
• How do we predict where our program might need to be extended, and how can we make that easy for other programmers?

Each chapter consists of a walkthrough of a program that solves a canonical problem in software engineering in at most 500 source lines of code. We hope that the material in this book will help readers understand the varied approaches that engineers take when solving problems in different domains, and will serve as a basis for projects that extend or modify the contributions here.

Contributors

Name	Affiliation	Project	Online	GitHub
Mike DiBernardo	Wave	editorial	@mdibernardo mikedebo.ca	MichaelDiBernardo
Amy Brown	indie	editorial	amyrbrown.ca @amyrbrown	amyrbrown
Allison Kaptur	Dropbox	byterun	@akaptur	akaptur
Audrey Tang	g0v.tw, Socialtext, Apple	spreadsheet	@audreyt	audreyt
Brandon Rhodes	Dropbox	contingent	@brandon_rhodes	brandon-rhodes
Carl Friedrich Bolz	King's College London	object model	cfbolz.de @cfbolz	cfbolz
Cate Huston		Image Filter app	www.accidentallyincode.com/ @catehstn	catehstn
Christian Muise	University of Melbourne	flow-shop	@cjmuise	haz
Daniel Jackson		same-origin-policy
Daniel Rocco	BrightLink Technology	contingent	@drocco007	drocco007
Dann Toliver	Bento Box	dagoba	danntoliver.com @dann	dxnn
Dessy Daskalov	Nudge Rewards	Pedometer	www.dessydaskalov.com @dess_e	dessy
Dethe Elza		blockcode		dethe
Dustin Mitchell	Mozilla	cluster		djmitche
Erick Dransch		Modeller	@ErickDransch	EkkiD
Eunsuk Kang		same-origin-policy
Greg Wilson		web-server	@gvwilson	gvwilson
Guido van Rossum	Dropbox	crawler	@gvanrossum	gvanrossum
A. Jesse Jiryu Davis	MongoDB	crawler	@jessejiryudavis	ajdavis
Jessica Hamrick	University of California, Berkeley	sampler	www.jesshamrick.com @jhamrick	jhamrick
Leah Hanson	Google	static analysis	@astrieanna	astrieanna
Leo Zovic		event-web-framework
Malini Das	Twitch	ci	malinidas.com @malinidas	malini
Marina Samuel	Mozilla	ocr	www.marinasamuel.com @emtwos	emtwo
Ned Batchelder	edX	templating engine	nedbatchelder.com @nedbat	nedbat
Santiago Perez De Rosso		same-origin-policy
Taavi Burns	Previously at Points, now at PagerDuty	data-store	@jaaaarel	taavi
Yoav Rubin	Microsoft	In-memory functional database	@yoavrubin	yoavrubin

Technical Reviewers

Amber Yust	Andrew Gwozdziewycz	Andrew Kuchling
Andrew Svetlov	Andy Shen	Anton Beloglazov
Ben Trofatter	Borys Pierov	Carise Fernandez
Charles Stanhope	Chris Atlee	Chris Seaton
Cyryl Płotnicki-Chudyk	Dan Langer	Dan Shapiro
David Pokorny	Eric Bouwers	Frederic De Groef
Graham Lee	Gregory Eric Sanderson	James O'Beirne
Jan de Baat	Jana Beck	Jessica McKellar
Jo Van Eyck	Joel Crocker	Johan Thelin
Johannes Fürmann	John Morrissey	Joseph Kaptur
Josh Crompton	Joshua T. Corbin	Kevin Huang
Maggie Zhou	Marc Towler	Marcin Milewski
Marco Lancini	Mark Reid	Matthias Bussonnier
Max Mautner	Meggin Kearney	Mike Aquino
Natalie Black	Nick Presta	Nikhil Almeida
Nolan Prescott	Paul Martin	Piotr Banaszkiewicz
Preston Holmes	Pulkit Sethi	Rail Aliiev
Ronen Narkis	Rose Ames	Sina Jahan
Stefan Turalski	William Lachance

500lines's Issues

cluster one unit test may not passed

One unit test in dir test/test_seed.py which does not successed when I tried python -m unittest test.test_seed

Transaction bug in functional db, the 'ts' field will be incorrect if execute more than one operations in a single transaction?

@yoavrubin

The functional db project is really cool, and I learned a lot from it. After reading it, I have a doubt about the Transaction section. I'm not familiar with Clojure, if I have a wrong understanding, please forgive me, thank you very much.

In the transact-on-db function, each operations will be executed on the initial-db and finally generated an updated database named transacted, and then the newest layer in transacted will be appended to the initial-db as the newest layer. I think there is a bug in the last process if more than one operations need to be executed in an transaction. Consider the following example:

1. The initial-db has two layers, the first layer is empty(ts is 0), and the second one has only one entity e with an attribute named a, and a's value is v1, ts is 1.
2. Execute two update operations in a transaction: the first one will update a to v2 and the second one will update a to v3. The first operation will generate a new layer, in which a's value is v2 and ts is 2, prev-ts is 1. The second operation will generate a new layer too, in which a's value is v3, ts is 3, prev-ts is 2.
3. After above two operations, transact-on-db will append the layer generated by second operation to initial-db's layers, now the initial-db has three layers, the first one's ts is 0, the second one's ts is 1, but the third one's ts is 3, and a's prev-ts is 2 in the third layer. At this time, if we invoke the evolution-of function on initial-db, we will fall into an infinite loop.

The above example is just one possible error, if we execute more than two update operations in a single transaction and then invoke evolution-of, we will fall into a 'layer not found' error.

installation instructions

Hi @ondras, thanks for your contributions, i face some difficulties.
1.where i can get the installation instructions.
2.when i run python build.py --pdf(the lib of requirements.txt is installed), i got "OSError: [Errno 2] No such file or directory",but the files is there
Thank yout!

Wrong line count in interpreter

"telling us that the LOAD_FAST instruction appears at position zero"
->
"telling us that the LOAD_FAST instruction appears at position six"

No?

What happens when a reponse takes very long

I tried to add:

            response = yield from asyncio.wait_for(
                self.session.get(url, allow_redirects=False), 20)

instead of

            response = yield from self.session.get(url, allow_redirects=False)

In order to prevent hanging from a server by introducing a max_timeout, but this seems to open up a lot of CancelledErrors (and a lot of Task was destroyed but it is pending). Any idea?

Modeller uses displaylists which are deprecated.

The modeller example is a great start, unfortunately it uses pre Opengl3.3 (displaylists, and Glut which only works with older GL).

It also uses Glut for drawing the Sphere, I've got no idea what this should be replaced with (generating a sphere in the code might push things over 500 lines, probably there is some library that can be used).

An url in “Web-server.markdown” is not found

In Web-server.markdown, there is a piece of code as below:

import requests
response = requests.get('http://aosabook.org/en/500lines/web-server/testpage.html')
print 'status code:', response.status_code
print 'content length:', response.headers['content-length']
print response.text

An URL "http://aosabook.org/en/500lines/web-server/testpage.html" is not found.

The 99 line link for web spreadsheet not found

The link with text "99 line" point to https://github.com/audreyt/500lines/tree/master/spreadsheet/code, which returns 404.

Should the link be https://github.com/audreyt/500lines/tree/master/spreadsheet instead?

DBDB > Organisational Units > Inserting and Updating, `storage.py` should be `physical.py` .

@taavi

Crawler: How to fix the error "ConnectionRefusedError: [Errno 111] Connection refused"?

Hi, I'm running the loop-with-callbacks.py in the crawler project.
But I always got an error when running a few time of the program.

Traceback (most recent call last):
  File "loop-with-callbacks.py", line 103, in <module>
    callback(event_key, event_mask)
  File "loop-with-callbacks.py", line 40, in connected
    self.sock.send(get.encode('ascii'))
ConnectionRefusedError: [Errno 111] Connection refused

I tried to replace self.sock.send(get.encode('ascii') with self.sock.sendall(get.encode('ascii')), but it got the same error.
I can't solve this problem, so I'm looking for some help.
And my Python version is 3.4.1, my system is Ubuntu.
Best wished!

Lisp code indentation at An Event-Driven Code Framework

http://aosabook.org/en/500L/an-event-driven-web-framework.html <- this is a page where Lisp code needs proper indenting to make it readable, possibly to some tab<=>space issues.

Solution: basically, put all the code into an Emacs buffer with Lisp mode and Slime powered on in another buffer to fix some indentation problems, indent the blocks, copy everything back in.
Possible readability improvement: grab an automatic JS Lisp highlighter from http://turtleware.eu/posts/Creating-a-project-homepage-with-the-SCLP.html (mouseover some nested code to see how it works).

pen test tool

someone should write a pen test tool chapter!

(hint: the web server chapter would be a great use case for that tool)

https://en.wikipedia.org/wiki/Penetration_test

Python interpreter won't work with Python 3.6

Python 3.6 is transitioning to a regularized 16-bit wordcode

Crawler not performing as mentioned in README

The Crawler is finding fewer URLs than mentioned in the README for me. I am running Crawl.py but haven't successfully built the 500Lines project. Does that matter? I'm using Windows 10, PyCharm by JetBrains, Python 3.6.1 (because asyncio is required for the crawler to run and asyncio wasn't implemented until Python 3.4).

Specific examples:
Site, distinct URLs mentioned in README, distinct URLs I received
dropbox.com, ~2500, 78
xkcd.com, +1500, 38

A simple object model: /self instead of self

Some of the functions defined in the "Simple object model" episode have a first argument called /self instead of self

crawler vs scrapy

is it faster than scrapy when crawling the xkcd.com? is there a benchmark?

An unreasonable infinite loops in ‘ci/code/dispatcher.py’

In function redistribute in line 148:

def redistribute(server):
    while not server.dead:
        for commit in server.pending_commits:
            print "running redistribute"
            print server.pending_commits
            dispatch_tests(server, commit)
            time.sleep(5)

the time.sleep(5) should be placed in while-loop , or it may consume all available processor time

x

Through some series of keystrokes I accidentally opened an issue. Oops.

Publish on https://www.gitbook.com/

I think this would be a great platform for our books.

program index

I wonder is there should be a file that contain a short description for every program.

Figures are not showing

The pictures are not loading for the figures in the pedometer article and the clustering by consensus article.
I experienced this issue both on Chrome and Firefox.

not sure where to put this: suggestion of NLP in 500 lines

don t know if this would fit the project
http://honnibal.wordpress.com/2013/12/18/a-simple-fast-algorithm-for-natural-language-dependency-parsing/

Rigorous input parsing

I'd love to do a chapter on input handling with formal parsers. I've written a C library (with bindings for lots of other languages) intended for exactly this purpose, and am in the process of editing a series of video tutorials for implementing JSON-RPC with composed parsers. This would be very easy to turn into a chapter.

Control flow diagram not showing up for ci project

At this link: http://aosabook.org/en/500L/pages/a-continuous-integration-system.html#figure-12.1,
the control flow diagram is not showing up.

Update: I think images for all the lessons can't be found

small mistake in interpreter/interpreter.markdown

in Conditionals and Loops, the last sentence of line 313:
'When x < 10' should be 'When x < 5'

bug in cluster/code/run.py

line 34 of the file is

Request(node, input, req_done).start()

But cluster.py doesn't define anything named Request, and the program just cannot run. I think it should be Requester (defined at line 483 in cluster.py).

Em dashes

Some chapters seem to have correct rendering for em dashes, like http://aosabook.org/en/500L/pages/a-web-crawler-with-asyncio-coroutines.html

Others show '---' instead of '—', like http://aosabook.org/en/500L/dagoba-an-in-memory-graph-database.html and http://aosabook.org/en/500L/pages/a-simple-web-server.html

Some error in project "ocr"

In neural_network_design.py

The outer loop for j in xrange(100): is useless since the prediction of the neural network is definite.
I think the author tried to combined the training process in the outer loop. Anyway, some correction should be made.

Unmatched template tag in Templite class comment

There is an umatched template tag in https://github.com/jinguanz/500lines/blob/master/template-engine/code/templite.py#L85. It should be "{% endfor %}" instead of "{% endif %}".

Assertion failure in cluster.py

if you run with python run.py 903 you may encounter an assertion failure.

AssertionError: next slot to commit is already decided

Somewhat messy format in "web-server/chapter.md"

Somewhat messy format here: web-server/chapter.md

And also missing several diagrams - several "FIXME" left.

In addition, it's not finished yet. (not updated for many months)

scrapy

is it faster than scrapy when crawling the xkcd.com? is there a benchmark?

crawler - tests failing - see output

The crawler project's test.py is failing during the setup of every test but I am not familiar enough with asyncio to understand why. System information:
Windows 10
Python 3.5

Here is the output:

C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\site-packages\aiohttp\web.py:54: ResourceWarning: loop argument is deprecated
warnings.warn("loop argument is deprecated", ResourceWarning)
EEEEEEEEEEEEEEEEE

ERROR: test_content_type (main.TestCrawler)

Traceback (most recent call last):
File "test.py", line 51, in setUp
self.app = self.loop.run_until_complete(self._create_server())
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\base_events.py", line 342, in run_until_complete
return future.result()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\futures.py", line 274, in result
raise self._exception
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\tasks.py", line 239, in _step
result = coro.send(value)
File "test.py", line 64, in _create_server
handler_factory = app.make_handler(debug=True)
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\site-packages\aiohttp\web.py", line 224, in make_handler
self._set_loop(loop)
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\site-packages\aiohttp\web.py", line 110, in _set_loop
loop = asyncio.get_event_loop()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\events.py", line 589, in get_event_loop
return get_event_loop_policy().get_event_loop()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\events.py", line 535, in get_event_loop
% threading.current_thread().name)
RuntimeError: There is no current event loop in thread 'MainThread'.

======================================================================
ERROR: test_deep_root (main.TestCrawler)

Traceback (most recent call last):
File "test.py", line 51, in setUp
self.app = self.loop.run_until_complete(self._create_server())
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\base_events.py", line 342, in run_until_complete
return future.result()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\futures.py", line 274, in result
raise self._exception
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\tasks.py", line 239, in _step
result = coro.send(value)
File "test.py", line 64, in _create_server
handler_factory = app.make_handler(debug=True)
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\site-packages\aiohttp\web.py", line 224, in make_handler
self._set_loop(loop)
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\site-packages\aiohttp\web.py", line 110, in _set_loop
loop = asyncio.get_event_loop()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\events.py", line 589, in get_event_loop
return get_event_loop_policy().get_event_loop()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\events.py", line 535, in get_event_loop
% threading.current_thread().name)
RuntimeError: There is no current event loop in thread 'MainThread'.

======================================================================
ERROR: test_encoding (main.TestCrawler)

Traceback (most recent call last):
File "test.py", line 51, in setUp
self.app = self.loop.run_until_complete(self._create_server())
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\base_events.py", line 342, in run_until_complete
return future.result()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\futures.py", line 274, in result
raise self._exception
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\tasks.py", line 239, in _step
result = coro.send(value)
File "test.py", line 64, in _create_server
handler_factory = app.make_handler(debug=True)
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\site-packages\aiohttp\web.py", line 224, in make_handler
self._set_loop(loop)
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\site-packages\aiohttp\web.py", line 110, in _set_loop
loop = asyncio.get_event_loop()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\events.py", line 589, in get_event_loop
return get_event_loop_policy().get_event_loop()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\events.py", line 535, in get_event_loop
% threading.current_thread().name)
RuntimeError: There is no current event loop in thread 'MainThread'.

======================================================================
ERROR: test_exclude (main.TestCrawler)

Traceback (most recent call last):
File "test.py", line 51, in setUp
self.app = self.loop.run_until_complete(self._create_server())
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\base_events.py", line 342, in run_until_complete
return future.result()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\futures.py", line 274, in result
raise self._exception
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\tasks.py", line 239, in _step
result = coro.send(value)
File "test.py", line 64, in _create_server
handler_factory = app.make_handler(debug=True)
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\site-packages\aiohttp\web.py", line 224, in make_handler
self._set_loop(loop)
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\site-packages\aiohttp\web.py", line 110, in _set_loop
loop = asyncio.get_event_loop()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\events.py", line 589, in get_event_loop
return get_event_loop_policy().get_event_loop()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\events.py", line 535, in get_event_loop
% threading.current_thread().name)
RuntimeError: There is no current event loop in thread 'MainThread'.

======================================================================
ERROR: test_lenient_host_checking (main.TestCrawler)

Traceback (most recent call last):
File "test.py", line 51, in setUp
self.app = self.loop.run_until_complete(self._create_server())
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\base_events.py", line 342, in run_until_complete
return future.result()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\futures.py", line 274, in result
raise self._exception
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\tasks.py", line 239, in _step
result = coro.send(value)
File "test.py", line 64, in _create_server
handler_factory = app.make_handler(debug=True)
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\site-packages\aiohttp\web.py", line 224, in make_handler
self._set_loop(loop)
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\site-packages\aiohttp\web.py", line 110, in _set_loop
loop = asyncio.get_event_loop()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\events.py", line 589, in get_event_loop
return get_event_loop_policy().get_event_loop()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\events.py", line 535, in get_event_loop
% threading.current_thread().name)
RuntimeError: There is no current event loop in thread 'MainThread'.

======================================================================
ERROR: test_link (main.TestCrawler)

Traceback (most recent call last):
File "test.py", line 51, in setUp
self.app = self.loop.run_until_complete(self._create_server())
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\base_events.py", line 342, in run_until_complete
return future.result()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\futures.py", line 274, in result
raise self._exception
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\tasks.py", line 239, in _step
result = coro.send(value)
File "test.py", line 64, in _create_server
handler_factory = app.make_handler(debug=True)
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\site-packages\aiohttp\web.py", line 224, in make_handler
self._set_loop(loop)
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\site-packages\aiohttp\web.py", line 110, in _set_loop
loop = asyncio.get_event_loop()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\events.py", line 589, in get_event_loop
return get_event_loop_policy().get_event_loop()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\events.py", line 535, in get_event_loop
% threading.current_thread().name)
RuntimeError: There is no current event loop in thread 'MainThread'.

======================================================================
ERROR: test_link_cycle (main.TestCrawler)

Traceback (most recent call last):
File "test.py", line 51, in setUp
self.app = self.loop.run_until_complete(self._create_server())
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\base_events.py", line 342, in run_until_complete
return future.result()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\futures.py", line 274, in result
raise self._exception
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\tasks.py", line 239, in _step
result = coro.send(value)
File "test.py", line 64, in _create_server
handler_factory = app.make_handler(debug=True)
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\site-packages\aiohttp\web.py", line 224, in make_handler
self._set_loop(loop)
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\site-packages\aiohttp\web.py", line 110, in _set_loop
loop = asyncio.get_event_loop()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\events.py", line 589, in get_event_loop
return get_event_loop_policy().get_event_loop()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\events.py", line 535, in get_event_loop
% threading.current_thread().name)
RuntimeError: There is no current event loop in thread 'MainThread'.

======================================================================
ERROR: test_max_tasks (main.TestCrawler)

Traceback (most recent call last):
File "test.py", line 51, in setUp
self.app = self.loop.run_until_complete(self._create_server())
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\base_events.py", line 342, in run_until_complete
return future.result()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\futures.py", line 274, in result
raise self._exception
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\tasks.py", line 239, in _step
result = coro.send(value)
File "test.py", line 64, in _create_server
handler_factory = app.make_handler(debug=True)
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\site-packages\aiohttp\web.py", line 224, in make_handler
self._set_loop(loop)
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\site-packages\aiohttp\web.py", line 110, in _set_loop
loop = asyncio.get_event_loop()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\events.py", line 589, in get_event_loop
return get_event_loop_policy().get_event_loop()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\events.py", line 535, in get_event_loop
% threading.current_thread().name)
RuntimeError: There is no current event loop in thread 'MainThread'.

======================================================================
ERROR: test_max_tries (main.TestCrawler)

Traceback (most recent call last):
File "test.py", line 51, in setUp
self.app = self.loop.run_until_complete(self._create_server())
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\base_events.py", line 342, in run_until_complete
return future.result()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\futures.py", line 274, in result
raise self._exception
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\tasks.py", line 239, in _step
result = coro.send(value)
File "test.py", line 64, in _create_server
handler_factory = app.make_handler(debug=True)
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\site-packages\aiohttp\web.py", line 224, in make_handler
self._set_loop(loop)
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\site-packages\aiohttp\web.py", line 110, in _set_loop
loop = asyncio.get_event_loop()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\events.py", line 589, in get_event_loop
return get_event_loop_policy().get_event_loop()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\events.py", line 535, in get_event_loop
% threading.current_thread().name)
RuntimeError: There is no current event loop in thread 'MainThread'.

======================================================================
ERROR: test_non_html (main.TestCrawler)

Traceback (most recent call last):
File "test.py", line 51, in setUp
self.app = self.loop.run_until_complete(self._create_server())
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\base_events.py", line 342, in run_until_complete
return future.result()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\futures.py", line 274, in result
raise self._exception
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\tasks.py", line 239, in _step
result = coro.send(value)
File "test.py", line 64, in _create_server
handler_factory = app.make_handler(debug=True)
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\site-packages\aiohttp\web.py", line 224, in make_handler
self._set_loop(loop)
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\site-packages\aiohttp\web.py", line 110, in _set_loop
loop = asyncio.get_event_loop()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\events.py", line 589, in get_event_loop
return get_event_loop_policy().get_event_loop()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\events.py", line 535, in get_event_loop
% threading.current_thread().name)
RuntimeError: There is no current event loop in thread 'MainThread'.

======================================================================
ERROR: test_non_http (main.TestCrawler)

Traceback (most recent call last):
File "test.py", line 51, in setUp
self.app = self.loop.run_until_complete(self._create_server())
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\base_events.py", line 342, in run_until_complete
return future.result()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\futures.py", line 274, in result
raise self._exception
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\tasks.py", line 239, in _step
result = coro.send(value)
File "test.py", line 64, in _create_server
handler_factory = app.make_handler(debug=True)
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\site-packages\aiohttp\web.py", line 224, in make_handler
self._set_loop(loop)
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\site-packages\aiohttp\web.py", line 110, in _set_loop
loop = asyncio.get_event_loop()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\events.py", line 589, in get_event_loop
return get_event_loop_policy().get_event_loop()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\events.py", line 535, in get_event_loop
% threading.current_thread().name)
RuntimeError: There is no current event loop in thread 'MainThread'.

======================================================================
ERROR: test_prohibited_host (main.TestCrawler)

Traceback (most recent call last):
File "test.py", line 51, in setUp
self.app = self.loop.run_until_complete(self._create_server())
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\base_events.py", line 342, in run_until_complete
return future.result()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\futures.py", line 274, in result
raise self._exception
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\tasks.py", line 239, in _step
result = coro.send(value)
File "test.py", line 64, in _create_server
handler_factory = app.make_handler(debug=True)
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\site-packages\aiohttp\web.py", line 224, in make_handler
self._set_loop(loop)
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\site-packages\aiohttp\web.py", line 110, in _set_loop
loop = asyncio.get_event_loop()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\events.py", line 589, in get_event_loop
return get_event_loop_policy().get_event_loop()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\events.py", line 535, in get_event_loop
% threading.current_thread().name)
RuntimeError: There is no current event loop in thread 'MainThread'.

======================================================================
ERROR: test_redirect (main.TestCrawler)

Traceback (most recent call last):
File "test.py", line 51, in setUp
self.app = self.loop.run_until_complete(self._create_server())
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\base_events.py", line 342, in run_until_complete
return future.result()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\futures.py", line 274, in result
raise self._exception
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\tasks.py", line 239, in _step
result = coro.send(value)
File "test.py", line 64, in _create_server
handler_factory = app.make_handler(debug=True)
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\site-packages\aiohttp\web.py", line 224, in make_handler
self._set_loop(loop)
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\site-packages\aiohttp\web.py", line 110, in _set_loop
loop = asyncio.get_event_loop()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\events.py", line 589, in get_event_loop
return get_event_loop_policy().get_event_loop()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\events.py", line 535, in get_event_loop
% threading.current_thread().name)
RuntimeError: There is no current event loop in thread 'MainThread'.

======================================================================
ERROR: test_redirect_cycle (main.TestCrawler)

Traceback (most recent call last):
File "test.py", line 51, in setUp
self.app = self.loop.run_until_complete(self._create_server())
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\base_events.py", line 342, in run_until_complete
return future.result()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\futures.py", line 274, in result
raise self._exception
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\tasks.py", line 239, in _step
result = coro.send(value)
File "test.py", line 64, in _create_server
handler_factory = app.make_handler(debug=True)
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\site-packages\aiohttp\web.py", line 224, in make_handler
self._set_loop(loop)
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\site-packages\aiohttp\web.py", line 110, in _set_loop
loop = asyncio.get_event_loop()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\events.py", line 589, in get_event_loop
return get_event_loop_policy().get_event_loop()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\events.py", line 535, in get_event_loop
% threading.current_thread().name)
RuntimeError: There is no current event loop in thread 'MainThread'.

======================================================================
ERROR: test_redirect_join (main.TestCrawler)

Traceback (most recent call last):
File "test.py", line 51, in setUp
self.app = self.loop.run_until_complete(self._create_server())
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\base_events.py", line 342, in run_until_complete
return future.result()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\futures.py", line 274, in result
raise self._exception
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\tasks.py", line 239, in _step
result = coro.send(value)
File "test.py", line 64, in _create_server
handler_factory = app.make_handler(debug=True)
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\site-packages\aiohttp\web.py", line 224, in make_handler
self._set_loop(loop)
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\site-packages\aiohttp\web.py", line 110, in _set_loop
loop = asyncio.get_event_loop()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\events.py", line 589, in get_event_loop
return get_event_loop_policy().get_event_loop()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\events.py", line 535, in get_event_loop
% threading.current_thread().name)
RuntimeError: There is no current event loop in thread 'MainThread'.

======================================================================
ERROR: test_roots (main.TestCrawler)

Traceback (most recent call last):
File "test.py", line 51, in setUp
self.app = self.loop.run_until_complete(self._create_server())
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\base_events.py", line 342, in run_until_complete
return future.result()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\futures.py", line 274, in result
raise self._exception
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\tasks.py", line 239, in _step
result = coro.send(value)
File "test.py", line 64, in _create_server
handler_factory = app.make_handler(debug=True)
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\site-packages\aiohttp\web.py", line 224, in make_handler
self._set_loop(loop)
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\site-packages\aiohttp\web.py", line 110, in _set_loop
loop = asyncio.get_event_loop()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\events.py", line 589, in get_event_loop
return get_event_loop_policy().get_event_loop()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\events.py", line 535, in get_event_loop
% threading.current_thread().name)
RuntimeError: There is no current event loop in thread 'MainThread'.

======================================================================
ERROR: test_strict_host_checking (main.TestCrawler)

Traceback (most recent call last):
File "test.py", line 51, in setUp
self.app = self.loop.run_until_complete(self._create_server())
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\base_events.py", line 342, in run_until_complete
return future.result()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\futures.py", line 274, in result
raise self._exception
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\tasks.py", line 239, in _step
result = coro.send(value)
File "test.py", line 64, in _create_server
handler_factory = app.make_handler(debug=True)
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\site-packages\aiohttp\web.py", line 224, in make_handler
self._set_loop(loop)
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\site-packages\aiohttp\web.py", line 110, in _set_loop
loop = asyncio.get_event_loop()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\events.py", line 589, in get_event_loop
return get_event_loop_policy().get_event_loop()
File "C:\Users\Primary User\AppData\Local\Programs\Python\Python35\lib\asyncio\events.py", line 535, in get_event_loop
% threading.current_thread().name)
RuntimeError: There is no current event loop in thread 'MainThread'.

---

Ran 17 tests in 0.031s

FAILED (errors=17)

Julia and static analysis typo

"possibly it's own code"

Template engine doesn't support else

In the template-engine project the markdown states that you can use if else statements, but the code in the template-engine doesn't support it and also none of the test cases test for else statements either.

See markdown file:
Template Engine Markdown File

Search for {% else %} to jump to the issue

What's the "cgiutil" ? web-server/06-simple-cgi/testquery.py

web-server/06-simple-cgi/testquery.py

import time, cgiutil

rawData = cgiutil.getRawCgiData()
params = cgiutil.parseCgiData(rawData)

cgiutil is not in the standard library, and I can't find it by google.

Could anyone explain where I can find it?

many errors on building

OS: Mac 10.12.3
python Python 2.7.12, no virtualenv used

cmd:
python build.py --html

output:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/envoy/core.py", line 214, in run
out, err = cmd.run(data, timeout, kill_timeout, env, cwd)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/envoy/core.py", line 93, in run
raise self.exc
OSError: [Errno 2] No such file or directory

./_build/fix_html_title.sh html/content/pages/template-engine.md

sed: 1: "html/content/pages/temp ...": extra characters at the end of h command

python _build/preprocessor.py --chapter 22 --html-refs --output=./web-server/web-server.markdown.1 --latex ./web-server/web-server.markdown

pandoc --csl=minutiae/pdf/ieee.csl --mathjax -t html -f markdown+citations -o html/content/pages/web-server.md ./web-server/web-server.markdown.1

No such file or directory
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/envoy/core.py", line 214, in run
out, err = cmd.run(data, timeout, kill_timeout, env, cwd)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/envoy/core.py", line 93, in run
raise self.exc
OSError: [Errno 2] No such file or directory

./_build/fix_html_title.sh html/content/pages/web-server.md

sed: 1: "html/content/pages/web- ...": extra characters at the end of h command

make html
pelican /Users/baiyuxiong/code/500lines/html/content -o /Users/baiyuxiong/code/500lines/html/output -s /Users/baiyuxiong/code/500lines/html/pelicanconf.py

Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/2.7/bin/pelican", line 7, in
from pelican import main
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pelican/init.py", line 19, in
from pelican.generators import (ArticlesGenerator, PagesGenerator,
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pelican/generators.py", line 20, in
from pelican.cache import FileStampDataCacher
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pelican/cache.py", line 10, in
from pelican.utils import mkdir_p
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pelican/utils.py", line 29, in
from six.moves.html_parser import HTMLParser
ImportError: No module named html_parser
make: *** [html] Error 1

cp -a ./blockcode/blockcode-images html/output/pages/

cp -a ./ci/ci-images html/output/pages/

.......

Angular usage?

How does the super-large angular.js framework, used in the "Spreadsheet" chapter, fit into the philosophy of less-then-500-lines? The library itself has many thousands lines and is one of the largest web frameworks ever written.

Creating a small spreadsheet app using a vanilla ES6, on the other hand, would fit into 500 lines of HTML+CSS+JS while maintaining most of the functionality offered in the angular-based scenario.

Can anyone provide a project of a search engine?

Regex Implementation

I recently wrote a 150 line Regex implementation and accompanying blog post -- someone mentioned this book. Would you guys be interested on me creating a chapter for it?

Impl: https://github.com/rcoh/toyregex/blob/master/regex.scala
Post: http://rcoh.svbtle.com/no-magic-regular-expressions

preprocessor.py inserts leading '/' in code block for template engine chapter

See first code block under 'Helper Functions' section. There is a /self and /msg, which I guess has something to do with the leading paren to its left.

Running preprocessor.py on the source markdown inserts these slashes, so there must be a regexp in there that is misbehaving.

Incomplete citation in Dagoba article

Footnote 28 reads: "You can learn more about dependency resolution in ."

This is a placeholder issue to remind us to add the correct citation once that chapter is released.

MAking your own filters

"in a layout called the Sunflower layout. It’s the most efficient way to lay out circles." --> what does it mean for this to be the most "efficient way" to lay out circles?
At least this part could be a link.

500 Lines or *Fewer*

Less describes continuous quantities such as water. Fewer describes countable objects such as lines of code.

I'm not going to cite to a dictionary or other authority. I'm not going to argue that some English speakers do not often use less to describe discrete objects (though I would argue that most English speakers use the words as I describe above). I merely think that fewer would sound better in the title of this project than less does at least partially because of their traditional meanings. And based on a hypothesis of a correlation between interests in computational and natural grammar, and I would bet there are some programmers out there who may agree with me.

Just throwin' it out there.

Broken refs to figure 4 in Contingent article

At http://aosabook.org/en/500L/contingent-a-fully-dynamic-build-system.html I see these missing bits:

Refer again to .

Looking back at , you can see that,

In the markdown file for the chapter at these points are references to a figure 4; there's some kind of web formatting error. Figure 4.4 is itself missing, too.

some image maybe 404

I love 500lines articles, but some image load fail. please check.
like this link http://aosabook.org/en/500L/pages/blockcode-a-visual-programming-toolkit.html

URL's to figures are broken

All URL's to figures in all chapters are broken. For example, http://aosabook.org/en/500L/pages/blockcode-images/blockcode_ide.png

Test print issues

I've received the test print and given it a first scan. Here are the issues I've found:

• Introduction: "subdirectory of project folder" -> "subdirectory of each project folder"
• Introduction: Leah Hanson's italics have a bare "emph" in the beginning
• Blockcode: Spare ')' at the end of "important vehicles for learning"
• CI: control clow diagram "below is overview diagram of this system", missing "of"
• Cluster: 'class Scout' got orphaned on bottom of p.46
• Cluster: pp.52 class Timer doesn't really seem like it needs to be punted all the way to the facing page
• Cluster: "Paxos Made Practical" reference in References section should be an href, not a footnote
• Dagoba: "Method chaining lets us write" footnote got cut off after "instead of:"
• Image filter: pp.238 bottom: "Notice that:" is by itself bottom of page
• Modeller: pp.272 one line of code made it under the big image
• Object model: pp.292/293 huge whitespace before codeblock
• SOP: pp.350 bad quotes on "*.mit.edu"

I'm going to fix these now.

cc: @amyrbrown

Announce official print+web release of 500lines

Hi all,

I am looking forward to when these are released, but I'm unsure how to subscribe to that notification. Right now I'm just "watching" the repo but this is a very noisy way for me to watch. Mail list? meta-issue that I can subscribe to?

Thanks!
Michael

a Chinese translation of 'interpreter.markdown'

Hi, thank you for your hard work. I learned a lot from the chapter "A Python Interpreter Written in Python" and I have translated it into Chinese here, hoping more people can learn from it.