Jython has slowed down no-op incrementals

[facebook]

Jython has slowed down no-op incrementals

[jimpurbrick]

Thanks for this. I've seen Jython being a significant win for big projects, but I've just timed buck build buck again before and after Jython and seen a slowdown, so I'll see if I can reduce the overhead.

[bolinfest]

@spearce Is this on Mac or Linux? And with an encrypted hard drive or not encrypted?

[spearce]

Linux version 3.2.5, looks like its an unencrypted ext4 filesystem on local disk.

[jimpurbrick]

Using buckd should significantly reduce latency of all buck commands and dramatically reduce the duration of no-op incremental builds of complex projects as build files are cached between builds.

[spearce]

buckd helps, but not as much as I would like to see. :-)

MacOS 10.8.3 (encrypted disk, java 1.7.0_21)
no-op without buckd: 8s
no-op with buckd: 3s

It seems like buckd has roughly gained back the latency loss caused by switching to Jython. But this is at the expense of a lot more complexity. Thus far the only thing we have really gained is removing a dependency on the local system Python, but that falls flat if your project uses a genrule() that runs a python_application().

[spearce]

buckd's security model also isn't viable in all environments; you really have to trust every process on your machine in order to run buckd. Given that nailgun uses a native C binary as the client I'm surprised they don't have a version that uses UNIX domain sockets.

[shs96c]

There have been some discussions at facebook about sniffing the local system to see if there's a python version that meets the requirements for buck (so Python 2.5 with the imports required for buck.py) and using that in preference to jython if asked.

Right now, it's gone no further than a discussion, as for a large tree our measurements suggest using jython is an overall speed win.

[bolinfest]

Fixed! 1023858