rielly-open-problems dept.

Rik Van Riel, linux kernel developer, posted a call to arms a few years ago for CS researchers to
consider focusing anew on systems research. Rob Pike wrote a paper in 2000, basically chiding the research community saying that Systems Software Research is Irrelevant -- the title sounds misleading: you would think Pike claims systems research as a field was dead; what he actually writes in the paper is that most researchers are into performance analyses (that too, using flawed methods) rather than building new kinds of systems, and so recent research is mediocre and irrelevant :-).

(Before all you grad students start throwing your Systems dissertations at me, this is what Rob Pike -- member of UNIX team at Bell Labs, programming style guru, father of Inferno and Plan9 -- claims, not me :-)).

I recently read a paper (CAR) about, of all things, Cache replacement algorithms. I came across the above links while researching ARC, the cache replacement algorithm for databases that CAR is based on (CAR is a clock-based implementation of ARC that's more suitable for OS page cache replacement). ARC is a pretty effective scan-resistant algorithm (it performs well even for bursts of sequential accesses that totally screw up LRU), and very simple conceptually (which makes it all the more brilliant). Turns out a lot of commodity open-source OS kernels (Linux/OpenSolaris) and Databases (PostgreSQL) considered and then abandoned ARC implementations because of patent issues with IBM. Too bad :-(.

If, what Rob Pike and Rik Van Riel are saying is true, I have a hypothesis for why this may be happening. Systems Research saw its greatest achievements bank in the '70s and '80s. This was when commodity hardware was not available, and software was at a pretty raw stage -- if you needed a compiler, there was no gcc, you had to write your own -- this is when most researchers grappled with building abstractions and systems -- because they had no choice -- there were no abstractions to work with.

Now that we have reasonably stable abstractions (they may not be the best, but they exist :-)) like UNIX, files and mouse-driven windowing systems, researchers have moved on to other areas. Systems problems are an annoyance at best, and not a high-priority. If systems problems were a bug in a bug-tracking system, they were triaged as "P5, S5" a long time ago.

Perhaps with increasing disk and memory size, hybrid drives, network speeds rapidly outpacing abysmal storage speeds etc. etc., we might see these bugs clawing their way back onto the priority heap. Until then, I leave you with the humorous, yet sublime words of Rob Pike:

...I started keeping a list of these annoyances but it got too long and depressing so I just learned to live with them again. We really are using a 1970s era operating system well past its sell-by date. We get a lot done, and we have fun, but let's face it, the fundamental design of Unix is older than many of the readers of Slashdot, while lots of different, great ideas about computing and networks have been developed in the last 30 years. Using Unix is the computing equivalent of listening only to music by David Cassidy.