Happy New Year! A Note on the Cool Papers Development Experience

新年快乐！记录一下 Cool Papers 的开发体验 苏剑林 · 2024-01-01 · Machine Learning website paper cool paper

Last week, in I Built a Website to Help Me Skim Papers: Cool Papers], I shared a paper-skimming website I built myself, called Cool Papers, and it got a nice reception from some users. However, "the more people use it, the more problems surface" — once the user count picked up, I realized just how sloppy my earlier code really was. I've spent the entire past week fixing bugs non-stop, and as of this afternoon I was still finding new ones to fix. This post is a quick summary of my thoughts from the process of building the site and squashing bugs.

Cool Papers: https://papers.cool

Technology

As it happens, the domain "papers.cool" has been registered for more than four years, which shows that I actually had plans for something like Cool Papers a long time ago, and even made a few early prototypes. The reason the site only came into being for real four years later boils down to a single root cause: my skills weren't up to it. more

On one hand, my web development skills weren't up to it. I don't really know how to build websites — at best I can do minor patchwork fixes here and there. Although this blog, "Scientific Spaces," has been running for over a decade, it's built on an off-the-shelf blogging system, installed the way you'd install any piece of software, and even the template is just a haphazard modification of someone else's open-source theme. Building a complete website from scratch touches on all sorts of technical areas, and as an outsider I simply couldn't keep up. On the other hand, the models themselves weren't up to it either — without a sufficiently capable model to assist with paper-skimming, even if I cobbled together a website, what would its selling point be? How could paper-skimming actually be made "cool"?

Fortunately, the arrival of large models has, to some extent, solved both problems at once. On the web-development side, whatever I don't know how to do, I can just ask GPT-4 or Kimi directly. As long as you're patient, have ideas, and have some basic programming/web knowledge, you can get a website built. It has to be said that large models are an incredibly powerful productivity tool for programming — almost all of Cool Papers' source code was written with the help of GPT-4 and Kimi. On the model side, Kimi supports contexts of up to 128k tokens, which is enough to feed an entire paper in and get accurate FAQ answers out — undoubtedly a very cool way to quickly get a handle on a paper. So the selling point was there too. In this context of large models, Cool Papers seemed almost ready to happen on its own.

Art

That said, things weren't quite so simple. Large models solved the "technology" problem, but not the "art" problem. For instance, with the help of large models, given an existing website, I might be able to copy it more or less faithfully — but writing one from scratch myself, that's where I completely fall apart. This is an "art," or rather "aesthetics," problem — in web development terms, it's called the "front end." Quite a few readers probably think Cool Papers looks a bit ugly, and I'm sorry, but I really did try my best — what you see now is already the result of repeated tweaking on top of a template GPT-4 wrote for me. Large models can rescue me from my lack of technical skill, but they can't rescue me from my utterly blank artistic sense.

What's worse, I frequently get bogged down in obsessive attention to detail — sometimes I can't write a single line of code for half a day just because I haven't decided how to name a variable, or I'll spend ages adjusting a margin or padding by half a pixel, and so on. For website development, where the sheer volume of work matters a great deal, this kind of obsessiveness is clearly a huge liability, and for front-end work in particular — which cares about overall aesthetics rather than local details — it's even worse. So really, I was never cut out for this line of work. Throughout the whole development process, all I could do was keep telling myself "good enough is good enough," and ask users to put up with "good enough" alongside me. If any front-end wizards out there are willing to help make it prettier, I would be deeply grateful.

Backend

Having talked at length about the front end, let's talk about the back end. Put simply, "website = frontend + backend," and "frontend = HTML + CSS + JS" — this part is fairly universal web programming. The backend, on the other hand, is a much broader space: this blog's backend, for example, uses PHP, once hailed as "the best language ever." For me, the only programming language I'm actually familiar with is Python, so naturally I went with Python for development. There are many frameworks for building websites in Python — Django, Flask, Tornado, and so on — and I chose a rather niche option: Bottle.

The reason I used Bottle is actually simple: it was the very first Python web framework I encountered years ago, so I just stuck with it. Really, any of them would do about as well; for Cool Papers what mattered was being somewhat lightweight, and more importantly, the underlying logic driving the whole site.

The biggest difference between Cool Papers and a typical website is that it has no content (papers) of its own — all its content comes from elsewhere (currently Arxiv), so the backend involves fetching content from other sites. Initially, when it was only used internally with a small number of users, this part of the code was written directly into the page routing — that is, content was downloaded in real time whenever a user visited the page. Although Arxiv provides an API for fetching paper data, that API has rate limits, and once the user base grew, the interval between requests could become very short, creating a real risk of a very high request rate that would get all downloads banned by Arxiv.

So, for the sake of stability, all operations involving network communication need to go through a queue with a stable access interval. Specifically, three parts need such a queue: 1) fetching the day's list of papers from Arxiv; 2) downloading paper PDFs from Arxiv (for Kimi to use); 3) conversing with Kimi to generate the FAQ. Figuring out how to design these three queues so that they run and interact stably, without "dragging each other down," took quite a lot of my time. In particular, many bugs only reveal themselves once traffic picks up, so this period of bug-fixing has been, quite literally, non-stop. In the end, even after considering all this, any network operation carries some risk of failure, so the processes in the queue also need a watchdog mechanism to automatically restart after an interruption.

Updates

Since launching last week, Cool Papers has been fortunate enough to win recognition from quite a few readers, who also offered plenty of suggestions for improvement — some of which have already made it into the latest version:

1. Opening up all categories: at launch last week, only a handful of Arxiv categories were supported. As more and more readers requested categories they wanted to see, I decided to open up all categories, and let users choose which ones to display on the homepage;
2. Feed subscription support: many readers have a habit of subscribing via RSS, so it was suggested I add RSS links. This has now been added — though it actually uses the more standard Atom format rather than RSS (nearly all feed readers support both formats anyway);
3. Markdown parsing: the FAQ content generated by Kimi is, to some extent, in Markdown format, and parsing it properly gives a much better reading experience;
4. Click count display: there's now a number after [PDF] and [Kimi], representing the number of times each button has been clicked, which to some extent reflects how popular a given paper is;
5. Other detail improvements: such as improved mobile experience, improved stability of [Kimi], etc.

I've also set up a new GitHub project to keep a changelog for future updates and to gather feedback from users:

GitHub: https://github.com/bojone/papers.cool

There are still some valuable suggestions that haven't yet been implemented in Cool Papers — some are still in development or design, while others might not fit well with what Cool Papers is meant to be. So far, Cool Papers' purpose is to help you "skim/filter" papers, not to "read" them — it's meant to consider features that other paper-reading sites don't offer, specifically in service of fast skimming. For example, the focus of "skimming" is on being "timely" and "comprehensive," so changes that might introduce delays or the risk of missing papers probably won't be adopted.

Lastly, some readers have asked about being able to access historical papers. For papers that are already in the database, you can in fact access them via https://papers.cool/arxiv/<paper_id>. For papers not yet in the database, the load such access would create is still being evaluated (mainly out of concern that [Kimi] might get abused by crawlers), and this might be opened up experimentally later on.

Summary

That's about it for now. Calling it a "summary" is a bit generous — really it's just the rambling account of a website-development newbie, hardly worth much attention, so please bear with me, experts. Finally, happy New Year to everyone — may the coming year bring you nothing but success, may your skills skyrocket, and may bugs and errors vanish without a trace!