[melkjug-dev] Improving Random Article Picks + CouchDB Caching

Luke Tucker

2009-05-13 19:14:41 UTC

Post by Randall Leeds
Taking a look at issue #358 [1]
I started thinking about random selections of CouchDB views again.
Could we implement a _list function as described in [2] which can randomly
spit out K articles?
The page says that CouchDB generates etags for these calls but doesn't seem
to suggest anywhere that they are cached by CouchDB itself, meaning that if
we felt like living dangerously we could make the calls non-idempotent.

Hmm, yeah I don't quite see enough of how/when these are actually run
internally to venture a guess? It seems likely that internally they would
still be tracking something to know when those etags had become invalid that
did not require actually re-running the whole _list function. In that case,
(and in keeping with how most couchy stuff seems to work) they probably only
run again when the docs (and only on the docs) that changed?

Post by Randall Leeds
I might be overlooking some nasty negative effects this has on our
performance and our bucket generation. I know there's been some refactoring
around this since I last fully understood the couch silo.

Probably not, it's all pretty basic.

Post by Randall Leeds
As a (slightly tangential) note: CouchDB tosses back etags for lots of
things. Do we have a cache sitting between melk.model and the couch
instance? Seems like that might be a nice thing to have. If we do, and we
"live dangerously" (as described above), we'd have to configure it to ignore
the _list URIs.

We do not currently have any caching setup for couchdb in deployment, but we
would probably benefit.

- Luke