Matt Barkau
2009-03-31 05:46:41 UTC
Hi Luke; thanks for the explanation. I've signed up on the dev list, and
will keep an eye on INewsRatingService.
Once there are a large number of users, it might be cool to try different
weighting algorithm tweaks on random groups of users, measure the usage
trending, pick the best, and apply that feedback to the ratings engine.
Thanks again.
Matt
will keep an eye on INewsRatingService.
Once there are a large number of users, it might be cool to try different
weighting algorithm tweaks on random groups of users, measure the usage
trending, pick the best, and apply that feedback to the ratings engine.
Thanks again.
Matt
I'll note that we're not doing proper collaborative filtering yet... but
melkjug does do a lot of offline/background processing that happens
periodically and some that is farmed out in response to certain user
requests. We take pains to make sure that for large instances this work can
happen in separate services / servers -- for lighter
instances run with paster, the work is just executed by threadpools in the
web server process. That said, I'm not sure I would say that processor load
has been mitigated -- we're still definitely figuring it out as we go and
the load can get quite painful even if nobody is looking... :)
- Luke
Thanks a lot, Matt! I had fun giving it
I'm actually not the best person to ask, as my contributions to melkjug
have been pretty minimal. I gave the talk because the rest of the team
couldn't make it to pycon. I've included CC'ed Luke Tucker, mekjug's evil
genius, and he should be able to answer this.
Doug
Hi Doug!
I had to head back home this evening, but had a couple questions on Melkjug
architecture - by the way, that was the funniest talk I heard this year.
(Runner up was the assertion that "backslashes are evil because they have
goatees, and goatees are, of course, evil". Probably not funny without the
supporting slides, though.)
Anyway,,, for Melkjug's collaborative filtering, how did you mitigate the
processor requirements for that extra computation (more than typical
webserver load)? Or was it actually not a big server hit? Does that work
run only nightly? Is there some other black art which made this a light
load?
I ask because I'm in the very early stages of planning an independent
personal health ~forum which will attempt to ontologically bridge western
and eastern health care - launchpad.net/selfhealth . It's planned to be on
Plone, using OpenPlans, at selfhealth.org.
Thanks much!
melkjug does do a lot of offline/background processing that happens
periodically and some that is farmed out in response to certain user
requests. We take pains to make sure that for large instances this work can
happen in separate services / servers -- for lighter
instances run with paster, the work is just executed by threadpools in the
web server process. That said, I'm not sure I would say that processor load
has been mitigated -- we're still definitely figuring it out as we go and
the load can get quite painful even if nobody is looking... :)
- Luke
Thanks a lot, Matt! I had fun giving it
I'm actually not the best person to ask, as my contributions to melkjug
have been pretty minimal. I gave the talk because the rest of the team
couldn't make it to pycon. I've included CC'ed Luke Tucker, mekjug's evil
genius, and he should be able to answer this.
Doug
Hi Doug!
I had to head back home this evening, but had a couple questions on Melkjug
architecture - by the way, that was the funniest talk I heard this year.
(Runner up was the assertion that "backslashes are evil because they have
goatees, and goatees are, of course, evil". Probably not funny without the
supporting slides, though.)
Anyway,,, for Melkjug's collaborative filtering, how did you mitigate the
processor requirements for that extra computation (more than typical
webserver load)? Or was it actually not a big server hit? Does that work
run only nightly? Is there some other black art which made this a light
load?
I ask because I'm in the very early stages of planning an independent
personal health ~forum which will attempt to ontologically bridge western
and eastern health care - launchpad.net/selfhealth . It's planned to be on
Plone, using OpenPlans, at selfhealth.org.
Thanks much!