pegging trunk to specific versions of dependencies

[Joshua Bronson]

In getting a working dev instance running at dev.melkjug.org, I've recently
had to peg melkjug trunk to specific versions of various dependencies in its
setup.py to prevent incompatible newer versions from being installed:

http://trac.openplans.org/melkjug/changeset/1940
http://trac.openplans.org/melkjug/changeset/1941
http://trac.openplans.org/melkjug/changeset/1948
I remember hearing about some ups and downs to putting this in setup.py...
What are best practices re dependency management? Should I be going about
this some other way?

There could very well be additional newer packages that are getting
installed that have introduced breakages I haven't run across yet. Should I
go ahead and peg trunk to all the specific package versions I have in my
working local instance (installed I can't remember how long ago)?

[Luke Tucker]

Josh,

For dev versions if we have a really strict blocker requirement I
think it's okay as a temporary measure. I do not think we should peg
to versions otherwise in dev because we should move along with these
packages when there are bug fixes or other good reasons to do so. It
also gets us a head start in discovering incompatibilities etc. in
later versions. When there is a problem, we can make a judgement call
about whether to change our code or wait to update to theirs. In this
case, I'd actually like to see a bit more info: http://trac.openplans.org/melkjug/ticket/336#comment
:2

For deployable production / versions in testing we use an installer
that maintains a fixed copy of the exact dependency we want to deploy
with.
These are conservative versions that have been proven in development
practice and are known to work.

The curdle package is used to build these installers ( https://svn.openplans.org/melk/curdle/trunk/
)

To update a dependancy for production, you can alter (for example)
https://svn.openplans.org/melk/curdle/trunk/deps/
update the ref in:
https://svn.openplans.org/melk/curdle/trunk/Makefile

Note these deps must satisfy what is listed in the setup.py's of
various packages or the installer will fail.

- Luke

Post by Joshua Bronson
In getting a working dev instance running at dev.melkjug.org, I've
recently had to peg melkjug trunk to specific versions of various
dependencies in its setup.py to prevent incompatible newer versions
http://trac.openplans.org/melkjug/changeset/1940
http://trac.openplans.org/melkjug/changeset/1941
http://trac.openplans.org/melkjug/changeset/1948
I remember hearing about some ups and downs to putting this in
setup.py... What are best practices re dependency management? Should
I be going about this some other way?
There could very well be additional newer packages that are getting
installed that have introduced breakages I haven't run across yet.
Should I go ahead and peg trunk to all the specific package versions
I have in my working local instance (installed I can't remember how
long ago)?

[Joshua Bronson]

Josh,
For dev versions if we have a really strict blocker requirement I think
it's okay as a temporary measure. I do not think we should peg to versions
otherwise in dev because we should move along with these packages when there
are bug fixes or other good reasons to do so. It also gets us a head start
in discovering incompatibilities etc. in later versions. When there is a
problem, we can make a judgement call about whether to change our code or
wait to update to theirs.

Agreed, thanks for laying it down.
http://trac.openplans.org/melkjug/changeset/1949 should be more in line with
this. Note I'm maintaining a dependency on Pylons<0.9.7dev since I think it
will take a bit more work than we want to invest at the moment to get up to
date. I put in a ticket for this so we can prioritize it accordingly:
http://trac.openplans.org/melkjug/ticket/337

http://trac.openplans.org/melkjug/ticket/336#comment:2

http://trac.openplans.org/melkjug/ticket/336#comment:3 (and by "i'm stumped"
i meant "this is probably not that crazy a bug but it's not immediately
obvious and i'm not sure if i should be spending much more time on this at
the moment given my other priorities" phew)

For deployable production / versions in testing we use an installer that
maintains a fixed copy of the exact dependency we want to deploy with.
These are conservative versions that have been proven in development practice and are known to work.
The curdle package is used to build these installers (
https://svn.openplans.org/melk/curdle/trunk/ )
To update a dependancy for production, you can alter (for example)
https://svn.openplans.org/melk/curdle/trunk/deps/
https://svn.openplans.org/melk/curdle/trunk/Makefile
Note these deps must satisfy what is listed in the setup.py's of various
packages or the installer will fail.
- Luke
In getting a working dev instance running at dev.melkjug.org, I've
recently had to peg melkjug trunk to specific versions of various
dependencies in its setup.py to prevent incompatible newer versions from
http://trac.openplans.org/melkjug/changeset/1940
http://trac.openplans.org/melkjug/changeset/1941
http://trac.openplans.org/melkjug/changeset/1948
I remember hearing about some ups and downs to putting this in setup.py...
What are best practices re dependency management? Should I be going about
this some other way?
There could very well be additional newer packages that are getting
installed that have introduced breakages I haven't run across yet. Should I
go ahead and peg trunk to all the specific package versions I have in my
working local instance (installed I can't remember how long ago)?

[Luke Tucker]

On Tue, Jan 27, 2009 at 11:41 AM, Luke Tucker
Josh,
For dev versions if we have a really strict blocker requirement I
think it's okay as a temporary measure. I do not think we should
peg to versions otherwise in dev because we should move along with
these packages when there are bug fixes or other good reasons to do
so. It also gets us a head start in discovering incompatibilities
etc. in later versions. When there is a problem, we can make a
judgement call about whether to change our code or wait to update to
theirs.
Agreed, thanks for laying it down. http://trac.openplans.org/melkjug/changeset/1949
should be more in line with this. Note I'm maintaining a
dependency on Pylons<0.9.7dev since I think it will take a bit more
work than we want to invest at the moment to get up to date. I put
in a ticket for this so we can prioritize it accordingly: http://trac.openplans.org/melkjug/ticket/337

Thanks, yeah I do want to get up to speed with Pylons as soon as it
makes sense.

In this case, I'd actually like to see a bit more info: http://trac.openplans.org/melkjug/ticket/336#comment
:2
http://trac.openplans.org/melkjug/ticket/336#comment:3 (and by "i'm
stumped" i meant "this is probably not that crazy a bug but it's not
immediately obvious and i'm not sure if i should be spending much
more time on this at the moment given my other priorities" phew)

Cool, not a huge deal, I think I know what's up and it's fine to work
around it. (attached some notes to the ticket)

- Luke