get rid of distprep?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

get rid of distprep?

Peter Eisentraut-6
I'm thinking about whether we should get rid of the distprep target, the
step in the preparation of the official source tarball that creates a
bunch of prebuilt files using bison, flex, perl, etc. for inclusion in
the tarball.  I think this concept is no longer fitting for contemporary
software distribution.

There is a lot of interest these days in making the artifacts of
software distribution traceable, for security and legal reasons.  You
can trace the code from an author into Git, from Git into a tarball,
somewhat from a tarball into a binary package (for example using
reproduceable builds), from a binary package onto a user's system.
Having some mystery prebuilt files in the middle there does not feel
right.  Packaging guidelines nowadays tend to disfavor such practices
and either suggest, recommend, or require removing and rebuilding such
files.  This whole thing was fairly cavalier when we shipped gram.c,
scan.c, and one or two other files, but now the number of prebuilt files
is more than 100, not including the documentation, so this is a bit more
serious.

Practically, who even uses source tarballs these days?  They are a
vehicle for packagers, but packagers are not really helped by adding a
bunch of prebuilt files.  I think this practice started before there
even were things like rpm.  Nowadays, most people who want to work with
the source should and probably do use git, so making the difference
between a git checkout and a source tarball smaller would probably be
good.  And it would also make the actual tarball smaller.

The practical costs of this are also not negligible.  Because of the
particular way configure handles bison and flex, it happens a bunch of
times on new and test systems that the build proceeds and then tells you
you should have installed bison 5 minutes ago.   Also, extensions cannot
rely on bison, flex, or perl being available, except it often works so
it's not dealt with correctly.

Who benefits from these prebuilt files?  I doubt anyone actually has
problems obtaining useful installations of bison, flex, or perl.  There
is the documentation build, but that also seems pretty robust nowadays
and in any case you don't need to build the documentation to get a
useful installation.  We could make some adjustments so that not
building the documentation is more accessible.  The only users of this
would appear to be those not using git and not using any packaging.
That number is surely not zero, but it's probably very small and doesn't
seem worth catering to specifically.

Thoughts?

--
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


Reply | Threaded
Open this post in threaded view
|

Re: get rid of distprep?

Tom Lane-2
Peter Eisentraut <[hidden email]> writes:
> I'm thinking about whether we should get rid of the distprep target, ...

> Who benefits from these prebuilt files?  I doubt anyone actually has
> problems obtaining useful installations of bison, flex, or perl.

I'm sure it was a bigger issue twenty years ago, but yeah, nowadays
our minimum requirements for those tools are so ancient that everybody
who cares to build from source should have usable versions available.

I think the weak spot in your argument, though, is the documentation.
There is basically nothing that is standardized or reproducible in
that toolchain, as every platform names and subdivides the relevant
packages differently, if they exist at all.  I was reminded of that
just recently when I updated my main workstation to RHEL8, and had to
jump through a lot of hoops to get everything installed that's needed
to build the docs (and I still lack the tools for some of the weirder
products such as epub).  I'd be willing to say "you must have bison,
flex, and perl to build" --- and maybe we could even avoid having a
long discussion about what "perl" means in this context --- but I
fear the doc tools situation would be a mess.

> The only users of this
> would appear to be those not using git and not using any packaging.

No, there's the packagers themselves who would be bearing the brunt of
rediscovering how to build the docs on their platforms.  And if the
argument is that there's a benefit to them of making the build more
reproducible, I'm not sure I buy it, because of (1) timestamps in the
output files and (2) docbook's willingness to try to download missing
bits off the net.  (2) is a huge and not very obvious hazard to
reproducibility.

But maybe you ought to be surveying -packagers about the question
instead of theorizing here.  Would *they* see this as a net benefit?

One other point to consider is that distprep or no distprep, I'd be
quite sad if the distclean target went away.  That's extremely useful
in normal development workflows to tear down everything that depends
on configure output, without giving up some of the more expensive
build products such as gram.c and preproc.c.

                        regards, tom lane