port of INSTALL file generation to XSLT

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

port of INSTALL file generation to XSLT

Peter Eisentraut-6
A further step toward getting rid of the DSSSL tool chain requirement,
here is a patch to change the generation of the text INSTALL file to use
XLST and Pandoc.

The change to Pandoc is not essential to this change, but it creates
much better looking output and simplifies the locale/encoding handling
over using Lynx.

We'll need to get Pandoc installed on borka and check that that version
works as well as the one I have been using.

--
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


--
Sent via pgsql-hackers mailing list ([hidden email])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

0001-Create-INSTALL-file-via-XSLT-and-Pandoc.patch (8K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: port of INSTALL file generation to XSLT

Magnus Hagander-2


On Sat, Dec 31, 2016 at 6:57 AM, Peter Eisentraut <[hidden email]> wrote:
A further step toward getting rid of the DSSSL tool chain requirement,
here is a patch to change the generation of the text INSTALL file to use
XLST and Pandoc.

The change to Pandoc is not essential to this change, but it creates
much better looking output and simplifies the locale/encoding handling
over using Lynx.

Yeah, that seems a lot more clean than using Lynx.

 

We'll need to get Pandoc installed on borka and check that that version
works as well as the one I have been using.

Borka being a standard debian jessie install has Pandoc 1.12.4.2~dfsg-1+b14. Should be no problem installing that.

I ran a "make INSTALL" on a jessie box, and it comes out with some formatting still in the file, see attachment. That seems incorrect.

--


--
Sent via pgsql-hackers mailing list ([hidden email])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

INSTALL (89K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: port of INSTALL file generation to XSLT

Peter Eisentraut-6
On 12/31/16 07:33, Magnus Hagander wrote:
> Borka being a standard debian jessie install has Pandoc
> 1.12.4.2~dfsg-1+b14. Should be no problem installing that.
>
> I ran a "make INSTALL" on a jessie box, and it comes out with some
> formatting still in the file, see attachment. That seems incorrect.

It appears we need pandoc 1.13 to get the good output.  This won't be
available until Debian stretch.

So for PG10, I propose the attached revised patch which keeps using lynx
but uses xsltproc instead of jade.

--
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


--
Sent via pgsql-hackers mailing list ([hidden email])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

v2-0001-Create-INSTALL-file-via-XSLT.patch (9K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: port of INSTALL file generation to XSLT

Magnus Hagander-2


On Mon, Feb 27, 2017 at 4:55 PM, Peter Eisentraut <[hidden email]> wrote:
On 12/31/16 07:33, Magnus Hagander wrote:
> Borka being a standard debian jessie install has Pandoc
> 1.12.4.2~dfsg-1+b14. Should be no problem installing that.
>
> I ran a "make INSTALL" on a jessie box, and it comes out with some
> formatting still in the file, see attachment. That seems incorrect.

It appears we need pandoc 1.13 to get the good output.  This won't be
available until Debian stretch.

So for PG10, I propose the attached revised patch which keeps using lynx
but uses xsltproc instead of jade.


It is available for people not using debian though? Might it be worthwhile to make it dependent on the version of pandoc -- so use that method if people have the newer pandoc and fall back to lynx if they don't? 

--
Reply | Threaded
Open this post in threaded view
|

Re: port of INSTALL file generation to XSLT

Peter Eisentraut-6
On 2/28/17 08:57, Magnus Hagander wrote:

>     It appears we need pandoc 1.13 to get the good output.  This won't be
>     available until Debian stretch.
>
>     So for PG10, I propose the attached revised patch which keeps using lynx
>     but uses xsltproc instead of jade.
>
>
> It is available for people not using debian though? Might it be
> worthwhile to make it dependent on the version of pandoc -- so use that
> method if people have the newer pandoc and fall back to lynx if they don't?

Well, this really only runs once every couple of months on borka and
here or there for those building their own snapshot tarballs.  I don't
think we need to cater to a wide audience here.  In fact, variety could
be bad here:  We don't want to find out that a tarball was rolled with
the wrong variant.

The pandoc change can be revisited independently.  The main point right
now is to get away from the DSSSL toolchain.

--
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


--
Sent via pgsql-hackers mailing list ([hidden email])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply | Threaded
Open this post in threaded view
|

Re: port of INSTALL file generation to XSLT

Magnus Hagander-2
On Wed, Mar 1, 2017 at 3:58 AM, Peter Eisentraut <[hidden email]> wrote:
On 2/28/17 08:57, Magnus Hagander wrote:
>     It appears we need pandoc 1.13 to get the good output.  This won't be
>     available until Debian stretch.
>
>     So for PG10, I propose the attached revised patch which keeps using lynx
>     but uses xsltproc instead of jade.
>
>
> It is available for people not using debian though? Might it be
> worthwhile to make it dependent on the version of pandoc -- so use that
> method if people have the newer pandoc and fall back to lynx if they don't?

Well, this really only runs once every couple of months on borka and
here or there for those building their own snapshot tarballs.  I don't
think we need to cater to a wide audience here.  In fact, variety could
be bad here:  We don't want to find out that a tarball was rolled with
the wrong variant.

Good point. Let 's just go for it then.


--
Reply | Threaded
Open this post in threaded view
|

Re: port of INSTALL file generation to XSLT

Peter Eisentraut-6
On 3/1/17 20:23, Magnus Hagander wrote:

> On Wed, Mar 1, 2017 at 3:58 AM, Peter Eisentraut
> <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     On 2/28/17 08:57, Magnus Hagander wrote:
>     >     It appears we need pandoc 1.13 to get the good output.  This won't be
>     >     available until Debian stretch.
>     >
>     >     So for PG10, I propose the attached revised patch which keeps using lynx
>     >     but uses xsltproc instead of jade.
>     >
>     >
>     > It is available for people not using debian though? Might it be
>     > worthwhile to make it dependent on the version of pandoc -- so use that
>     > method if people have the newer pandoc and fall back to lynx if they don't?
>
>     Well, this really only runs once every couple of months on borka and
>     here or there for those building their own snapshot tarballs.  I don't
>     think we need to cater to a wide audience here.  In fact, variety could
>     be bad here:  We don't want to find out that a tarball was rolled with
>     the wrong variant.
>
>
> Good point. Let 's just go for it then.

Committed that way.  Thanks.

--
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


--
Sent via pgsql-hackers mailing list ([hidden email])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply | Threaded
Open this post in threaded view
|

Re: port of INSTALL file generation to XSLT

Peter Eisentraut-6
In reply to this post by Peter Eisentraut-6
On 2/27/17 10:55, Peter Eisentraut wrote:
> It appears we need pandoc 1.13 to get the good output.  This won't be
> available until Debian stretch.

I understand that borka is updated to stretch now.  So we could give
this another try.  Updated patch attached.

--
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

0001-Create-INSTALL-file-using-Pandoc.patch (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: port of INSTALL file generation to XSLT

Mitar
The following review has been posted through the commitfest application:
make installcheck-world:  not tested
Implements feature:       not tested
Spec compliant:           not tested
Documentation:            tested, failed

Hi!

I tested this. Patch applied cleanly and INSTALL file was produced. Formatting looks differently from before, but I think that it looks better. We lost central alignment of some headings, but many code/command snippets are now better/correctly indented. So I think this is overall better.

Documentation about documentation generation should be updated though. Please add pandoc to requirements here [1].

[1] https://www.postgresql.org/docs/devel/docguide-toolsets.html


Mitar

The new status of this patch is: Waiting on Author
Reply | Threaded
Open this post in threaded view
|

Re: port of INSTALL file generation to XSLT

Peter Eisentraut-6
On 09/01/2019 10:05, Mi Tar wrote:
> I tested this. Patch applied cleanly and INSTALL file was produced. Formatting looks differently from before, but I think that it looks better. We lost central alignment of some headings, but many code/command snippets are now better/correctly indented. So I think this is overall better.
>
> Documentation about documentation generation should be updated though. Please add pandoc to requirements here [1].
>
> [1] https://www.postgresql.org/docs/devel/docguide-toolsets.html

Committed with a fix for that, thanks.

--
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Reply | Threaded
Open this post in threaded view
|

Re: port of INSTALL file generation to XSLT

Tom Lane-2
Peter Eisentraut <[hidden email]> writes:
> On 09/01/2019 10:05, Mi Tar wrote:
>> I tested this. Patch applied cleanly and INSTALL file was produced. Formatting looks differently from before, but I think that it looks better. We lost central alignment of some headings, but many code/command snippets are now better/correctly indented. So I think this is overall better.

> Committed with a fix for that, thanks.

So this has got a couple of problems:

1. No pandoc on borka, where we build tarballs:

pgsql@borka:~$ which pandoc
pgsql@borka:~$

Probably we can get the sysadmin team to fix that, but ...

2. If there's no pandoc, this coding silently produces a zero-size
INSTALL file.  I do not find that acceptable.

pgsql@borka:~$ mk-release 96b8b8b6f9d8de4af01a77797273ad88c7a8e32e
...
Commit 96b8b8b6f9d8de4af01a77797273ad88c7a8e32e packaged as  postgresql-12devel.tar.bz2 postgresql-12devel.tar.bz2.md5 postgresql-12devel.tar.bz2.sha256 postgresql-12devel.tar.gz postgresql-12devel.tar.gz.md5 postgresql-12devel.tar.gz.sha256
pgsql@borka:~$ tar tvfj output/postgresql-12devel.tar.bz2 | grep INSTALL
-rw-r--r-- pgsql/pgsql       0 2019-01-11 15:09 postgresql-12devel/INSTALL


                        regards, tom lane

Reply | Threaded
Open this post in threaded view
|

Re: port of INSTALL file generation to XSLT

Tom Lane-2
I wrote:
> 1. No pandoc on borka, where we build tarballs:
> pgsql@borka:~$ which pandoc
> pgsql@borka:~$

That part's sorted, anyway.

pgsql@borka:~$ pandoc --version
pandoc 1.17.2
Compiled with texmath 0.8.6.7, highlighting-kate 0.6.3.
Syntax highlighting is supported for the following languages:
...


> 2. If there's no pandoc, this coding silently produces a zero-size
> INSTALL file.  I do not find that acceptable.

Seems like it might be sufficient for the rule to be

        $(PANDOC) $< -t plain > $@.tmp
        $(ICONV) -f utf8 -t us-ascii//TRANSLIT < $@.tmp > $@
        rm -f $@.tmp

Failure would leave a .tmp file behind, but I doubt we care enough
about that to work harder than this.

                        regards, tom lane

Reply | Threaded
Open this post in threaded view
|

Re: port of INSTALL file generation to XSLT

Mitar
Hi!

On Fri, Jan 11, 2019 at 1:05 PM Tom Lane <[hidden email]> wrote:
> Failure would leave a .tmp file behind, but I doubt we care enough
> about that to work harder than this.

Maybe just make sure that "make clean" removes it?


Mitar

--
http://mitar.tnode.com/
https://twitter.com/mitar_m

Reply | Threaded
Open this post in threaded view
|

Re: port of INSTALL file generation to XSLT

Peter Eisentraut-6
In reply to this post by Tom Lane-2
On 11/01/2019 22:05, Tom Lane wrote:
>> 2. If there's no pandoc, this coding silently produces a zero-size
>> INSTALL file.  I do not find that acceptable.
>
> Seems like it might be sufficient for the rule to be
>
> $(PANDOC) $< -t plain > $@.tmp
> $(ICONV) -f utf8 -t us-ascii//TRANSLIT < $@.tmp > $@
> rm -f $@.tmp

Fixed.  (We use the same pattern for the dtrace invocation.)

I suppose the same issue would have existed with the previous rule using
lynx, but no one has ever run into it.

--
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services