Setting rpath on llvmjit.so?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
21 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Setting rpath on llvmjit.so?

Jeff Davis-8
Currently, we don't set rpath for llvmjit.so to include `llvm-config --
libdir`. That means you need to keep LD_LIBRARY_PATH set properly to
use any non-system installation of LLVM.

This isn't unique to LLVM (we only set rpath to the postgres $libdir),
so I thought it might be project policy. But I think it will be worse
for LLVM (and perhaps ICU) where people are more likely to want to run
specific versions. Comments?

Included a simple patch, but it could use some review from someone who
knows more about portable linking and our build system.

Regards,
        Jeff Davis

PS: I also noticed that a autoconf run on master generates some diffs
in configure.


llvm-rpath.diff (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Setting rpath on llvmjit.so?

Andres Freund
Hi,

On 2018-04-14 12:32:35 -0700, Jeff Davis wrote:
> Currently, we don't set rpath for llvmjit.so to include `llvm-config --
> libdir`. That means you need to keep LD_LIBRARY_PATH set properly to
> use any non-system installation of LLVM.

Or pass in the appropriate flags to configure. My configure invoking
script has: LDFLAGS='-Wl,-rpath,/home/andres/build/llvm/6/opt/install/lib' etc
in it.


> This isn't unique to LLVM (we only set rpath to the postgres $libdir),
> so I thought it might be project policy. But I think it will be worse
> for LLVM (and perhaps ICU) where people are more likely to want to run
> specific versions. Comments?

I'm not sure we want to do that - and if so to which libraries. I tend
towards thinking that if we do that, we should do it uniformly, not just
for LLVM. At the very least it'd have to respect --disable-rpath.


> PS: I also noticed that a autoconf run on master generates some diffs
> in configure.

I only see one copyright year related diff. Perhaps you're using a
patched distribution autoconf?

Greetings,

Andres Freund

Reply | Threaded
Open this post in threaded view
|

Re: Setting rpath on llvmjit.so?

Tom Lane-2
Andres Freund <[hidden email]> writes:
> On 2018-04-14 12:32:35 -0700, Jeff Davis wrote:
>> PS: I also noticed that a autoconf run on master generates some diffs
>> in configure.

> I only see one copyright year related diff. Perhaps you're using a
> patched distribution autoconf?

The project policy is to use exactly the GNU distribution of autoconf.
Distros tend to hack it around (especially since it's been so long
since their last release :-(), so you more or less have to install a
private copy to get matching results.  Fortunately it's pretty
trivial to build/install from source.

                        regards, tom lane

Reply | Threaded
Open this post in threaded view
|

Re: Setting rpath on llvmjit.so?

Andres Freund


On April 14, 2018 1:56:08 PM PDT, Tom Lane <[hidden email]> wrote:

>Andres Freund <[hidden email]> writes:
>> On 2018-04-14 12:32:35 -0700, Jeff Davis wrote:
>>> PS: I also noticed that a autoconf run on master generates some
>diffs
>>> in configure.
>
>> I only see one copyright year related diff. Perhaps you're using a
>> patched distribution autoconf?
>
>The project policy is to use exactly the GNU distribution of autoconf.
>Distros tend to hack it around (especially since it's been so long
>since their last release :-(), so you more or less have to install a
>private copy to get matching results.  Fortunately it's pretty
>trivial to build/install from source.

Fwiw, I see one copyright year related diff with unmodified upstream autoconf.

Andres
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Reply | Threaded
Open this post in threaded view
|

Re: Setting rpath on llvmjit.so?

Tom Lane-2
Andres Freund <[hidden email]> writes:
> On April 14, 2018 1:56:08 PM PDT, Tom Lane <[hidden email]> wrote:
>> The project policy is to use exactly the GNU distribution of autoconf.

> Fwiw, I see one copyright year related diff with unmodified upstream autoconf.

Really?  Where did you get autoconf from?  My archived copy of
autoconf-2.69.tar.gz, fetched 2013-11-15, has internal file
dates of (mostly) 2012-04-24.  That matches the file dates at
https://ftp.gnu.org/gnu/autoconf/
so I've not actually pulled down fresh bits to compare, but ...

                        regards, tom lane

Reply | Threaded
Open this post in threaded view
|

Re: Setting rpath on llvmjit.so?

Andres Freund
On 2018-04-14 17:10:21 -0400, Tom Lane wrote:

> Andres Freund <[hidden email]> writes:
> > On April 14, 2018 1:56:08 PM PDT, Tom Lane <[hidden email]> wrote:
> >> The project policy is to use exactly the GNU distribution of autoconf.
>
> > Fwiw, I see one copyright year related diff with unmodified upstream autoconf.
>
> Really?  Where did you get autoconf from?  My archived copy of
> autoconf-2.69.tar.gz, fetched 2013-11-15, has internal file
> dates of (mostly) 2012-04-24.  That matches the file dates at
> https://ftp.gnu.org/gnu/autoconf/
> so I've not actually pulled down fresh bits to compare, but ...

The v2.69 git tag.  So I guess that's set somewhere in the tarball
building. Hrmpf.

The half unmaintainedness of autoconf (no release in five years counts
as that imo), sure makes it look like a good idea to move on to cmake or
such at some point...

Greetings,

Andres Freund

Reply | Threaded
Open this post in threaded view
|

Re: Setting rpath on llvmjit.so?

Yuriy Zhuravlev

2018-04-15 6:13 GMT+09:00 Andres Freund <[hidden email]>:
On 2018-04-14 17:10:21 -0400, Tom Lane wrote:
> Andres Freund <[hidden email]> writes:
> > On April 14, 2018 1:56:08 PM PDT, Tom Lane <[hidden email]> wrote:
> >> The project policy is to use exactly the GNU distribution of autoconf.
>
> > Fwiw, I see one copyright year related diff with unmodified upstream autoconf.
>
> Really?  Where did you get autoconf from?  My archived copy of
> autoconf-2.69.tar.gz, fetched 2013-11-15, has internal file
> dates of (mostly) 2012-04-24.  That matches the file dates at
> https://ftp.gnu.org/gnu/autoconf/
> so I've not actually pulled down fresh bits to compare, but ...

The v2.69 git tag.  So I guess that's set somewhere in the tarball
building. Hrmpf.

The half unmaintainedness of autoconf (no release in five years counts
as that imo), sure makes it look like a good idea to move on to cmake or
such at some point...


My cmake branch still working and I supporting stable postgres releases:
https://github.com/stalkerg/postgres_cmake
 
Also, I looked into Meson but unfortunately some features still not exist for building postgres. 
Anyway, my branch exist, cmake working fine maybe without minor features like generating documentation. 
My cmake version also have minor different behaviour but it because cmake not build your project directly, it more like project generator (same as automake and meson).

We can start conversation about it again, I am open for this. 

Regards
Reply | Threaded
Open this post in threaded view
|

Re: Setting rpath on llvmjit.so?

Robert Haas
In reply to this post by Andres Freund
On Sat, Apr 14, 2018 at 5:13 PM, Andres Freund <[hidden email]> wrote:
> The half unmaintainedness of autoconf (no release in five years counts
> as that imo), sure makes it look like a good idea to move on to cmake or
> such at some point...

I don't necessarily see that as a reason to move.  If we're hitting
bugs in autoconf that are causing problems, and we can't get them
fixed in the upstream, that's a reason.  But switching build systems
won't be frictionless, so I'm not in favor of it unless there's a
clear payoff.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Reply | Threaded
Open this post in threaded view
|

Re: Setting rpath on llvmjit.so?

Tom Lane-2
Robert Haas <[hidden email]> writes:
> ... But switching build systems
> won't be frictionless,

I think we have a nominee for Understatement of the Year.

                        regards, tom lane

Reply | Threaded
Open this post in threaded view
|

Re: Setting rpath on llvmjit.so?

Andres Freund
In reply to this post by Robert Haas
On 2018-04-17 15:56:14 -0400, Robert Haas wrote:

> On Sat, Apr 14, 2018 at 5:13 PM, Andres Freund <[hidden email]> wrote:
> > The half unmaintainedness of autoconf (no release in five years counts
> > as that imo), sure makes it look like a good idea to move on to cmake or
> > such at some point...
>
> I don't necessarily see that as a reason to move.  If we're hitting
> bugs in autoconf that are causing problems, and we can't get them
> fixed in the upstream, that's a reason.  But switching build systems
> won't be frictionless, so I'm not in favor of it unless there's a
> clear payoff.

I'd not advocate for this solely based on the age of autoconf. But the
separate windows buildsystem which makes it very hard to build
extensions separately is a good reason on its own. As is the fact that
recursive make as we're using it has significant issues. Both of those
would be avoided by using cmake or such.

Greetings,

Andres Freund

Reply | Threaded
Open this post in threaded view
|

Re: Setting rpath on llvmjit.so?

Andres Freund
In reply to this post by Yuriy Zhuravlev
Hi,

On 2018-04-16 11:26:14 +0900, Yuriy Zhuravlev wrote:
> My cmake branch still working and I supporting stable postgres releases:
> https://github.com/stalkerg/postgres_cmake

> Anyway, my branch exist, cmake working fine maybe without minor features
> like generating documentation.

> We can start conversation about it again, I am open for this.

Well, before it does everything, there's little point in reviewing
whether it's mergeable or not.

I still think cmake is the least unreasonable path going forward.

Greetings,

Andres Freund

Reply | Threaded
Open this post in threaded view
|

Re: Setting rpath on llvmjit.so?

Yuriy Zhuravlev
Well, before it does everything, there's little point in reviewing
>  whether it's mergeable or not.

For this significant case, it's not working as you expect.
First, Postgres community should find consensus about migration to CMake (or alternative). Now, this project too huge to work on it without sure in importance.
Second, a few main contributors should start helping to fix bugs and deep into architecture. It's very important because build system very tightly bound with all source code and one man can't know all rarely cases. Moreover, it's needed to understand some restriction CMake (and another project generators) and find consensus in the community about it. For example why we can't build contrib modules independent on main postgres with CMake. 
Also, you should understand what you can't keep the whole feature set and behaviors in the new build system, postgres too big for this now (probably you will have many new features but you will lose too). It's why main contributors should know new build system to find solutions and consensus for decisions.

We can open a new thread for discussion about the first question, the second question should be later. 

Reply | Threaded
Open this post in threaded view
|

Re: Setting rpath on llvmjit.so?

Peter Eisentraut-6
In reply to this post by Andres Freund
On 4/17/18 16:14, Andres Freund wrote:
> I still think cmake is the least unreasonable path going forward.

I would rather try to make Meson work and if needed add features back
into Meson.

--
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Reply | Threaded
Open this post in threaded view
|

Re: Setting rpath on llvmjit.so?

Robert Haas
In reply to this post by Andres Freund
On Tue, Apr 17, 2018 at 4:13 PM, Andres Freund <[hidden email]> wrote:
> I'd not advocate for this solely based on the age of autoconf. But the
> separate windows buildsystem which makes it very hard to build
> extensions separately is a good reason on its own. As is the fact that
> recursive make as we're using it has significant issues. Both of those
> would be avoided by using cmake or such.

How much of the pain building extensions on Windows comes from our
build system and how much of it comes from the fact that it's Windows?
 My guess would have been that the latter is by far the bigger issue,
but maybe I'm wrong.

There is really nothing keeping us from removing or reducing the use
of recursive make without switching to cmake.  And it's probably not
even that much work.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Reply | Threaded
Open this post in threaded view
|

Re: Setting rpath on llvmjit.so?

Tom Lane-2
In reply to this post by Peter Eisentraut-6
Peter Eisentraut <[hidden email]> writes:
> On 4/17/18 16:14, Andres Freund wrote:
>> I still think cmake is the least unreasonable path going forward.

> I would rather try to make Meson work and if needed add features back
> into Meson.

I'm wondering whether that will result in expending a lot of effort
to move from a poorly-supported build system to a different
poorly-supported build system.  As a project, we have an astonishingly
poor track record for choosing outside projects to rely on :-(.

One thing I particularly dislike about Meson, after reading its wikipedia
entry for a moment, is that it doesn't use "make" as the backend build
system.  That means two major tooling changes not one, and double the
opportunity to find ourselves stuck with unsupported infrastructure.
I also politely decline the offer to be forced to use XCode on mac.

                        regards, tom lane

Reply | Threaded
Open this post in threaded view
|

Re: Setting rpath on llvmjit.so?

Yuriy Zhuravlev
In reply to this post by Robert Haas
Current autoconf system not working on Windows at all, what we talk about?

On Wed, 18 Apr 2018, 23:57 Robert Haas, <[hidden email]> wrote:
On Tue, Apr 17, 2018 at 4:13 PM, Andres Freund <[hidden email]> wrote:
> I'd not advocate for this solely based on the age of autoconf. But the
> separate windows buildsystem which makes it very hard to build
> extensions separately is a good reason on its own. As is the fact that
> recursive make as we're using it has significant issues. Both of those
> would be avoided by using cmake or such.

How much of the pain building extensions on Windows comes from our
build system and how much of it comes from the fact that it's Windows?
 My guess would have been that the latter is by far the bigger issue,
but maybe I'm wrong.

There is really nothing keeping us from removing or reducing the use
of recursive make without switching to cmake.  And it's probably not
even that much work.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Reply | Threaded
Open this post in threaded view
|

Re: Setting rpath on llvmjit.so?

Andres Freund
On 2018-04-18 15:54:59 +0000, Yuriy Zhuravlev wrote:
> Current autoconf system not working on Windows at all, what we talk about?

We generate windows project files. See src/tools/msvc/

- Andres

Reply | Threaded
Open this post in threaded view
|

Re: Setting rpath on llvmjit.so?

Yuriy Zhuravlev
In reply to this post by Tom Lane-2


I also politely decline the offer to be forced to use XCode on mac.

Why? We supporting MSVC and not nmake, what difference with xcode? Also, it's just extra benefit from cmake/meson.
Reply | Threaded
Open this post in threaded view
|

Re: Setting rpath on llvmjit.so?

Yuriy Zhuravlev
In reply to this post by Andres Freund
I talked about autoconf build system, /tools/msvc it's extra home build system.

On Thu, 19 Apr 2018, 00:58 Andres Freund, <[hidden email]> wrote:
On 2018-04-18 15:54:59 +0000, Yuriy Zhuravlev wrote:
> Current autoconf system not working on Windows at all, what we talk about?

We generate windows project files. See src/tools/msvc/

- Andres
Reply | Threaded
Open this post in threaded view
|

Re: Setting rpath on llvmjit.so?

Jason Petersen
In reply to this post by Tom Lane-2
On Apr 18, 2018, at 8:57 AM, Tom Lane <[hidden email]> wrote:

I'm wondering whether that will result in expending a lot of effort to move from a poorly-supported build system to a different poorly-supported build system.

I’m not sure whether the former is autoconf/make or cmake, but count me as another vote for cmake over meson. CMake has direct support from Microsoft (Visual Studio 2017 can auto-ingest CMake-based files as soon as they’re checked out) and JetBrains CLion and is in use by LLVM, KDE, Qt, Blender, libpng, cURL, LAPACK, MySQL/MariaDB, OpenCV, SDL, the Dolphin Gamecube emulator etc.

So the tooling around people using it is there and it’s used by some very large and mature projects covering a variety of domains: I don’t know if it’s “poorly supported” (the documentation leaves something to be desired), but if CMake has trouble down the line a lot of very important projects will be in trouble. It seems like a pretty safe bet given the sheer inertia of the list above.

--
Jason Petersen
Software Engineer | Citus Data
303.736.9255

12