BUG #16825: When building on Windows, cl /? retrun 'x64' not AMD64 and the build does not create x64 environment

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

BUG #16825: When building on Windows, cl /? retrun 'x64' not AMD64 and the build does not create x64 environment

PG Bug reporting form
The following bug has been logged on the website:

Bug reference:      16825
Logged by:          Robert Grange
Email address:      [hidden email]
PostgreSQL version: 13.1
Operating system:   Windows
Description:        

Hi,

in src\tools\msvc\Solution.pm the following test is not sufficient
                $self->{platform} =
                  ($output =~ /^\/favor:<.+AMD64/m) ? 'x64' : 'Win32';
because the cl /? returns:
                Microsoft (R) C/C++ Optimizing Compiler Version 19.28.29336 for x64
                Copyright (C) Microsoft Corporation.  All rights reserved.

better to check like this ?
                $self->{platform} =
                  ($output =~ /^\/favor:<.+AMD64/m) ? 'x64' : ($output =~ /.x64$/m) ?
'x64' : 'Win32';

Regards
Robert

Reply | Threaded
Open this post in threaded view
|

Re: BUG #16825: When building on Windows, cl /? retrun 'x64' not AMD64 and the build does not create x64 environment

Juan José Santamaría Flecha


On Fri, Jan 15, 2021 at 9:21 AM PG Bug reporting form <[hidden email]> wrote:

in src\tools\msvc\Solution.pm the following test is not sufficient
                $self->{platform} =
                  ($output =~ /^\/favor:<.+AMD64/m) ? 'x64' : 'Win32';
because the cl /? returns:
                Microsoft (R) C/C++ Optimizing Compiler Version 19.28.29336 for x64
                Copyright (C) Microsoft Corporation.  All rights reserved.
 
I see no bug here, that expression is looking for the line: "/favor:<blend|AMD64|INTEL64|ATOM> select processor to optimize for, one of:", which is one of the first printed options.
To make sure this is working in your environment, you should see "Detected hardware platform: x64" when building.

Regards,

Juan José Santamaría Flecha


 
Reply | Threaded
Open this post in threaded view
|

RE: BUG #16825: When building on Windows, cl /? retrun 'x64' not AMD64 and the build does not create x64 environment

robionekenobi

Hi,

 

Yes, I see

 

Don’t know why when I’m doing the cl /? In the Visual Studio path of the x64 Native Command Prompt, the response contains all the help (including the /flavor: ….)

 

Once I change to the postgresql\src\tools\msvc for to build, the ‘cl /?’ Only gives back 2 lines and say invalid option.

 

But I also remarked that if you pass ‘cl -?’, it works correctly in both case. Do you think it is possible to change the ‘cl /?’ by ‘cl -?’ ?

 

Thanks for your help

 

Regards

Robert

 

 

From: Juan José Santamaría Flecha <[hidden email]>
Sent: Saturday, January 16, 2021 10:16 PM
To: [hidden email]; PostgreSQL mailing lists <[hidden email]>
Subject: Re: BUG #16825: When building on Windows, cl /? retrun 'x64' not AMD64 and the build does not create x64 environment

 

 

 

On Fri, Jan 15, 2021 at 9:21 AM PG Bug reporting form <[hidden email]> wrote:


in src\tools\msvc\Solution.pm the following test is not sufficient
                $self->{platform} =
                  ($output =~ /^\/favor:<.+AMD64/m) ? 'x64' : 'Win32';
because the cl /? returns:
                Microsoft (R) C/C++ Optimizing Compiler Version 19.28.29336 for x64
                Copyright (C) Microsoft Corporation.  All rights reserved.

 

I see no bug here, that expression is looking for the line: "/favor:<blend|AMD64|INTEL64|ATOM> select processor to optimize for, one of:", which is one of the first printed options.

To make sure this is working in your environment, you should see "Detected hardware platform: x64" when building.

 

Regards,

 

Juan José Santamaría Flecha

 

 

 

Reply | Threaded
Open this post in threaded view
|

RE: BUG #16825: When building on Windows, cl /? retrun 'x64' not AMD64 and the build does not create x64 environment

robionekenobi
In reply to this post by Juan José Santamaría Flecha

Hi again,

 

I searched a little bit more, and this occurs only if the drive where I run the ‘cl /?’ Is a ‘Subst Drive’

 

Therefor, I’ve changed this to a real Drive, and the error does no more occurs. It is a bug in windows with ‘Subst drive’ probably

 

So, you can either close the bug, or change the ‘cl /?’ by a ‘cl /help’ (which is also an official cl option)

 

Regards

Robert

 

From: [hidden email] <[hidden email]>
Sent: Sunday, January 17, 2021 9:31 AM
To: 'Juan José Santamaría Flecha' <[hidden email]>; 'PostgreSQL mailing lists' <[hidden email]>
Subject: RE: BUG #16825: When building on Windows, cl /? retrun 'x64' not AMD64 and the build does not create x64 environment

 

Hi,

 

Yes, I see

 

Don’t know why when I’m doing the cl /? In the Visual Studio path of the x64 Native Command Prompt, the response contains all the help (including the /flavor: ….)

 

Once I change to the postgresql\src\tools\msvc for to build, the ‘cl /?’ Only gives back 2 lines and say invalid option.

 

But I also remarked that if you pass ‘cl -?’, it works correctly in both case. Do you think it is possible to change the ‘cl /?’ by ‘cl -?’ ?

 

Thanks for your help

 

Regards

Robert

 

 

From: Juan José Santamaría Flecha <[hidden email]>
Sent: Saturday, January 16, 2021 10:16 PM
To: [hidden email]; PostgreSQL mailing lists <[hidden email]>
Subject: Re: BUG #16825: When building on Windows, cl /? retrun 'x64' not AMD64 and the build does not create x64 environment

 

 

 

On Fri, Jan 15, 2021 at 9:21 AM PG Bug reporting form <[hidden email]> wrote:


in src\tools\msvc\Solution.pm the following test is not sufficient
                $self->{platform} =
                  ($output =~ /^\/favor:<.+AMD64/m) ? 'x64' : 'Win32';
because the cl /? returns:
                Microsoft (R) C/C++ Optimizing Compiler Version 19.28.29336 for x64
                Copyright (C) Microsoft Corporation.  All rights reserved.

 

I see no bug here, that expression is looking for the line: "/favor:<blend|AMD64|INTEL64|ATOM> select processor to optimize for, one of:", which is one of the first printed options.

To make sure this is working in your environment, you should see "Detected hardware platform: x64" when building.

 

Regards,

 

Juan José Santamaría Flecha

 

 

 

Reply | Threaded
Open this post in threaded view
|

Re: BUG #16825: When building on Windows, cl /? retrun 'x64' not AMD64 and the build does not create x64 environment

Juan José Santamaría Flecha

On Sun, Jan 17, 2021 at 7:22 PM <[hidden email]> wrote:

Hi again,

 

I searched a little bit more, and this occurs only if the drive where I run the ‘cl /?’ Is a ‘Subst Drive’

 

Therefor, I’ve changed this to a real Drive, and the error does no more occurs. It is a bug in windows with ‘Subst drive’ probably

 

So, you can either close the bug, or change the ‘cl /?’ by a ‘cl /help’ (which is also an official cl option)


Thanks for the reproducible example, it behaves like that for me too.

I guess 'cl' is interpreting "/?" as filename, not as option, when the current path is a ‘Subst Drive’. If you call 'cl /? foo' it works, and also if you create a one-letter named dummy file inside the ‘Subst Drive’.

I would close this report noting that using a ‘Subst Drive’ is problematic anyhow, as other scripts might also fail when navigating the expected folder layout.

Regards,

Juan José Santamaría Flecha
Reply | Threaded
Open this post in threaded view
|

Re: BUG #16825: When building on Windows, cl /? retrun 'x64' not AMD64 and the build does not create x64 environment

Michael Paquier-2
On Mon, Jan 18, 2021 at 11:42:33PM +0100, Juan José Santamaría Flecha wrote:
> On Sun, Jan 17, 2021 at 7:22 PM <[hidden email]> wrote:
> I guess 'cl' is interpreting "/?" as filename, not as option, when the
> current path is a ‘Subst Drive’. If you call 'cl /? foo' it works, and also
> if you create a one-letter named dummy file inside the ‘Subst Drive’.
>
> I would close this report noting that using a ‘Subst Drive’ is problematic
> anyhow, as other scripts might also fail when navigating the expected
> folder layout.

If you patch the scripts by changing the option used by the cl
command, are there cases where the installation able to complete when
using a subst driver?  It seems to me that if things work could work
even in this case and result in a correct installation, then there is
an argument for making the MSVC scripts easier to use for the
end-user.
--
Michael

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: BUG #16825: When building on Windows, cl /? retrun 'x64' not AMD64 and the build does not create x64 environment

Juan José Santamaría Flecha

On Tue, Jan 19, 2021 at 6:23 AM Michael Paquier <[hidden email]> wrote:

If you patch the scripts by changing the option used by the cl
command, are there cases where the installation able to complete when
using a subst driver?  It seems to me that if things work could work
even in this case and result in a correct installation, then there is
an argument for making the MSVC scripts easier to use for the
end-user.

The scripts assume that the 'msvc' folder is three levels deep, 'src/tools/msvc'. If you respect that, you can build with the 'cl /help' patch. Please find attached the patch for so.

Regards,

Juan José Santamaría

0001-WIN32-Make-x64-arquitecture-detection-more-resilient.patch (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: BUG #16825: When building on Windows, cl /? retrun 'x64' not AMD64 and the build does not create x64 environment

Michael Paquier-2
On Tue, Jan 19, 2021 at 09:31:22AM +0100, Juan José Santamaría Flecha wrote:
> The scripts assume that the 'msvc' folder is three levels deep,
> 'src/tools/msvc'. If you respect that, you can build with the 'cl /help'
> patch. Please find attached the patch for so.

So, I have been playing with subst and some virtual drives, and this
allows to run builds if for example pointing to the top of a git repo,
which is fancy.

Requiring at least three repository levels is something that we are
going to need anyway as a build requires a full access to the tree,
but I really see no reason to help users with such configurations as
that's basically free for us to do so.  I can see the difference of
output between cl /? and /help for virtual drives, and that's really
surprising, but I see no reason to not apply this patch as this breaks
nothing and /help is an official option.

If there are no objections, I am planning to apply this patch to
HEAD.
--
Michael

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: BUG #16825: When building on Windows, cl /? retrun 'x64' not AMD64 and the build does not create x64 environment

Michael Paquier-2
On Wed, Jan 20, 2021 at 04:24:47PM +0900, Michael Paquier wrote:
> If there are no objections, I am planning to apply this patch to
> HEAD.

Done as of 733d670.  There may be a point in back-patching and
classifying that as a bug, but I have played it conservative as that's
an improvement and nobody really complained about that AFAIK.
--
Michael

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: BUG #16825: When building on Windows, cl /? retrun 'x64' not AMD64 and the build does not create x64 environment

Jurko Gospodnetić
   Hi.

On 21.1.2021. 3:12, Michael Paquier wrote:

> On Wed, Jan 20, 2021 at 04:24:47PM +0900, Michael Paquier wrote:
>> If there are no objections, I am planning to apply this patch to
>> HEAD.
>
> Done as of 733d670.  There may be a point in back-patching and
> classifying that as a bug, but I have played it conservative as that's
> an improvement and nobody really complained about that AFAIK.
> --
> Michael
>

   Thanks for patching this.

   And actually I did report it a while ago, just hadn't found the time
to follow up and rebase the patch to develop:

------

   Hi.

   I'm not sure what the process is here so just attempting a polite ping.

   I've posted a message (reattached to the bottom of this e-mail) about
this problem, including a fix for it, almost a month about and there has
thus far been no response nor can I see the change applied to

https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/tools/msvc/Solution.pm;hb=HEAD

   Please let me know if I should be posting about this somewhere else.

   Kind regards,
     Jurko Gospodnetić


On 7.10.2020. 18:08, PG Bug reporting form wrote:
 > The following bug has been logged on the website:
 >
 > Bug reference:      16660
 > Logged by:          Jurko Gospodnetić
 > Email address:      [hidden email]
 > PostgreSQL version: 9.6.19
 > Operating system:   Windows
 > Description:
 >
 > If you run the 64-bit postgres build from a virtual subst drive (see
 > Windows `help subst` command), it will fail.
 >
 > Internally it will fail to detect that the used Visual C++ compiler
 > executable `cl.exe` has been configured for producing 64-bit targets
 > and will thus configure the generated Visual Studio projects &
 > solution for building 32-bit targets even through the used compiler
 > will actually be producing 64-bit ones.
 >
 > Even deeper, the original problem stems from `cl.exe` behaviour, which
 > seems to produce no compiler option output when invoked as `cl /?`
 > from a subst drive. On one occasion it was reported that it did show
 > compiler options on some subst drives but not others, but this could
 > not be reproduced later on so such behaviour can be at most considered
 > random.
 >
 > Invoking the executable as `cl /help` works correctly on all tested
 > drives. Go figure.
 >
 > Here's a patch fixing the problem based on the code in the 9.6.19
 > distribution:

 From a3ccfe68dbe5190f743cd13a4fc90faaf6478370 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jurko=20Gospodneti=C4=87?= <[hidden email]>
Date: Wed, 7 Oct 2020 16:17:29 +0200
Subject: [PATCH] patch compiler architecture detection on virtual subst
drives

---
src/tools/msvc/Solution.pm | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm
index 7ec7120..29ebab5 100644
--- a/src/tools/msvc/Solution.pm
+++ b/src/tools/msvc/Solution.pm
@@ -75,8 +75,13 @@ sub DeterminePlatform
   {
      my $self = shift;
-    # Examine CL help output to determine if we are in 32 or 64-bit mode.
-    my $output = `cl /? 2>&1`;
+    # examine CL help output to determine if we are in 32 or 64-bit mode
+    #
+    # we intentionally use `cl /help` as opposed to `cl /?` as the
former works
+    # when the current drive is a virtual `subst` drive (see `help
subst`) while
+    # the latter does not (noticed at least with compiler executable
version
+    # `Microsoft (R) C/C++ Optimizing Compiler Version 19.16.27043 for
x64`)
+    my $output = `cl /help 2>&1`;
      $? >> 8 == 0 or die "cl command not found";
      $self->{platform} = ($output =~ /^\/favor:<.+AMD64/m) ? 'x64' :
'Win32';
      print "Detected hardware platform: $self->{platform}\n";


Reply | Threaded
Open this post in threaded view
|

Re: BUG #16825: When building on Windows, cl /? retrun 'x64' not AMD64 and the build does not create x64 environment

Michael Paquier-2
On Fri, Jan 22, 2021 at 01:11:30PM +0100, Jurko Gospodnetić wrote:
> And actually I did report it a while ago, just hadn't found the time to
> follow up and rebase the patch to develop:

Indeed, my apologies for that.  pgsql-bugs is a list with a lot of
traffic, so it is easy to miss something.  I can see that this thread
is not registered in the commit fest app, making this even easier to
forget:
https://commitfest.postgresql.org/
--
Michael

signature.asc (849 bytes) Download Attachment