I've dumped here the very basics on how I'm running the critters in docker. https://github.com/linux-on-ibm-power/docker_pgbuildfarm Might get a chance to write more doco next week. |
Daniel, Your config is seriously awry on pretty much all your animals. I see this: 'invocation_args' => [ '--verbose=2', '--keepall', '--from-source=/source', 'HEAD' ], from-source should never be used for a reporting animal. Quite apart from any other reason, it inhibits git updates. It's designed for offline use. I use it sometimes to check patches by running them through a very complete set of tests. I'm mildly surprised that it actually reports at all with this option, and I will stop it from doing so in the next release. Please fix this ASAP. You also probably don't want --keepall either, nor --verbose=2, but those are less serious. cheers andrew |
On 09/09/2016 09:54 AM, Andrew Dunstan wrote: > > I'm mildly surprised that it actually reports at all with this option, > and I will stop it from doing so in the next release. Actually, the script has this already. $nosend is set if $from_source is used. Have you hacked the run script? If so, please don't. cheers andrew |
In reply to this post by Andrew Dunstan
Andrew,
Relax, the git updates are done outside of the pgbuildfarm scripts. Below is the start of the log that triggered the build this morning. The keepall has no effect as anything done in the run stage is removed. I'll reexamine verbose, I think that was debugging only. Perhaps I can get $scm->log_id() into the build to prove this a bit more. As a server side thought, maybe even check this value server side to ensure that results aren't getting submitting when/if people's mirror scripts break for some reason. 07:59:28 Started by upstream project "Pgbuildfarm/REL9___6___STABLE" build number 14 07:59:28 originally caused by: 07:59:28 Branch indexing 07:59:28 Branch indexing 07:59:28 Branch indexing 07:59:28 Branch indexing 07:59:28 Branch indexing 07:59:29 [lockable-resources] acquired lock on [ka4-perf-1] 07:59:29 [EnvInject] - Loading node environment variables. 07:59:29 Building remotely on p84-vm (ubuntu ppc64le_docker_build power ppc64le_docker docker ppc64le) in workspace /var/lib/jenkins/workspace/Pgbuildfarm/REL9___6___STABLE/CC/gcc/CONFIG/ppc64le/HOST_OS/debian-8.4 07:59:29 [WS-CLEANUP] Deleting project workspace... 07:59:29 [WS-CLEANUP] Done 07:59:29 Using checkout strategy: Default 07:59:29 Last Built Revision: Revision f6802936a53671c9db24273ab2ae94291a68afcd (origin/REL9_6_STABLE) 07:59:29 Cloning the remote Git repository 07:59:30 Cloning repository git://gitlab.ozlabs.ibm.com/mirror/postgresql.git 07:59:30 > git init /var/lib/jenkins/workspace/Pgbuildfarm/REL9___6___STABLE/CC/gcc/CONFIG/ppc64le/HOST_OS/debian-8.4 # timeout=30 07:59:30 ERROR: Reference path does not exist: /var/lib/jenkins/git/postgresql.git 07:59:30 Fetching upstream changes from git://gitlab.ozlabs.ibm.com/mirror/postgresql.git 07:59:30 > git --version # timeout=30 07:59:30 > git -c core.askpass=true fetch --tags --progress git://gitlab.ozlabs.ibm.com/mirror/postgresql.git +refs/heads/*:refs/remotes/origin/* 08:00:22 > git config remote.origin.url git://gitlab.ozlabs.ibm.com/mirror/postgresql.git # timeout=30 08:00:22 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=30 08:00:22 > git config remote.origin.url git://gitlab.ozlabs.ibm.com/mirror/postgresql.git # timeout=30 08:00:22 Fetching upstream changes from git://gitlab.ozlabs.ibm.com/mirror/postgresql.git 08:00:22 > git -c core.askpass=true fetch --tags --progress git://gitlab.ozlabs.ibm.com/mirror/postgresql.git +refs/heads/*:refs/remotes/origin/* 08:00:22 > git config remote.origin1.url git://gitlab.ozlabs.ibm.com/mirror/postgresql.git # timeout=30 08:00:22 Fetching upstream changes from git://gitlab.ozlabs.ibm.com/mirror/postgresql.git 08:00:22 > git -c core.askpass=true fetch --tags --progress git://gitlab.ozlabs.ibm.com/mirror/postgresql.git +refs/remotes/*:refs/remotes/* 08:00:22 Checking out Revision fa7b3a88dd5d7f6d18614eefc6cc0662e31bd451 (origin/REL9_6_STABLE) 08:00:22 > git config core.sparsecheckout # timeout=30 08:00:22 > git checkout -f fa7b3a88dd5d7f6d18614eefc6cc0662e31bd451 08:00:23 > git rev-list f6802936a53671c9db24273ab2ae94291a68afcd # timeout=30 08:00:24 Run condition [Always] enabling prebuild for step [Archive the artifacts] 08:00:24 Set build name. 08:00:24 New build name is '#14_origin/REL9_6_STABLE_fa7b3a88dd5d7f_gcc_ppc64le' 08:00:24 [debian-8.4] $ /bin/bash -xe /tmp/hudson1484895481194831141.sh 08:00:24 + mkdir .source 08:00:24 + mv aclocal.m4 config configure configure.in contrib COPYRIGHT doc GNUmakefile.in HISTORY Makefile README README.git src .git .source 08:00:24 + mv .source source 08:00:24 + mkdir artifacts 08:00:24 + export https_proxy=http://proxy:3142/ http_proxy=http://proxy:3142/ ftp_proxy=http://proxy:3128/ 08:00:24 + https_proxy=http://proxy:3142/ 08:00:24 + http_proxy=http://proxy:3142/ 08:00:24 + ftp_proxy=http://proxy:3128/ 08:00:24 + git clone --branch secret --recursive https://gitlab.ozlabs.ibm.com/CI/pgbuildfarm-script.git scripts 08:00:24 Cloning into 'scripts'... 08:00:24 + cd scripts 08:00:24 + ./docker_run_build.sh 08:00:24 + set -euo pipefail 08:00:24 + DOC=jenkins-pgbuildfarm-debian-8.4-gcc 08:00:24 + mkdir -m 0777 -p /var/lib/jenkins/workspace/Pgbuildfarm/REL9___6___STABLE/CC/gcc/CONFIG/ppc64le/HOST_OS/debian-8.4/artifacts /var/lib/jenkins/workspace/Pgbuildfarm/REL9___6___STABLE/CC/gcc/CONFIG/ppc64le/HOST_OS/debian-8.4/ccache 08:00:24 ++ id -u 08:00:24 + docker build --build-arg=BUILD_UID=1000 --build-arg=CC=gcc -t jenkins-pgbuildfarm-debian-8.4-gcc --file Docker.debian-8.4 . 08:00:25 Sending build context to Docker daemon 150.5 kB 08:00:25 Step 1 : FROM debian:8.4 08:00:25 ---> e0a06a89ed81 08:00:25 Step 2 : ARG BUILD_UID 08:00:25 ---> Using cache 08:00:25 ---> 1e5c3052ee81 08:00:25 Step 3 : ARG CC 08:00:25 ---> Using cache 08:00:25 ---> 112d62a74f5a 08:00:25 Step 4 : ENV DEBIAN_FRONTEND noninteractive 08:00:25 ---> Using cache 08:00:25 ---> 1916fcad9ff1 08:00:25 Step 5 : ENV http_proxy http://proxy:3142/ 08:00:25 ---> Using cache 08:00:25 ---> 424d4ef4f6ae 08:00:25 Step 6 : ENV https_proxy http://proxy:3142/ 08:00:25 ---> Using cache 08:00:25 ---> 1086c2a06ab5 08:00:25 Step 7 : ENV ftp_proxy http://proxy:3128/ 08:00:25 ---> Using cache 08:00:25 ---> 8aa0d6c5635d 08:00:25 Step 8 : VOLUME /ccache /artifacts /source 08:00:25 ---> Using cache 08:00:25 ---> 623f4bdc87c3 08:00:25 Step 9 : RUN for p in http https ftp; do echo Acquire::${p}::proxy '"http://proxy:3128";' ; done >> /etc/apt/apt.conf.d/proxy 08:00:25 ---> Using cache 08:00:25 ---> d2d59ab6f116 08:00:25 Step 10 : RUN sed -e 's/^deb/deb-src/g' /etc/apt/sources.list > /etc/apt/sources.list.d/src.list && apt-get update && apt-get upgrade -y && apt-get build-dep -y postgresql && apt-get install -y curl software-properties-common ccache sudo libwww-perl libhttp-message-perl libdigest-sha-perl libbsd-resource-perl python-dev libreadline-dev zlib1g-dev libgss-dev libssl-dev libxml2-dev libxslt1-dev libldap2-dev libkrb5-dev tcl8.6-dev tcl-dev libperl-dev bison flex 08:00:25 ---> Using cache 08:00:25 ---> becbb0b8e4f1 08:00:25 Step 11 : RUN curl --proxy proxy:3128 http://www.pgbuildfarm.org/downloads/latest-client.tgz | tar -zxf - && ln -s build-farm-* build-farm 08:00:25 ---> Using cache 08:00:25 ---> f36f872dcc21 08:00:25 Step 12 : RUN /bin/bash -c "[ \"${CC}\" == 'clang' ] && apt-get install -y clang || true" 08:00:25 ---> Using cache 08:00:25 ---> 19d88445b121 08:00:25 Step 13 : RUN /bin/bash -c "set -v; if [[ `uname -m` == \"ppc64le\" && \"${CC}\" =~ ^at ]]; then curl --proxy proxy:3128 ftp://ftp.unicamp.br/pub/linuxpatch/toolchain/at/debian/dists/jessie/6976a827.gpg.key | apt-key add -; [[ \"${CC}\" =~ ^at[[:digit:]]+\.[[:digit:]]+$ ]] && echo 'deb ftp://ftp.unicamp.br/pub/linuxpatch/toolchain/at/debian jessie '${CC} >> /etc/apt/sources.list || echo 'deb http://ausgsa.ibm.com/projects/p/ppctoolchain/web/public/at/debian jessie '${CC%-*} >> /etc/apt/sources.list; apt-get update && apt-get install -y advance-toolchain-${CC}-devel; fi" 08:00:25 ---> Using cache 08:00:25 ---> a9c673d7d4d4 08:00:25 Step 14 : RUN /bin/bash -c "set -v ; if [[ \"${CC}\" =~ 'gcc-' ]]; then echo 'deb http://httpredir.debian.org/debian unstable main' > /etc/apt/sources.list.d/unstable.list && apt-get update && apt-get install -y ${CC} ; fi" 08:00:25 ---> Using cache 08:00:25 ---> 308e80392e08 08:00:25 Step 15 : RUN /bin/bash -c "if [[ \"${CC}\" =~ clang- ]]; then if [[ \"${CC}\" =~ clang-4 ]]; then echo \"deb http://apt.llvm.org/jessie/ llvm-toolchain-jessie main\" >> /etc/apt/sources.list; else echo \"deb http://apt.llvm.org/jessie/ llvm-toolchain-jessie-${CC#clang-} main\" >> /etc/apt/sources.list; fi ; curl --proxy proxy:3128 http://apt.llvm.org/llvm-snapshot.gpg.key | apt-key add - && apt-get update && apt-get install -y \"${CC}\" || exit 1 ; fi" 08:00:25 ---> Using cache 08:00:25 ---> bbb0d5ca6c81 08:00:25 Step 16 : RUN useradd -m -u $BUILD_UID --home-dir /build build && mkdir /build/postgresql_build && chown build: /build/postgresql_build 08:00:25 ---> Using cache 08:00:25 ---> abbfe7451e48 08:00:25 Step 17 : COPY build_pg.sh /build/build_pg.sh 08:00:25 ---> Using cache 08:00:25 ---> b2af0217bbc0 08:00:25 Step 18 : COPY build-farm.conf build-farm/build-farm.conf 08:00:25 ---> Using cache 08:00:25 ---> cd701f1536c2 08:00:25 Step 19 : COPY secrets.conf build-farm/secrets.conf 08:00:25 ---> Using cache 08:00:25 ---> b705bec5e6c8 08:00:25 Step 20 : RUN sed -i -e '/nosend=1/d' -e '/nostatus=1/d' /build-farm/run_build.pl 08:00:25 ---> Using cache 08:00:25 ---> f1f5643e2fc1 08:00:25 Step 21 : WORKDIR /build 08:00:25 ---> Using cache 08:00:25 ---> d62538c8ef65 08:00:25 Step 22 : CMD /usr/bin/sudo -E -H -u build CC=${CC} GIT_BRANCH=${GIT_BRANCH} ./build_pg.sh 08:00:25 ---> Using cache 08:00:25 ---> 3c45d6937cd9 08:00:25 Successfully built 3c45d6937cd9 08:00:25 + DOCKER_OPT= 08:00:25 + docker run --rm=true --cpu-shares=2 --shm-size=32G --name jenkins-pgbuildfarm-debian-8.4-gcc_REL9_6_STABLE --volume /var/lib/jenkins/workspace/Pgbuildfarm/REL9___6___STABLE/CC/gcc/CONFIG/ppc64le/HOST_OS/debian-8.4/source:/source --volume /var/lib/jenkins/workspace/Pgbuildfarm/REL9___6___STABLE/CC/gcc/CONFIG/ppc64le/HOST_OS/debian-8.4/ccache:/ccache --volume /var/lib/jenkins/workspace/Pgbuildfarm/REL9___6___STABLE/CC/gcc/CONFIG/ppc64le/HOST_OS/debian-8.4/artifacts:/artifacts --env CC=gcc --env HOST_OS=debian-8.4 --env GIT_BRANCH=origin/REL9_6_STABLE -t jenkins-pgbuildfarm-debian-8.4-gcc 08:00:25 + set -v 08:00:25 08:00:25 # keep caches in different directories 08:00:25 CCACHE=$CC 08:00:25 + CCACHE=gcc 08:00:25 08:00:25 # for secret lookup 08:00:25 COMPILER=${CC} 08:00:25 + COMPILER=gcc 08:00:25 08:00:25 case $CC in 08:00:25 at*) 08:00:25 P=$(echo /opt/${CC/-/-*}/bin) 08:00:25 CC=${P}/gcc 08:00:25 AR=${P}/ar 08:00:25 PATH=${P}:${PATH} 08:00:25 ;; 08:00:25 xlc*) 08:00:25 CC=/usr/bin/xlc_r 08:00:25 ;; 08:00:25 *) 08:00:25 which ${CC} || CC=${CC%-*} 08:00:25 ;; 08:00:25 esac 08:00:25 + case $CC in 08:00:25 + which gcc 08:00:25 /usr/bin/gcc 08:00:25 08:00:25 export PERL5LIB=/build-farm 08:00:25 + export PERL5LIB=/build-farm 08:00:25 + PERL5LIB=/build-farm 08:00:25 export CC AR PATH CCACHE COMPILER 08:00:25 + export CC AR PATH CCACHE COMPILER 08:00:25 08:00:25 # Debug 08:00:25 # if ! /build-farm/run_build.pl --nosend --nostatus --verbose=2 --keepall --from-source=/source ${GIT_BRANCH#*/}; 08:00:25 08:00:25 if ! /build-farm/run_build.pl --verbose=2 --keepall --from-source=/source ${GIT_BRANCH#*/}; 08:00:25 then 08:00:25 cp ~/postgresql_build/*/*.fromsource-logs/* /artifacts/ 08:00:25 exit 1 08:00:25 fi 08:00:25 + /build-farm/run_build.pl --verbose=2 --keepall --from-source=/source REL9_6_STABLE 08:00:25 For a ppc64le, on debian-8.4 using gcc I am a bleak 08:00:25 Sun Sep 11 22:00:25 2016: buildfarm run for bleak:REL9_6_STABLE starting 08:00:25 [22:00:25] running configure ... 08:00:55 ======== configure output =========== 08:00:55 checking build system type... powerpc64le-unknown-linux-gnu 08:00:55 checking host system type... powerpc64le-unknown-linux-gnu 08:00:55 checking which template to use... linux 08:00:55 checking whether to build with 64-bit integer date/time support... yes On 09/09/16 23:54, Andrew Dunstan wrote: > > Daniel, > > > Your config is seriously awry on pretty much all your animals. I see this: > > > 'invocation_args' => [ > '--verbose=2', > '--keepall', > '--from-source=/source', > 'HEAD' > ], > > > from-source should never be used for a reporting animal. Quite apart > from any other reason, it inhibits git updates. It's designed for > offline use. I use it sometimes to check patches by running them through > a very complete set of tests. I'm mildly surprised that it actually > reports at all with this option, and I will stop it from doing so in the > next release. > > > Please fix this ASAP. > > > You also probably don't want --keepall either, nor --verbose=2, but > those are less serious. > > > cheers > > > andrew > > |
Daniel Black <[hidden email]> writes:
> Relax, the git updates are done outside of the pgbuildfarm scripts. > Below is the start of the log that triggered the build this morning. The problem with inventing your own approach here is that your buildfarm members' reports do not provide any evidence of what they are building against. Please examine http://buildfarm.postgresql.org/cgi-bin/show_status.pl and note that your animals are not showing last-commit-id in their lines on the summary page, and if you drill down to their build reports, those don't show anything about "changes since last run" either, eg compare http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=demoiselle&dt=2016-09-12%2000%3A36%3A03 http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=guaibasaurus&dt=2016-09-12%2000%3A17%3A01 That greatly reduces the usefulness of the reports. regards, tom lane |
On 09/11/2016 08:49 PM, Tom Lane wrote: > Daniel Black <[hidden email]> writes: >> Relax, the git updates are done outside of the pgbuildfarm scripts. >> Below is the start of the log that triggered the build this morning. > The problem with inventing your own approach here is that your buildfarm > members' reports do not provide any evidence of what they are building > against. Please examine > > http://buildfarm.postgresql.org/cgi-bin/show_status.pl > > and note that your animals are not showing last-commit-id in their lines > on the summary page, and if you drill down to their build reports, those > don't show anything about "changes since last run" either, eg compare > > http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=demoiselle&dt=2016-09-12%2000%3A36%3A03 > http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=guaibasaurus&dt=2016-09-12%2000%3A17%3A01 > > That greatly reduces the usefulness of the reports. > > I also don't see any point in doing this. Please run this software as intended. (BTW, telling someone to relax is quite offensively patronizing.) cheers andre |
On 12/09/16 11:08, Andrew Dunstan wrote: > > > On 09/11/2016 08:49 PM, Tom Lane wrote: >> Daniel Black <[hidden email]> writes: >>> Relax, the git updates are done outside of the pgbuildfarm scripts. >>> Below is the start of the log that triggered the build this morning. >> The problem with inventing your own approach here is that your buildfarm >> members' reports do not provide any evidence of what they are building >> against. Please examine >> >> http://buildfarm.postgresql.org/cgi-bin/show_status.pl >> >> and note that your animals are not showing last-commit-id in their lines >> on the summary page, and if you drill down to their build reports, those >> don't show anything about "changes since last run" either, eg compare >> >> http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=demoiselle&dt=2016-09-12%2000%3A36%3A03 >> >> http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=guaibasaurus&dt=2016-09-12%2000%3A17%3A01 >> >> >> That greatly reduces the usefulness of the reports. so I see. > I also don't see any point in doing this. Mainly fitting in with current infrastructure. > Please run this software as intended. Will look into it. > (BTW, telling someone to relax is quite offensively patronizing.) Wasn't intended. Sorry. |
On 09/11/2016 09:25 PM, Daniel Black wrote: > > >> Please run this software as intended. > Will look into it. > > These machines have not reported for a week. I am going to place them in suspended state pending assurances that they have been fixed so that the client runs as designed. Except in extraordinary cases, there should be no need to change anything except the client's configuration file. cheers andrew |
On 19/09/16 21:47, Andrew Dunstan wrote: > > > On 09/11/2016 09:25 PM, Daniel Black wrote: >> >> >>> Please run this software as intended. >> Will look into it. >> >> > > > These machines have not reported for a week. Apologies - really busy + was ill for a few days. > I am going to place them in > suspended state pending assurances that they have been fixed so that the > client runs as designed. Almost there. > Except in extraordinary cases, there should be no need to change > anything except the client's configuration file. The builds don't always run on the same hardware however the architecture and build environment is identical. |
Daniel Black wrote:
> On 19/09/16 21:47, Andrew Dunstan wrote: > > > > These machines have not reported for a week. > > Apologies - really busy + was ill for a few days. > > > I am going to place them in suspended state pending assurances that > > they have been fixed so that the client runs as designed. > > Almost there. Any news on this? -- Álvaro Herrera PostgreSQL Expert, https://www.2ndQuadrant.com/ -- Sent via buildfarm-members mailing list ([hidden email]) To make changes to your subscription: http://www.postgresql.org/mailpref/buildfarm-members |
Free forum by Nabble | Edit this page |