Skip to content

Flaky linux tests #18539

Closed
Closed

Description

@maclover7
  • Version: master
  • Platform: Fedora24, Ubuntu1404, possibly other Linux distros as well
  • Subsystem: test

There seems to be a more recurring issue with running native addons tests on Linux. I have seen many fail lately with the message "Failed to add alternative version links to index". I pasted two examples below, but I remember there being more in the past.


Example from https://ci.nodejs.org/job/node-test-commit-linux/16068/nodes=ubuntu1404-64/console:

Details
Building addon /home/iojs/build/workspace/node-test-commit-linux/nodes/ubuntu1404-64/test/addons/zlib-binding/
gyp info it worked if it ends with ok
gyp info using node-gyp@3.6.2
gyp info using node@10.0.0-pre | linux | x64
gyp info chdir /home/iojs/build/workspace/node-test-commit-linux/nodes/ubuntu1404-64/test/addons/zlib-binding/
gyp info spawn /usr/bin/python
gyp info spawn args [ '/home/iojs/build/workspace/node-test-commit-linux/nodes/ubuntu1404-64/deps/npm/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '/home/iojs/build/workspace/node-test-commit-linux/nodes/ubuntu1404-64/test/addons/zlib-binding/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/home/iojs/build/workspace/node-test-commit-linux/nodes/ubuntu1404-64/deps/npm/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/home/iojs/build/workspace/node-test-commit-linux/nodes/ubuntu1404-64/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=/home/iojs/build/workspace/node-test-commit-linux/nodes/ubuntu1404-64',
gyp info spawn args   '-Dnode_gyp_dir=/home/iojs/build/workspace/node-test-commit-linux/nodes/ubuntu1404-64/deps/npm/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=/home/iojs/build/workspace/node-test-commit-linux/nodes/ubuntu1404-64/$(Configuration)/node.lib',
gyp info spawn args   '-Dmodule_root_dir=/home/iojs/build/workspace/node-test-commit-linux/nodes/ubuntu1404-64/test/addons/zlib-binding',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.' ]
gyp info spawn make
gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build', '--jobs', 2 ]
make[2]: Entering directory `/home/iojs/build/workspace/node-test-commit-linux/nodes/ubuntu1404-64/test/addons/zlib-binding/build'
  CXX(target) Release/obj.target/binding/binding.o
  SOLINK_MODULE(target) Release/obj.target/binding.node
  COPY Release/binding.node
make[2]: Leaving directory `/home/iojs/build/workspace/node-test-commit-linux/nodes/ubuntu1404-64/test/addons/zlib-binding/build'
gyp info ok 
touch test/addons/.buildstamp
Failed to add alternative version links to index
Aborted (core dumped)
make[2]: *** [out/doc/api/string_decoder.json] Error 134
make[2]: *** Waiting for unfinished jobs....
make[2]: Leaving directory `/home/iojs/build/workspace/node-test-commit-linux/nodes/ubuntu1404-64'
make[1]: *** [doc-only] Error 2
make[1]: Leaving directory `/home/iojs/build/workspace/node-test-commit-linux/nodes/ubuntu1404-64'
make: *** [run-ci] Error 2
Build step 'Execute shell' marked build as failure
Run condition [Always] enabling perform for step [[]]
Run condition [Always] enabling perform for step [[]]
TAP Reports Processing: START
Looking for TAP results report in workspace using pattern: *.tap
Did not find any matching files. Setting build result to FAILURE.
Checking ^not ok
Jenkins Text Finder: File set '*.tap' is empty
Notifying upstream projects of job completion
Finished: FAILURE

Example from https://ci.nodejs.org/job/node-test-commit-linux/16068/nodes=ubuntu1404-64/console:

Details
Building addon /home/iojs/build/workspace/node-test-commit-linux/nodes/ubuntu1404-64/test/addons/zlib-binding/
gyp info it worked if it ends with ok
gyp info using node-gyp@3.6.2
gyp info using node@10.0.0-pre | linux | x64
gyp info chdir /home/iojs/build/workspace/node-test-commit-linux/nodes/ubuntu1404-64/test/addons/zlib-binding/
gyp info spawn /usr/bin/python
gyp info spawn args [ '/home/iojs/build/workspace/node-test-commit-linux/nodes/ubuntu1404-64/deps/npm/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '/home/iojs/build/workspace/node-test-commit-linux/nodes/ubuntu1404-64/test/addons/zlib-binding/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/home/iojs/build/workspace/node-test-commit-linux/nodes/ubuntu1404-64/deps/npm/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/home/iojs/build/workspace/node-test-commit-linux/nodes/ubuntu1404-64/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=/home/iojs/build/workspace/node-test-commit-linux/nodes/ubuntu1404-64',
gyp info spawn args   '-Dnode_gyp_dir=/home/iojs/build/workspace/node-test-commit-linux/nodes/ubuntu1404-64/deps/npm/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=/home/iojs/build/workspace/node-test-commit-linux/nodes/ubuntu1404-64/$(Configuration)/node.lib',
gyp info spawn args   '-Dmodule_root_dir=/home/iojs/build/workspace/node-test-commit-linux/nodes/ubuntu1404-64/test/addons/zlib-binding',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.' ]
gyp info spawn make
gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build', '--jobs', 2 ]
make[2]: Entering directory `/home/iojs/build/workspace/node-test-commit-linux/nodes/ubuntu1404-64/test/addons/zlib-binding/build'
  CXX(target) Release/obj.target/binding/binding.o
  SOLINK_MODULE(target) Release/obj.target/binding.node
  COPY Release/binding.node
make[2]: Leaving directory `/home/iojs/build/workspace/node-test-commit-linux/nodes/ubuntu1404-64/test/addons/zlib-binding/build'
gyp info ok 
touch test/addons/.buildstamp
Failed to add alternative version links to index
Aborted (core dumped)
make[2]: *** [out/doc/api/string_decoder.json] Error 134
make[2]: *** Waiting for unfinished jobs....
make[2]: Leaving directory `/home/iojs/build/workspace/node-test-commit-linux/nodes/ubuntu1404-64'
make[1]: *** [doc-only] Error 2
make[1]: Leaving directory `/home/iojs/build/workspace/node-test-commit-linux/nodes/ubuntu1404-64'
make: *** [run-ci] Error 2
Build step 'Execute shell' marked build as failure
Run condition [Always] enabling perform for step [[]]
Run condition [Always] enabling perform for step [[]]
TAP Reports Processing: START
Looking for TAP results report in workspace using pattern: *.tap
Did not find any matching files. Setting build result to FAILURE.
Checking ^not ok
Jenkins Text Finder: File set '*.tap' is empty
Notifying upstream projects of job completion
Finished: FAILURE

Activity

  1. added
    testIssues and PRs related to Node.js core tests and test infrastructure.
    addonsIssues and PRs related to native addons.
    flaky-testIssues and PRs involving tests that fail intermittently in CI.
    on Feb 2, 2018
  2. maclover7 commented on Feb 3, 2018

    @maclover7
    ContributorAuthor

    Happened on https://ci.nodejs.org/job/node-test-commit-plinux/15054/nodes=ppcle-ubuntu1404/console

    Details
    Building addon /home/iojs/build/workspace/node-test-commit-plinux/nodes/ppcle-ubuntu1404/test/addons/zlib-binding/
    gyp info it worked if it ends with ok
    gyp info using node-gyp@3.6.2
    gyp info using node@10.0.0-pre | linux | ppc64
    gyp info chdir /home/iojs/build/workspace/node-test-commit-plinux/nodes/ppcle-ubuntu1404/test/addons/zlib-binding/
    gyp info spawn /usr/bin/python
    gyp info spawn args [ '/home/iojs/build/workspace/node-test-commit-plinux/nodes/ppcle-ubuntu1404/deps/npm/node_modules/node-gyp/gyp/gyp_main.py',
    gyp info spawn args   'binding.gyp',
    gyp info spawn args   '-f',
    gyp info spawn args   'make',
    gyp info spawn args   '-I',
    gyp info spawn args   '/home/iojs/build/workspace/node-test-commit-plinux/nodes/ppcle-ubuntu1404/test/addons/zlib-binding/build/config.gypi',
    gyp info spawn args   '-I',
    gyp info spawn args   '/home/iojs/build/workspace/node-test-commit-plinux/nodes/ppcle-ubuntu1404/deps/npm/node_modules/node-gyp/addon.gypi',
    gyp info spawn args   '-I',
    gyp info spawn args   '/home/iojs/build/workspace/node-test-commit-plinux/nodes/ppcle-ubuntu1404/common.gypi',
    gyp info spawn args   '-Dlibrary=shared_library',
    gyp info spawn args   '-Dvisibility=default',
    gyp info spawn args   '-Dnode_root_dir=/home/iojs/build/workspace/node-test-commit-plinux/nodes/ppcle-ubuntu1404',
    gyp info spawn args   '-Dnode_gyp_dir=/home/iojs/build/workspace/node-test-commit-plinux/nodes/ppcle-ubuntu1404/deps/npm/node_modules/node-gyp',
    gyp info spawn args   '-Dnode_lib_file=/home/iojs/build/workspace/node-test-commit-plinux/nodes/ppcle-ubuntu1404/$(Configuration)/node.lib',
    gyp info spawn args   '-Dmodule_root_dir=/home/iojs/build/workspace/node-test-commit-plinux/nodes/ppcle-ubuntu1404/test/addons/zlib-binding',
    gyp info spawn args   '-Dnode_engine=v8',
    gyp info spawn args   '--depth=.',
    gyp info spawn args   '--no-parallel',
    gyp info spawn args   '--generator-output',
    gyp info spawn args   'build',
    gyp info spawn args   '-Goutput_dir=.' ]
    make[2]: Entering directory `/home/iojs/build/workspace/node-test-commit-plinux/nodes/ppcle-ubuntu1404/test/addons/zlib-binding/build'
      CXX(target) Release/obj.target/binding/binding.o
    gyp info spawn make
    gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build', '--jobs', 4 ]
      SOLINK_MODULE(target) Release/obj.target/binding.node
      COPY Release/binding.node
    make[2]: Leaving directory `/home/iojs/build/workspace/node-test-commit-plinux/nodes/ppcle-ubuntu1404/test/addons/zlib-binding/build'
    gyp info ok 
    touch test/addons/.buildstamp
    make[1]: Leaving directory `/home/iojs/build/workspace/node-test-commit-plinux/nodes/ppcle-ubuntu1404'
    make: *** [run-ci] Error 2
    Process leaked file descriptors. See https://jenkins.io/redirect/troubleshooting/process-leaked-file-descriptors for more information
    Build step 'Execute shell' marked build as failure
    Run condition [Always] enabling perform for step [[]]
    TAP Reports Processing: START
    Looking for TAP results report in workspace using pattern: *.tap
    Did not find any matching files. Setting build result to FAILURE.
    Checking ^not ok
    Jenkins Text Finder: File set '*.tap' is empty
    Sending e-mails to: michael_dawson@ca.ibm.com gib@uk.ibm.com
    Notifying upstream projects of job completion
    Finished: FAILURE
  3. richardlau commented on Feb 3, 2018

    @richardlau
    Member

    Failed to add alternative version links to index is a docs build error rather than addons.

  4. joyeecheung commented on Feb 4, 2018

    @joyeecheung
    Member

    AFAIK Failed to add alternative version links to ... does not cause the build to fail, it just means that the docs built don't have the correct version links but they would build just fine.

  5. joyeecheung commented on Feb 4, 2018

    @joyeecheung
    Member

    The real issue in these logs is this

    Aborted (core dumped)
    

    simply building the docs should not trigger something like this...it would be helpful if we can get one of these cores to debug.

  6. gibfahn commented on Feb 5, 2018

    @gibfahn
    Member

    The real issue in these logs is this

    The third failure doesn't include that error, so maybe a different cause.

    Not sure if the core file actually gets generated (if ulimit -c is set), but if it does it probably gets cleaned up when git clean runs on the next run. So to get a core you'd probably want to take the machine offline, and try to reproduce manually to get a core.

  7. added
    docIssues and PRs related to Node.js documentation.
    and removed
    flaky-testIssues and PRs involving tests that fail intermittently in CI.
    on Feb 6, 2018
  8. changed the title [-]Flaky native addons tests[/-] [+]Flaky linux tests[/+] on Feb 6, 2018
  9. added
    flaky-testIssues and PRs involving tests that fail intermittently in CI.
    and removed
    addonsIssues and PRs related to native addons.
    on Feb 6, 2018
  10. maclover7 commented on Feb 6, 2018

    @maclover7
    ContributorAuthor

    Slightly tweaked title for now, until can gather more information.

  11. maclover7 commented on Feb 7, 2018

    @maclover7
    ContributorAuthor

    There is also an issue about this in nodejs/build: nodejs/build#1049

    cc @Trott

  12. richardlau commented on Feb 12, 2018

    @richardlau
    Member
  13. maclover7 commented on May 24, 2018

    @maclover7
    ContributorAuthor

    I forget what issue/PR, but I believe this is resolved now. At the very least, hasn't happened recently enough that people have noticed and commented. Please reopen if I'm mistaken.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    docIssues and PRs related to Node.js documentation.flaky-testIssues and PRs involving tests that fail intermittently in CI.testIssues and PRs related to Node.js core tests and test infrastructure.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions