Tests that fail on gaoss but not mvnc, generated from non-source-equivalent GAV pairs

[wtwhite]

I've added rows to the spreadsheet for the results of tests generated by EvoSuite on pairs of GAVs that are not source equivalent (#34, #35). These new rows are coloured peach and have "N" in the newly added leftmost column ("From source-equivalent GAV pair?").

None of the 10 obfs test classes had any tests fail, but gaoss had 11 test classes with one or more failures, 4 of which had exactly the same failures on mvnc itself and are thus effectively false positives. This leaves 7 test classes with failures on gaoss only (marked in green on the spreadsheet):

run/gaoss/io/undertow/undertow-servlet/2.2.23.Final/io.undertow.servlet.spec.UpgradeServletInputStream_ESTest.out
run/gaoss/io/undertow/undertow-servlet/2.2.24.Final/io.undertow.servlet.spec.UpgradeServletInputStream_ESTest.out
run/gaoss/io/undertow/undertow-servlet/2.2.25.Final/io.undertow.servlet.spec.UpgradeServletInputStream_ESTest.out
run/gaoss/io/undertow/undertow-servlet/2.2.26.Final/io.undertow.servlet.spec.UpgradeServletInputStream_ESTest.out
run/gaoss/io/undertow/undertow-servlet/2.2.28.Final/io.undertow.servlet.spec.UpgradeServletInputStream_ESTest.out
run/gaoss/io/undertow/undertow-servlet/2.2.31.Final/io.undertow.servlet.spec.UpgradeServletInputStream_ESTest.out
run/gaoss/io/undertow/undertow-servlet/2.2.32.Final/io.undertow.servlet.spec.UpgradeServletInputStream_ESTest.out

Investigate.

[wtwhite]

run/gaoss/io/undertow/undertow-servlet/2.2.23.Final/io.undertow.servlet.spec.UpgradeServletInputStream_ESTest.out:

JUnit version 4.13
......E..E....E.E.E........
Time: 1.462
There were 5 failures:
1) test16(io.undertow.servlet.spec.UpgradeServletInputStream_ESTest)
java.lang.NoSuchMethodError: java.nio.ByteBuffer.flip()Ljava/nio/ByteBuffer;
        at io.undertow.servlet.spec.UpgradeServletInputStream.readIntoBufferNonBlocking(UpgradeServletInputStream.java:170)
        at io.undertow.servlet.spec.UpgradeServletInputStream.available(UpgradeServletInputStream.java:210)
        at io.undertow.servlet.spec.UpgradeServletInputStream_ESTest.test16(UpgradeServletInputStream_ESTest.java:310)
2) test15(io.undertow.servlet.spec.UpgradeServletInputStream_ESTest)
java.lang.NoSuchMethodError: java.nio.ByteBuffer.flip()Ljava/nio/ByteBuffer;
        at io.undertow.servlet.spec.UpgradeServletInputStream.readIntoBuffer(UpgradeServletInputStream.java:151)
        at io.undertow.servlet.spec.UpgradeServletInputStream.read(UpgradeServletInputStream.java:125)
        at io.undertow.servlet.spec.UpgradeServletInputStream.read(UpgradeServletInputStream.java:112)
        at io.undertow.servlet.spec.UpgradeServletInputStream_ESTest.test15(UpgradeServletInputStream_ESTest.java:297)
3) test01(io.undertow.servlet.spec.UpgradeServletInputStream_ESTest)
java.lang.NoSuchMethodError: java.nio.ByteBuffer.flip()Ljava/nio/ByteBuffer;
        at io.undertow.servlet.spec.UpgradeServletInputStream.readIntoBuffer(UpgradeServletInputStream.java:151)
        at io.undertow.servlet.spec.UpgradeServletInputStream.read(UpgradeServletInputStream.java:125)
        at io.undertow.servlet.spec.UpgradeServletInputStream.read(UpgradeServletInputStream.java:112)
        at io.undertow.servlet.spec.UpgradeServletInputStream_ESTest.test01(UpgradeServletInputStream_ESTest.java:61)
4) test12(io.undertow.servlet.spec.UpgradeServletInputStream_ESTest)
java.lang.NoSuchMethodError: java.nio.ByteBuffer.flip()Ljava/nio/ByteBuffer;
        at io.undertow.servlet.spec.UpgradeServletInputStream.readIntoBuffer(UpgradeServletInputStream.java:151)
        at io.undertow.servlet.spec.UpgradeServletInputStream.read(UpgradeServletInputStream.java:125)
        at io.undertow.servlet.spec.UpgradeServletInputStream_ESTest.test12(UpgradeServletInputStream_ESTest.java:250)
5) test00(io.undertow.servlet.spec.UpgradeServletInputStream_ESTest)
java.lang.NoSuchMethodError: java.nio.ByteBuffer.flip()Ljava/nio/ByteBuffer;
        at io.undertow.servlet.spec.UpgradeServletInputStream.readIntoBuffer(UpgradeServletInputStream.java:151)
        at io.undertow.servlet.spec.UpgradeServletInputStream.read(UpgradeServletInputStream.java:125)
        at io.undertow.servlet.spec.UpgradeServletInputStream.read(UpgradeServletInputStream.java:112)
        at io.undertow.servlet.spec.UpgradeServletInputStream_ESTest.test00(UpgradeServletInputStream_ESTest.java:40)

FAILURES!!!
Tests run: 22,  Failures: 5

I have eyeballed the other 6 JUnit4 test runner outputs, and every test failure across them is the same java.lang.NoSuchMethodError: java.nio.ByteBuffer.flip()Ljava/nio/ByteBuffer;.

[jensdietrich]

@wtwhite I just had a quick look at the javadoc:

java.nio.ByteBuffer.flip()Ljava/nio/ByteBuffer; exists in Java 17 but not in Java 8 , here the method is inherited, and has a different return type (Buffer). For many clients this is source compatible (unless they expect a ByteBuffer to be returned) but not binary compatible. So I guess that the version that breaks was compiled with a new version of Java and when the tests were run an older version that does not have this change was used.

I think this is an interesting case, and different to the one we saw before, although this is something a static analysis could also figure out.

[wtwhite]

Ah thanks @jensdietrich, that sounds like the root cause.

However, the manifest includes Build-Jdk: 11.0.17 and Java-Version: 11.0.17, suggesting it was built with JDK 11:

wtwhite@wtwhite-vuw-vm:~/code/regression-test-generation/find-diff-classes$ unzip -p /home/wtwhite/code/craw-redhat-oss/wget/crawl/google-aoss/io/undertow/undertow-servlet/2.2.23.Final/undertow-servlet-2.2.23.Final.jar META-INF/MANIFEST.MF
Manifest-Version: 1.0
Created-By: Apache Maven Bundle Plugin
Built-By: root
Build-Jdk: 11.0.17
Specification-Title: Undertow Servlet
Specification-Version: 2.2
Specification-Vendor: JBoss by Red Hat
Implementation-Title: Undertow Servlet
Implementation-Version: 2.2.23.Final
Implementation-Vendor-Id: io.undertow
Implementation-Vendor: JBoss by Red Hat
Implementation-URL: http://www.jboss.org/undertow-parent/undertow-servle
 t
Java-Vendor: Eclipse Adoptium
Java-Version: 11.0.17
Os-Arch: amd64
Os-Name: Linux
Os-Version: 5.10.161+
Scm-Connection: scm:git://github.com/undertow-io/undertow.git/undertow-s
 ervlet
Scm-Revision: 569eebca669420e313793fc377ae3c59036526cd
Scm-Url: https://github.com/undertow-io/undertow/undertow-servlet
Bnd-LastModified: 1680200453530
Build-Jdk-Spec: 11
Bundle-Description: Undertow
Bundle-DocURL: http://www.jboss.org
Bundle-License: http://repository.jboss.org/licenses/apache-2.0.txt
Bundle-ManifestVersion: 2
Bundle-Name: Undertow Servlet
Bundle-SymbolicName: io.undertow.servlet
Bundle-Vendor: JBoss by Red Hat
Bundle-Version: 2.2.23.Final
Export-Package: io.undertow.servlet;uses:="io.undertow.servlet.api,javax
 .servlet,org.jboss.logging";version="2.2.23.Final",io.undertow.servlet.
 api;uses:="io.undertow.security.api,io.undertow.security.idm,io.underto
 w.server,io.undertow.server.handlers,io.undertow.server.handlers.resour
 ce,io.undertow.server.session,io.undertow.servlet,io.undertow.servlet.c
 ore,io.undertow.servlet.handlers,io.undertow.servlet.spec,javax.servlet
 ,javax.servlet.descriptor,javax.servlet.http,org.jboss.logging";version
 ="2.2.23.Final",io.undertow.servlet.attribute;uses:="io.undertow.attrib
 ute,io.undertow.server";version="2.2.23.Final",io.undertow.servlet.comp
 at.rewrite;uses:="io.undertow.server,io.undertow.servlet.handlers,javax
 .servlet.http";version="2.2.23.Final",io.undertow.servlet.core;uses:="i
 o.undertow.io,io.undertow.security.api,io.undertow.security.idm,io.unde
 rtow.server,io.undertow.server.handlers.form,io.undertow.server.session
 ,io.undertow.servlet,io.undertow.servlet.api,io.undertow.servlet.handle
 rs,io.undertow.servlet.spec,javax.servlet,javax.servlet.http,org.xnio";
 version="2.2.23.Final",io.undertow.servlet.handlers;uses:="io.undertow.
 server,io.undertow.server.handlers.builder,io.undertow.server.session,i
 o.undertow.servlet.api,io.undertow.servlet.core,io.undertow.servlet.spe
 c,io.undertow.util,javax.servlet,javax.servlet.http";version="2.2.23.Fi
 nal",io.undertow.servlet.handlers.security;uses:="io.undertow.security.
 api,io.undertow.security.handlers,io.undertow.security.idm,io.undertow.
 security.impl,io.undertow.server,io.undertow.server.handlers.form,io.un
 dertow.server.session,io.undertow.servlet.api,io.undertow.servlet.spec"
 ;version="2.2.23.Final",io.undertow.servlet.predicate;uses:="io.underto
 w.attribute,io.undertow.predicate,io.undertow.server,javax.servlet";ver
 sion="2.2.23.Final",io.undertow.servlet.spec;uses:="io.undertow.connect
 or,io.undertow.io,io.undertow.server,io.undertow.server.handlers,io.und
 ertow.server.handlers.form,io.undertow.server.session,io.undertow.servl
 et.api,io.undertow.servlet.core,io.undertow.servlet.handlers,io.underto
 w.util,javax.servlet,javax.servlet.descriptor,javax.servlet.http,org.xn
 io,org.xnio.channels";version="2.2.23.Final",io.undertow.servlet.sse;us
 es:="javax.servlet,javax.servlet.annotation";version="2.2.23.Final",io.
 undertow.servlet.util;uses:="io.undertow.server,io.undertow.servlet.api
 ,io.undertow.servlet.handlers,io.undertow.servlet.spec,io.undertow.util
 ,javax.servlet.http";version="2.2.23.Final",io.undertow.servlet.websock
 ets;uses:="io.undertow.connector,io.undertow.server,io.undertow.util,io
 .undertow.websockets,io.undertow.websockets.core,io.undertow.websockets
 .core.protocol,io.undertow.websockets.spi,javax.servlet,javax.servlet.h
 ttp,org.xnio";version="2.2.23.Final"
Import-Package: io.undertow;version="[2.2,3)",io.undertow.attribute;vers
 ion="[2.2,3)",io.undertow.connector;version="[2.2,3)",io.undertow.io;ve
 rsion="[2.2,3)",io.undertow.predicate;version="[2.2,3)",io.undertow.sec
 urity.api;version="[2.2,3)",io.undertow.security.handlers;version="[2.2
 ,3)",io.undertow.security.idm;version="[2.2,3)",io.undertow.security.im
 pl;version="[2.2,3)",io.undertow.server;version="[2.2,3)",io.undertow.s
 erver.handlers;version="[2.2,3)",io.undertow.server.handlers.builder;ve
 rsion="[2.2,3)",io.undertow.server.handlers.cache;version="[2.2,3)",io.
 undertow.server.handlers.form;version="[2.2,3)",io.undertow.server.hand
 lers.resource;version="[2.2,3)",io.undertow.server.handlers.sse;version
 ="[2.2,3)",io.undertow.server.protocol.http;version="[2.2,3)",io.undert
 ow.server.session;version="[2.2,3)",io.undertow.util;version="[2.2,3)",
 io.undertow.websockets;version="[2.2,3)",io.undertow.websockets.core;ve
 rsion="[2.2,3)",io.undertow.websockets.core.protocol;version="[2.2,3)",
 io.undertow.websockets.core.protocol.version07;version="[2.2,3)",io.und
 ertow.websockets.core.protocol.version08;version="[2.2,3)",io.undertow.
 websockets.core.protocol.version13;version="[2.2,3)",io.undertow.websoc
 kets.spi;version="[2.2,3)",javax.annotation.security;version="[1.2,3)",
 javax.servlet;version="[4.0,5)",javax.servlet.annotation;version="[4.0,
 5)",javax.servlet.descriptor;version="[4.0,5)",javax.servlet.http;versi
 on="[4.0,5)",org.jboss.logging;version="[3.4,4)",org.xnio;version="[3.8
 ,4)",org.xnio.channels;version="[3.8,4)",org.xnio.conduits;version="[3.
 8,4)"
Require-Capability: osgi.ee;filter:="(&(osgi.ee=JavaSE)(version=1.8))"
Tool: Bnd-5.1.1.202006162103

But JDK 11 inherits flip(), just like JDK 8... So why would the compiler generate a call to this nonexistent method?

I double-checked the bytecode for readIntoBuffer() in this class (one of two methods that triggers the NoSuchMethodError) and indeed it makes a call to java/nio/ByteBuffer.flip:()Ljava/nio/ByteBuffer; at offset 57:

wtwhite@wtwhite-vuw-vm:~/code/regression-test-generation/find-diff-classes$ javap -c -p -cp /home/wtwhite/code/craw-redhat-oss/wget/crawl/google-aoss/io/undertow/undertow-servlet/2.2.23.Final/undertow-servlet-2.2.23.Final.jar io.undertow.servlet.spec.UpgradeServletInputStream
--snip--
  private void readIntoBuffer() throws java.io.IOException;
    Code:
       0: aload_0
       1: getfield      #2                  // Field pooled:Lio/undertow/connector/PooledByteBuffer;
       4: ifnonnull     90
       7: aload_0
       8: getfield      #4                  // Field state:I
      11: iconst_4
      12: invokestatic  #10                 // Method org/xnio/Bits.anyAreSet:(II)Z
      15: ifne          90
      18: aload_0
      19: aload_0
      20: getfield      #7                  // Field bufferPool:Lio/undertow/connector/ByteBufferPool;
      23: invokeinterface #34,  1           // InterfaceMethod io/undertow/connector/ByteBufferPool.allocate:()Lio/undertow/connector/PooledByteBuffer;
      28: putfield      #2                  // Field pooled:Lio/undertow/connector/PooledByteBuffer;
      31: aload_0
      32: getfield      #5                  // Field channel:Lorg/xnio/channels/StreamSourceChannel;
      35: aload_0
      36: getfield      #2                  // Field pooled:Lio/undertow/connector/PooledByteBuffer;
      39: invokeinterface #28,  1           // InterfaceMethod io/undertow/connector/PooledByteBuffer.getBuffer:()Ljava/nio/ByteBuffer;
      44: invokestatic  #35                 // Method org/xnio/channels/Channels.readBlocking:(Ljava/nio/channels/ReadableByteChannel;Ljava/nio/ByteBuffer;)I
      47: istore_1
      48: aload_0
      49: getfield      #2                  // Field pooled:Lio/undertow/connector/PooledByteBuffer;
      52: invokeinterface #28,  1           // InterfaceMethod io/undertow/connector/PooledByteBuffer.getBuffer:()Ljava/nio/ByteBuffer;
      57: invokevirtual #36                 // Method java/nio/ByteBuffer.flip:()Ljava/nio/ByteBuffer;
      60: pop
      61: iload_1
      62: iconst_m1
      63: if_icmpne     90
      66: aload_0
      67: dup
      68: getfield      #4                  // Field state:I
      71: iconst_4
      72: ior
      73: putfield      #4                  // Field state:I
      76: aload_0
      77: getfield      #2                  // Field pooled:Lio/undertow/connector/PooledByteBuffer;
      80: invokeinterface #33,  1           // InterfaceMethod io/undertow/connector/PooledByteBuffer.close:()V
      85: aload_0
      86: aconst_null
      87: putfield      #2                  // Field pooled:Lio/undertow/connector/PooledByteBuffer;
      90: return
--snip--

I'm not sure how to explain this. Either the Build-Jdk: 11.0.17 header in the manifest is lying, or something is misconfigured?

The class was compiled to target version 52 (JDK 8), though of course this by itself doesn't tell us much about the build environment:

wtwhite@wtwhite-vuw-vm:~/code/regression-test-generation/find-diff-classes$ javap -v -cp /home/wtwhite/code/craw-redhat-oss/wget/crawl/google-aoss/io/undertow/undertow-servlet/2.2.23.Final/undertow-servlet-2.2.23.Final.jar io.undertow.servlet.spec.UpgradeServletInputStream|grep version
  minor version: 0
  major version: 52

[jensdietrich]

Can we please rerun the generated tests with Java 11 as this was the Java version used in the build (by GAOSS).

[jensdietrich]

Possible Scenario that would explain difference

// client code
ByteBuffer bb = new ByteBuffer();
Buffer fb = bb.flip();

// v0 -- ByteBuffer::flip missing but inherits from Buffer
// v1 -- ByteBuffer::flip implemented and returns Buffer
// v2 -- ByteBuffer::flip  changed to return ByteBuffer

SCENARIO:

compile client with v2 - callsite for ByteBuffer.flip()Ljava/nio/ByteBuffer
run client with v1 or v0 - NoSuchMethodError as descriptor does not match

[jensdietrich]

@behnazh-w to be discussed further: it looks like the gaoss version declares that it was built with Java 11 but ACC to the references found in bytecode used some later version. We could design some checks to test claims of Java build versions being used. Perhaps animalsniffer already does this ??

[wtwhite]

As discussed with @jensdietrich just now, the mvnc version of readIntoBuffer() calls java/nio/ByteBuffer.flip:()Ljava/nio/Buffer; instead, i.e., it asks for a method that returns Buffer instead of ByteBuffer:

wtwhite@wtwhite-vuw-vm:~/code/regression-test-generation/find-diff-classes$ javap -c -p -cp /home/wtwhite/code/craw-redhat-oss/wget/crawl/repo1.maven.org/maven2/io/undertow/undertow-servlet/2.2.23.Final/undertow-servlet-2.2.23.Final.jar io.undertow.servlet.spec.UpgradeServletInputStream
--snip--
  private void readIntoBuffer() throws java.io.IOException;
    Code:
       0: aload_0
       1: getfield      #2                  // Field pooled:Lio/undertow/connector/PooledByteBuffer;
       4: ifnonnull     90
       7: aload_0
       8: getfield      #4                  // Field state:I
      11: iconst_4
      12: invokestatic  #10                 // Method org/xnio/Bits.anyAreSet:(II)Z
      15: ifne          90
      18: aload_0
      19: aload_0
      20: getfield      #7                  // Field bufferPool:Lio/undertow/connector/ByteBufferPool;
      23: invokeinterface #34,  1           // InterfaceMethod io/undertow/connector/ByteBufferPool.allocate:()Lio/undertow/connector/PooledByteBuffer;
      28: putfield      #2                  // Field pooled:Lio/undertow/connector/PooledByteBuffer;
      31: aload_0
      32: getfield      #5                  // Field channel:Lorg/xnio/channels/StreamSourceChannel;
      35: aload_0
      36: getfield      #2                  // Field pooled:Lio/undertow/connector/PooledByteBuffer;
      39: invokeinterface #28,  1           // InterfaceMethod io/undertow/connector/PooledByteBuffer.getBuffer:()Ljava/nio/ByteBuffer;
      44: invokestatic  #35                 // Method org/xnio/channels/Channels.readBlocking:(Ljava/nio/channels/ReadableByteChannel;Ljava/nio/ByteBuffer;)I
      47: istore_1
      48: aload_0
      49: getfield      #2                  // Field pooled:Lio/undertow/connector/PooledByteBuffer;
      52: invokeinterface #28,  1           // InterfaceMethod io/undertow/connector/PooledByteBuffer.getBuffer:()Ljava/nio/ByteBuffer;
      57: invokevirtual #36                 // Method java/nio/ByteBuffer.flip:()Ljava/nio/Buffer;
      60: pop
      61: iload_1
      62: iconst_m1
      63: if_icmpne     90
      66: aload_0
      67: dup
      68: getfield      #4                  // Field state:I
      71: iconst_4
      72: ior
      73: putfield      #4                  // Field state:I
      76: aload_0
      77: getfield      #2                  // Field pooled:Lio/undertow/connector/PooledByteBuffer;
      80: invokeinterface #33,  1           // InterfaceMethod io/undertow/connector/PooledByteBuffer.close:()V
      85: aload_0
      86: aconst_null
      87: putfield      #2                  // Field pooled:Lio/undertow/connector/PooledByteBuffer;
      90: return
--snip--

The invokevirtual instruction at offset 57 will be satisfied by the the implementation of flip() in the parent class Buffer, explaining why there is no error when the tests are run against the mvnc version under JDK 8. (I had been thinking the error with gaoss came from the fact that the instruction specifies java/nio/ByteBuffer as the class for the flip() method, rather than the parent class where that method is actually defined -- but according to Jens the JVM is fine with this. The error comes from the mismatched return type -- asking for a more specific return type than the implementation can provide.)

The mvnc manifest includes Build-Jdk-Spec: 1.8, Java-Version: 1.8.0_302 and Build-Jdk: 1.8.0_302, indicating it was built under JDK 8:

wtwhite@wtwhite-vuw-vm:~/code/regression-test-generation/find-diff-classes$ unzip -p /home/wtwhite/code/craw-redhat-oss/wget/crawl/repo1.maven.org/maven2/io/undertow/undertow-servlet/2.2.23.Final/undertow-servlet-2.2.23.Final.jar META-INF/MANIFEST.MF
Manifest-Version: 1.0
Bundle-License: http://repository.jboss.org/licenses/apache-2.0.txt
Bundle-SymbolicName: io.undertow.servlet
Built-By: fla
Bnd-LastModified: 1676353747588
Specification-Title: Undertow Servlet
Implementation-Vendor-Id: io.undertow
Bundle-DocURL: http://www.jboss.org
Import-Package: io.undertow;version="[2.2,3)",io.undertow.attribute;ve
 rsion="[2.2,3)",io.undertow.connector;version="[2.2,3)",io.undertow.i
 o;version="[2.2,3)",io.undertow.predicate;version="[2.2,3)",io.undert
 ow.security.api;version="[2.2,3)",io.undertow.security.handlers;versi
 on="[2.2,3)",io.undertow.security.idm;version="[2.2,3)",io.undertow.s
 ecurity.impl;version="[2.2,3)",io.undertow.server;version="[2.2,3)",i
 o.undertow.server.handlers;version="[2.2,3)",io.undertow.server.handl
 ers.builder;version="[2.2,3)",io.undertow.server.handlers.cache;versi
 on="[2.2,3)",io.undertow.server.handlers.form;version="[2.2,3)",io.un
 dertow.server.handlers.resource;version="[2.2,3)",io.undertow.server.
 handlers.sse;version="[2.2,3)",io.undertow.server.protocol.http;versi
 on="[2.2,3)",io.undertow.server.session;version="[2.2,3)",io.undertow
 .util;version="[2.2,3)",io.undertow.websockets;version="[2.2,3)",io.u
 ndertow.websockets.core;version="[2.2,3)",io.undertow.websockets.core
 .protocol;version="[2.2,3)",io.undertow.websockets.core.protocol.vers
 ion07;version="[2.2,3)",io.undertow.websockets.core.protocol.version0
 8;version="[2.2,3)",io.undertow.websockets.core.protocol.version13;ve
 rsion="[2.2,3)",io.undertow.websockets.spi;version="[2.2,3)",javax.an
 notation.security;version="[1.2,3)",javax.servlet;version="[4.0,5)",j
 avax.servlet.annotation;version="[4.0,5)",javax.servlet.descriptor;ve
 rsion="[4.0,5)",javax.servlet.http;version="[4.0,5)",org.jboss.loggin
 g;version="[3.4,4)",org.xnio;version="[3.8,4)",org.xnio.channels;vers
 ion="[3.8,4)",org.xnio.conduits;version="[3.8,4)"
Require-Capability: osgi.ee;filter:="(&(osgi.ee=JavaSE)(version=1.8))"
Scm-Url: https://github.com/undertow-io/undertow/undertow-servlet
Export-Package: io.undertow.servlet;uses:="io.undertow.servlet.api,jav
 ax.servlet,org.jboss.logging";version="2.2.23.Final",io.undertow.serv
 let.api;uses:="io.undertow.security.api,io.undertow.security.idm,io.u
 ndertow.server,io.undertow.server.handlers,io.undertow.server.handler
 s.resource,io.undertow.server.session,io.undertow.servlet,io.undertow
 .servlet.core,io.undertow.servlet.handlers,io.undertow.servlet.spec,j
 avax.servlet,javax.servlet.descriptor,javax.servlet.http,org.jboss.lo
 gging";version="2.2.23.Final",io.undertow.servlet.attribute;uses:="io
 .undertow.attribute,io.undertow.server";version="2.2.23.Final",io.und
 ertow.servlet.compat.rewrite;uses:="io.undertow.server,io.undertow.se
 rvlet.handlers,javax.servlet.http";version="2.2.23.Final",io.undertow
 .servlet.core;uses:="io.undertow.io,io.undertow.security.api,io.under
 tow.security.idm,io.undertow.server,io.undertow.server.handlers.form,
 io.undertow.server.session,io.undertow.servlet,io.undertow.servlet.ap
 i,io.undertow.servlet.handlers,io.undertow.servlet.spec,javax.servlet
 ,javax.servlet.http,org.xnio";version="2.2.23.Final",io.undertow.serv
 let.handlers;uses:="io.undertow.server,io.undertow.server.handlers.bu
 ilder,io.undertow.server.session,io.undertow.servlet.api,io.undertow.
 servlet.core,io.undertow.servlet.spec,io.undertow.util,javax.servlet,
 javax.servlet.http";version="2.2.23.Final",io.undertow.servlet.handle
 rs.security;uses:="io.undertow.security.api,io.undertow.security.hand
 lers,io.undertow.security.idm,io.undertow.security.impl,io.undertow.s
 erver,io.undertow.server.handlers.form,io.undertow.server.session,io.
 undertow.servlet.api,io.undertow.servlet.spec";version="2.2.23.Final"
 ,io.undertow.servlet.predicate;uses:="io.undertow.attribute,io.undert
 ow.predicate,io.undertow.server,javax.servlet";version="2.2.23.Final"
 ,io.undertow.servlet.spec;uses:="io.undertow.connector,io.undertow.io
 ,io.undertow.server,io.undertow.server.handlers,io.undertow.server.ha
 ndlers.form,io.undertow.server.session,io.undertow.servlet.api,io.und
 ertow.servlet.core,io.undertow.servlet.handlers,io.undertow.util,java
 x.servlet,javax.servlet.descriptor,javax.servlet.http,org.xnio,org.xn
 io.channels";version="2.2.23.Final",io.undertow.servlet.sse;uses:="ja
 vax.servlet,javax.servlet.annotation";version="2.2.23.Final",io.under
 tow.servlet.util;uses:="io.undertow.server,io.undertow.servlet.api,io
 .undertow.servlet.handlers,io.undertow.servlet.spec,io.undertow.util,
 javax.servlet.http";version="2.2.23.Final",io.undertow.servlet.websoc
 kets;uses:="io.undertow.connector,io.undertow.server,io.undertow.util
 ,io.undertow.websockets,io.undertow.websockets.core,io.undertow.webso
 ckets.core.protocol,io.undertow.websockets.spi,javax.servlet,javax.se
 rvlet.http,org.xnio";version="2.2.23.Final"
Bundle-Name: Undertow Servlet
Build-Jdk-Spec: 1.8
Implementation-Title: Undertow Servlet
Bundle-Description: Undertow
Implementation-Version: 2.2.23.Final
Java-Version: 1.8.0_302
Scm-Connection: scm:git://github.com/undertow-io/undertow.git/undertow
 -servlet
Specification-Vendor: JBoss by Red Hat
Os-Arch: amd64
Bundle-ManifestVersion: 2
Java-Vendor: Oracle Corporation
Bundle-Vendor: JBoss by Red Hat
Os-Name: Linux
Tool: Bnd-5.1.1.202006162103
Implementation-Vendor: JBoss by Red Hat
Os-Version: 6.0.15-200.fc36.x86_64
Originally-Created-By: Apache Maven Bundle Plugin
Scm-Revision: 569eebca669420e313793fc377ae3c59036526cd
Bundle-Version: 2.2.23.Final
Created-By: Apache Maven Bundle Plugin
Build-Jdk: 1.8.0_302
Specification-Version: 2.2
Implementation-URL: http://www.jboss.org/undertow-parent/undertow-serv
 let

[wtwhite]

Confusingly, running the tests under JDK 11 causes them all still to pass:

wtwhite@wtwhite-vuw-vm:~/code/regression-test-generation/find-diff-classes$ cat run_under_jdk11/gaoss/io/undertow/undertow-servlet/2.2.23.Final/io.undertow.servlet.spec.UpgradeServletInputStream_ESTest.out
JUnit version 4.13
......................
Time: 1.837

OK (22 tests)

wtwhite@wtwhite-vuw-vm:~/code/regression-test-generation/find-diff-classes$ grep 'OK' `find run_under_jdk11 -name '*_ESTest.out'`
run_under_jdk11/gaoss/io/undertow/undertow-servlet/2.2.26.Final/io.undertow.servlet.spec.UpgradeServletInputStream_ESTest.out:OK (24 tests)
run_under_jdk11/gaoss/io/undertow/undertow-servlet/2.2.28.Final/io.undertow.servlet.spec.UpgradeServletInputStream_ESTest.out:OK (22 tests)
run_under_jdk11/gaoss/io/undertow/undertow-servlet/2.2.31.Final/io.undertow.servlet.spec.UpgradeServletInputStream_ESTest.out:OK (21 tests)
run_under_jdk11/gaoss/io/undertow/undertow-servlet/2.2.25.Final/io.undertow.servlet.spec.UpgradeServletInputStream_ESTest.out:OK (23 tests)
run_under_jdk11/gaoss/io/undertow/undertow-servlet/2.2.32.Final/io.undertow.servlet.spec.UpgradeServletInputStream_ESTest.out:OK (23 tests)
run_under_jdk11/gaoss/io/undertow/undertow-servlet/2.2.24.Final/io.undertow.servlet.spec.UpgradeServletInputStream_ESTest.out:OK (33 tests)
run_under_jdk11/gaoss/io/undertow/undertow-servlet/2.2.23.Final/io.undertow.servlet.spec.UpgradeServletInputStream_ESTest.out:OK (22 tests)

The expectation was that, since JDK 11 does not provide a ByteBuffer.flip() that returns a ByteBuffer either (i.e., it's the same as JDK 8 in this respect), the same NoSuchMethodError should result.

Next step: Create a minimal reproducible example for this. @jensdietrich: Any other ideas? One possibility might be that the the code takes a different path on JDK 11 that avoids calling readIntoBuffer() or readIntoBufferNonBlocking() at all, so the error isn't triggered.

[jensdietrich]

@wtwhite this is strange -- I thought we have this setup for debugging, so you run the test in debug mode and place breakpoints In ByteBuffer:flip and UpgradeServletInputStream::readIntoBufferNonBlocking to see what is going on ?

I think we had confirmed that the callsite in UpgradeServletInputStream::readIntoBufferNonBlocking is for ByteBuffer::flip()LByteBuffer; for the gaoss version, right ?

[wtwhite]

@wtwhite this is strange -- I thought we have this setup for debugging, so you run the test in debug mode and place breakpoints In ByteBuffer:flip and UpgradeServletInputStream::readIntoBufferNonBlocking to see what is going on ?

@jensdietrich It's set up for debugging insofar as I compile and run the tests twice: Once, the "old way" (raw JUnit4 runners as per EvoSuite docs; no debugging but useful for parsing results) and again using "the Maven way" (which enables debugging). So far on this issue I've been working with the "old way" results and worried that swapping over might introduce unknown differences, so I've instead made a small standalone example that shows that the NoSuchMethodError occurs when running under JDK 8 but not under JDK 11, regardless of which compiler: #38. I've eliminated EvoSuite (and JUnit) from the classpath for this example, so we know there's no "funny business".

I think we had confirmed that the callsite in UpgradeServletInputStream::readIntoBufferNonBlocking is for ByteBuffer::flip()LByteBuffer; for the gaoss version, right ?

Yes that's right. So I don't know why JDK 11 has no problems with running this code... Time to debug.

[wtwhite]

Debugging test00() in Intellij under JDK 8 causes NoSuchMethodError as before:

/usr/lib/jvm/java-8-openjdk-amd64/bin/java -agentlib:jdwp=transport=dt_socket,address=127.0.0.1:56225,suspend=y,server=n -ea -Didea.test.cyclic.buffer.size=1048576 -javaagent:/home/wtwhite/intellij/idea-IC-232.9921.47/plugins/java/lib/rt/debugger-agent.jar -Dfile.encoding=UTF-8 -classpath /home/wtwhite/intellij/idea-IC-232.9921.47/lib/idea_rt.jar:/home/wtwhite/intellij/idea-IC-232.9921.47/plugins/junit/lib/junit5-rt.jar:/home/wtwhite/intellij/idea-IC-232.9921.47/plugins/junit/lib/junit-rt.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/charsets.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/cldrdata.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/dnsns.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/icedtea-sound.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/jaccess.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/java-atk-wrapper.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/localedata.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/nashorn.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/sunec.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/sunjce_provider.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/sunpkcs11.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/zipfs.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/jce.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/jfr.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/jsse.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/management-agent.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/resources.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar:/home/wtwhite/code/evosuite_generated_tests/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/target/test-classes:/home/wtwhite/code/craw-redhat-oss/wget/crawl/google-aoss/io/undertow/undertow-servlet/2.2.23.Final/undertow-servlet-2.2.23.Final.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/commons-codec-1.11.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/commons-logging-1.2.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/hamcrest-core-1.3.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/httpclient-4.5.12.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/httpcore-4.4.13.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/httpmime-4.5.12.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/jboss-annotations-api_1.3_spec-2.0.1.Final.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/jboss-logging-3.4.1.Final.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/jboss-logging-annotations-2.2.1.Final.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/jboss-logging-processor-2.2.1.Final.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/jboss-logmanager-2.1.14.Final.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/jboss-servlet-api_4.0_spec-2.0.0.Final.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/jboss-threads-3.1.0.Final.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/jdeparser-2.0.3.Final.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/junit-4.13.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/netty-all-4.1.50.Final.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/undertow-core-2.2.23.Final-tests.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/undertow-core-2.2.23.Final.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/wildfly-client-config-1.0.1.Final.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/wildfly-common-1.5.4.Final.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/wildfly-openssl-1.0.4.Final.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/xnio-api-3.8.7.Final.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/xnio-nio-3.8.7.Final.jar:/home/wtwhite/code/regression-test-generation/evosuite/evosuite-standalone-runtime-1.2.0.jar:/home/wtwhite/.m2/repository/junit/junit/4.13.2/junit-4.13.2.jar:/home/wtwhite/.m2/repository/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar com.intellij.rt.junit.JUnitStarter -ideVersion5 -junit4 io.undertow.servlet.spec.UpgradeServletInputStream_ESTest,test00
Connected to the target VM, address: '127.0.0.1:56225', transport: 'socket'
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Feb 14, 2014 8:21:21 PM <unknown> <unknown>
INFO: JBoss Threads version 3.1.0.Final
Feb 14, 2014 8:21:21 PM <unknown> <unknown>
INFO: JBoss Threads version 3.1.0.Final

java.lang.NoSuchMethodError: java.nio.ByteBuffer.flip()Ljava/nio/ByteBuffer;

	at io.undertow.servlet.spec.UpgradeServletInputStream.readIntoBuffer(UpgradeServletInputStream.java:151)
	at io.undertow.servlet.spec.UpgradeServletInputStream.read(UpgradeServletInputStream.java:125)
	at io.undertow.servlet.spec.UpgradeServletInputStream.read(UpgradeServletInputStream.java:112)
	at io.undertow.servlet.spec.UpgradeServletInputStream_ESTest.test00(UpgradeServletInputStream_ESTest.java:40)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:288)
	at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:282)
	at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
	at java.util.concurrent.FutureTask.run(FutureTask.java)
	at java.lang.Thread.run(Thread.java:750)

Disconnected from the target VM, address: '127.0.0.1:56225', transport: 'socket'

Process finished with exit code 255

After changing to JDK 11, the error goes away:

/usr/lib/jvm/java-1.11.0-openjdk-amd64/bin/java -agentlib:jdwp=transport=dt_socket,address=127.0.0.1:55259,suspend=y,server=n -ea -Didea.test.cyclic.buffer.size=1048576 -javaagent:/home/wtwhite/intellij/idea-IC-232.9921.47/plugins/java/lib/rt/debugger-agent.jar -Dfile.encoding=UTF-8 -classpath /home/wtwhite/intellij/idea-IC-232.9921.47/lib/idea_rt.jar:/home/wtwhite/intellij/idea-IC-232.9921.47/plugins/junit/lib/junit5-rt.jar:/home/wtwhite/intellij/idea-IC-232.9921.47/plugins/junit/lib/junit-rt.jar:/home/wtwhite/code/evosuite_generated_tests/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/target/test-classes:/home/wtwhite/code/craw-redhat-oss/wget/crawl/google-aoss/io/undertow/undertow-servlet/2.2.23.Final/undertow-servlet-2.2.23.Final.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/commons-codec-1.11.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/commons-logging-1.2.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/hamcrest-core-1.3.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/httpclient-4.5.12.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/httpcore-4.4.13.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/httpmime-4.5.12.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/jboss-annotations-api_1.3_spec-2.0.1.Final.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/jboss-logging-3.4.1.Final.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/jboss-logging-annotations-2.2.1.Final.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/jboss-logging-processor-2.2.1.Final.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/jboss-logmanager-2.1.14.Final.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/jboss-servlet-api_4.0_spec-2.0.0.Final.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/jboss-threads-3.1.0.Final.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/jdeparser-2.0.3.Final.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/junit-4.13.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/netty-all-4.1.50.Final.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/undertow-core-2.2.23.Final-tests.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/undertow-core-2.2.23.Final.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/wildfly-client-config-1.0.1.Final.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/wildfly-common-1.5.4.Final.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/wildfly-openssl-1.0.4.Final.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/xnio-api-3.8.7.Final.jar:/home/wtwhite/code/regression-test-generation-NON-SOURCE-EQUIV/find-diff-classes/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final/t/dependency/xnio-nio-3.8.7.Final.jar:/home/wtwhite/code/regression-test-generation/evosuite/evosuite-standalone-runtime-1.2.0.jar:/home/wtwhite/.m2/repository/junit/junit/4.13.2/junit-4.13.2.jar:/home/wtwhite/.m2/repository/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar com.intellij.rt.junit.JUnitStarter -ideVersion5 -junit4 io.undertow.servlet.spec.UpgradeServletInputStream_ESTest,test00
Connected to the target VM, address: '127.0.0.1:55259', transport: 'socket'
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.evosuite.runtime.GuiSupport (file:/home/wtwhite/code/regression-test-generation/evosuite/evosuite-standalone-runtime-1.2.0.jar) to field java.awt.GraphicsEnvironment.headless
WARNING: Please consider reporting this to the maintainers of org.evosuite.runtime.GuiSupport
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
09:21:21,320 INFO  (main) [org.jboss.threads] <null:-1> JBoss Threads version 3.1.0.Final
09:21:21,320 INFO  (main) [org.jboss.threads] <null:-1> JBoss Threads version 3.1.0.Final
Disconnected from the target VM, address: '127.0.0.1:55259', transport: 'socket'

Process finished with exit code 0

I then single-stepped through the code with F7 under JDK 11 until this line in the (fernflower-decompiled) class:

    private void readIntoBuffer() throws IOException {
        if (this.pooled == null && !Bits.anyAreSet(this.state, 4)) {
            this.pooled = this.bufferPool.allocate();
            int res = Channels.readBlocking(this.channel, this.pooled.getBuffer());
            this.pooled.getBuffer().flip();                 // <-------------------------------------------------
            if (res == -1) {
                this.state |= 4;
                this.pooled.close();
                this.pooled = null;
            }
        }

    }

Single-stepping further goes into the getBuffer() call and returns from it, leaving the cursor on the same line. I expected hitting F7 again to take me into a (decompiled version of) flip(), but it does not -- instead, I go straight to the next line. However, navigating to flip() with Ctrl-B to set a breakpoint shows me the following (decompiled) method in /usr/lib/jvm/java-1.11.0-openjdk-amd64!/java.base/java/nio/ByteBuffer.class, which returns ByteBuffer:

   public ByteBuffer flip() {
        super.flip();
        return this;
    }

The pathname Intellij reports doesn't make sense though -- /usr/lib/jvm/java-1.11.0-openjdk-amd64 is not a jar file, and although there is a directory with that name, it has no java.base subdirectory. I looked around inside anyway and found a jmods/ subfolder with various *.jmod files, including java.base.jmod, which is not quite a valid zip file but close enough that unzip opens it despite complaining, and inside is what I was looking for, namely an overriding flip() method that calls super() and returns this as a ByteBuffer:

wtwhite@wtwhite-vuw-vm:~/code/evosuite_generated_tests/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final$ unzip /usr/lib/jvm/java-1.11.0-openjdk-amd64/jmods/java.base.jmod classes/java/nio/ByteBuffer.class
Archive:  /usr/lib/jvm/java-1.11.0-openjdk-amd64/jmods/java.base.jmod
warning [/usr/lib/jvm/java-1.11.0-openjdk-amd64/jmods/java.base.jmod]:  4 extra bytes at beginning or within zipfile
  (attempting to process anyway)
  inflating: classes/java/nio/ByteBuffer.class  
wtwhite@wtwhite-vuw-vm:~/code/evosuite_generated_tests/compile/gaoss/io/undertow/undertow-servlet/2.2.23.Final$ javap -c -p -v classes/java/nio/ByteBuffer.class
--snip--
  public java.nio.ByteBuffer flip();
    descriptor: ()Ljava/nio/ByteBuffer;
    flags: ACC_PUBLIC
    Code:
      stack=1, locals=1, args_size=1
         0: aload_0
         1: invokespecial #41                 // Method java/nio/Buffer.flip:()Ljava/nio/Buffer;
         4: pop
         5: aload_0
         6: areturn
      LineNumberTable:
        line 1159: 0
        line 1160: 5
      LocalVariableTable:
        Start  Length  Slot  Name   Signature
            0       7     0  this   Ljava/nio/ByteBuffer;
--snip--

Conclusion: JDK 11 does provide an overriding flip() method that returns a ByteBuffer (and which is not marked synthetic)(, even though (as established earlier) the JDK 11 docs indicate that no such method exists.

@jensdietrich: What now do you think?

[jensdietrich]

@wtwhite I try to summarise this, could you pls have a look at this table and check / complete this ?

component: io/undertow/undertow-servlet/2.2.23

Build	Claimed Build Version from manifest	Test Case Behaviour - Java 8	Test Case Behaviour - Java 11
mvnc	1.8	pass	pass
gaoss	11.0.17	error with NoSuchMethodError	pass

@jensdietrich: Filled in the mvnc build number. LGTM!

[jensdietrich]

Ok then this makes sense -- when the gaoss version was build with Java 11, it compiled the callsite to invoke ByteBuffer.flip()ByteBuffer, when the mvnc version was build it compiled the callsite to invoke ByteBuffer.flip()Buffer as the method was missing. Now downgrading the Java version at runtime blows up as expected. This is not a very sensational find after all unfortunately, but still worth reporting. In a way, we probably find that our idea is not well supported by the current version of evosuite as it requires test to run with Java 8.