Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dependency on Aleph and Netty causes issues deploying to wildfly and jboss #182

Open
ckirkendall opened this issue Feb 7, 2017 · 10 comments

Comments

@ckirkendall
Copy link

The dependency on Netty and Aleph to manage the connection classpath collisions with Wildfly on some deployments. Wildfly uses an older version of netty that is not compatible with Aleph and will throw method not found exceptions.

2017-02-06 22:06:39,104 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-25) MSC000001: Failed to start service jboss.undertow.deployment.default-server.default-host./: org.jboss.msc.service.StartException in service jboss.undertow.deployment.default-server.default-host./: Failed to start service
    at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1904) [jboss-msc-1.2.2.Final.jar:1.2.2.Final]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_102]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_102]
    at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_102]
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: Application initialization failed
    at io.undertow.servlet.core.DeploymentManagerImpl.deploy(DeploymentManagerImpl.java:222)
    at org.wildfly.extension.undertow.deployment.UndertowDeploymentService.startContext(UndertowDeploymentService.java:87)
    at org.wildfly.extension.undertow.deployment.UndertowDeploymentService.start(UndertowDeploymentService.java:72)
    at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1948) [jboss-msc-1.2.2.Final.jar:1.2.2.Final]
    at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1881) [jboss-msc-1.2.2.Final.jar:1.2.2.Final]
    ... 3 more
Caused by: java.lang.RuntimeException: Application initialization failed
    at org.projectodd.wunderboss.as.ServletListener.contextInitialized(ServletListener.java:155)
    at io.undertow.servlet.core.ApplicationListeners.contextInitialized(ApplicationListeners.java:173)
    at io.undertow.servlet.core.DeploymentManagerImpl.deploy(DeploymentManagerImpl.java:193)
    ... 7 more
Caused by: java.lang.RuntimeException: java.lang.NoSuchMethodError: io.netty.channel.ChannelMetadata.<init>(ZI)V
    at org.projectodd.wunderboss.clojure.ClojureLanguage.eval(ClojureLanguage.java:73)
    at org.projectodd.wunderboss.ApplicationRunner.start(ApplicationRunner.java:52)
    at org.projectodd.wunderboss.as.ServletListener$1.start(ServletListener.java:104)
    at org.projectodd.wunderboss.as.ServletListener$2.run(ServletListener.java:116)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [rt.jar:1.8.0_102]
    at java.util.concurrent.FutureTask.run(FutureTask.java:266) [rt.jar:1.8.0_102]
    ... 3 more
Caused by: java.lang.NoSuchMethodError: io.netty.channel.ChannelMetadata.<init>(ZI)V
    at io.netty.channel.epoll.AbstractEpollServerChannel.<clinit>(AbstractEpollServerChannel.java:34)
    at java.lang.Class.forName0(Native Method) [rt.jar:1.8.0_102]
    at java.lang.Class.forName(Class.java:348) [rt.jar:1.8.0_102]
    at clojure.lang.RT.classForName(RT.java:2168)
    at clojure.lang.RT.classForName(RT.java:2177)
    at aleph.netty$start_server.<clinit>(netty.clj:668)
    at aleph.netty__init.load(Unknown Source)
    at aleph.netty__init.<clinit>(Unknown Source)
    at java.lang.Class.forName0(Native Method) [rt.jar:1.8.0_102]
    at java.lang.Class.forName(Class.java:348) [rt.jar:1.8.0_102]
    at clojure.lang.RT.classForName(RT.java:2168)
    at clojure.lang.RT.classForName(RT.java:2177)
    at clojure.lang.RT.loadClassForName(RT.java:2196)
    at clojure.lang.RT.load(RT.java:443)
    at clojure.lang.RT.load(RT.java:419)
    at clojure.core$load$fn__5677.invoke(core.clj:5893)
    at clojure.core$load.invokeStatic(core.clj:5892)
    at clojure.core$load.doInvoke(core.clj:5876)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.core$load_one.invokeStatic(core.clj:5697)
    at clojure.core$load_one.invoke(core.clj:5692)
    at clojure.core$load_lib$fn__5626.invoke(core.clj:5737)
    at clojure.core$load_lib.invokeStatic(core.clj:5736)
    at clojure.core$load_lib.doInvoke(core.clj:5717)
    at clojure.lang.RestFn.applyTo(RestFn.java:142)
    at clojure.core$apply.invokeStatic(core.clj:648)
    at clojure.core$load_libs.invokeStatic(core.clj:5774)
    at clojure.core$load_libs.doInvoke(core.clj:5758)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at clojure.core$apply.invokeStatic(core.clj:648)
    at clojure.core$require.invokeStatic(core.clj:5796)
    at clojure.core$require.doInvoke(core.clj:5796)
    at clojure.lang.RestFn.invoke(RestFn.java:482)
    at aleph.tcp$loading__5569__auto____26946.invoke(tcp.clj:1)
    at aleph.tcp__init.load(Unknown Source)
    at aleph.tcp__init.<clinit>(Unknown Source)
    at java.lang.Class.forName0(Native Method) [rt.jar:1.8.0_102]
    at java.lang.Class.forName(Class.java:348) [rt.jar:1.8.0_102]
    at clojure.lang.RT.classForName(RT.java:2168)
    at clojure.lang.RT.classForName(RT.java:2177)
    at clojure.lang.RT.loadClassForName(RT.java:2196)
    at clojure.lang.RT.load(RT.java:443)
    at clojure.lang.RT.load(RT.java:419)
    at clojure.core$load$fn__5677.invoke(core.clj:5893)
    at clojure.core$load.invokeStatic(core.clj:5892)
    at clojure.core$load.doInvoke(core.clj:5876)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.core$load_one.invokeStatic(core.clj:5697)
    at clojure.core$load_one.invoke(core.clj:5692)
    at clojure.core$load_lib$fn__5626.invoke(core.clj:5737)
    at clojure.core$load_lib.invokeStatic(core.clj:5736)
    at clojure.core$load_lib.doInvoke(core.clj:5717)
    at clojure.lang.RestFn.applyTo(RestFn.java:142)
    at clojure.core$apply.invokeStatic(core.clj:648)
    at clojure.core$load_libs.invokeStatic(core.clj:5774)
    at clojure.core$load_libs.doInvoke(core.clj:5758)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at clojure.core$apply.invokeStatic(core.clj:648)
    at clojure.core$require.invokeStatic(core.clj:5796)
    at clojure.core$require.doInvoke(core.clj:5796)
    at clojure.lang.RestFn.invoke(RestFn.java:512)
    at rethinkdb.core$loading__5569__auto____26944.invoke(core.clj:1)
    at rethinkdb.core__init.load(Unknown Source)
    at rethinkdb.core__init.<clinit>(Unknown Source)
    at java.lang.Class.forName0(Native Method) [rt.jar:1.8.0_102]
    at java.lang.Class.forName(Class.java:348) [rt.jar:1.8.0_102]
    at clojure.lang.RT.classForName(RT.java:2168)
    at clojure.lang.RT.classForName(RT.java:2177)
    at clojure.lang.RT.loadClassForName(RT.java:2196)
    at clojure.lang.RT.load(RT.java:443)
    at clojure.lang.RT.load(RT.java:419)
    at clojure.core$load$fn__5677.invoke(core.clj:5893)
    at clojure.core$load.invokeStatic(core.clj:5892)
    at clojure.core$load.doInvoke(core.clj:5876)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.core$load_one.invokeStatic(core.clj:5697)
    at clojure.core$load_one.invoke(core.clj:5692)
    at clojure.core$load_lib$fn__5626.invoke(core.clj:5737)
    at clojure.core$load_lib.invokeStatic(core.clj:5736)
    at clojure.core$load_lib.doInvoke(core.clj:5717)
    at clojure.lang.RestFn.applyTo(RestFn.java:142)
    at clojure.core$apply.invokeStatic(core.clj:648)
    at clojure.core$load_libs.invokeStatic(core.clj:5774)
    at clojure.core$load_libs.doInvoke(core.clj:5758)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at clojure.core$apply.invokeStatic(core.clj:648)
    at clojure.core$require.invokeStatic(core.clj:5796)
    at clojure.core$require.doInvoke(core.clj:5796)
    at clojure.lang.RestFn.invoke(RestFn.java:457)
    at rethinkdb.query$loading__5569__auto____21532.invoke(query.cljc:1)
    at rethinkdb.query__init.load(Unknown Source)
    at rethinkdb.query__init.<clinit>(Unknown Source)
    at java.lang.Class.forName0(Native Method) [rt.jar:1.8.0_102]
    at java.lang.Class.forName(Class.java:348) [rt.jar:1.8.0_102]
    at clojure.lang.RT.classForName(RT.java:2168)
    at clojure.lang.RT.classForName(RT.java:2177)
    at clojure.lang.RT.loadClassForName(RT.java:2196)
    at clojure.lang.RT.load(RT.java:443)
    at clojure.lang.RT.load(RT.java:419)
    at clojure.core$load$fn__5677.invoke(core.clj:5893)
    at clojure.core$load.invokeStatic(core.clj:5892)
    at clojure.core$load.doInvoke(core.clj:5876)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.core$load_one.invokeStatic(core.clj:5697)
    at clojure.core$load_one.invoke(core.clj:5692)
    at clojure.core$load_lib$fn__5626.invoke(core.clj:5737)
    at clojure.core$load_lib.invokeStatic(core.clj:5736)
    at clojure.core$load_lib.doInvoke(core.clj:5717)
    at clojure.lang.RestFn.applyTo(RestFn.java:142)
    at clojure.core$apply.invokeStatic(core.clj:648)
    at clojure.core$load_libs.invokeStatic(core.clj:5774)
    at clojure.core$load_libs.doInvoke(core.clj:5758)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at clojure.core$apply.invokeStatic(core.clj:648)
    at clojure.core$require.invokeStatic(core.clj:5796)
    at clojure.core$require.doInvoke(core.clj:5796)
    at clojure.lang.RestFn.invoke(RestFn.java:619)
    at lego.components.rethinkdb$loading__5569__auto____21530.invoke(rethinkdb.clj:1)
    at lego.components.rethinkdb__init.load(Unknown Source)
    at lego.components.rethinkdb__init.<clinit>(Unknown Source)
    at java.lang.Class.forName0(Native Method) [rt.jar:1.8.0_102]
    at java.lang.Class.forName(Class.java:348) [rt.jar:1.8.0_102]
    at clojure.lang.RT.classForName(RT.java:2168)
    at clojure.lang.RT.classForName(RT.java:2177)
    at clojure.lang.RT.loadClassForName(RT.java:2196)
    at clojure.lang.RT.load(RT.java:443)
    at clojure.lang.RT.load(RT.java:419)
    at clojure.core$load$fn__5677.invoke(core.clj:5893)
    at clojure.core$load.invokeStatic(core.clj:5892)
    at clojure.core$load.doInvoke(core.clj:5876)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.core$load_one.invokeStatic(core.clj:5697)
    at clojure.core$load_one.invoke(core.clj:5692)
    at clojure.core$load_lib$fn__5626.invoke(core.clj:5737)
    at clojure.core$load_lib.invokeStatic(core.clj:5736)
    at clojure.core$load_lib.doInvoke(core.clj:5717)
    at clojure.lang.RestFn.applyTo(RestFn.java:142)
    at clojure.core$apply.invokeStatic(core.clj:648)
    at clojure.core$load_libs.invokeStatic(core.clj:5774)
    at clojure.core$load_libs.doInvoke(core.clj:5758)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at clojure.core$apply.invokeStatic(core.clj:648)
    at clojure.core$require.invokeStatic(core.clj:5796)
    at clojure.core$require.doInvoke(core.clj:5796)
    at clojure.lang.RestFn.invoke(RestFn.java:3894)
    at epiphany.system$loading__5569__auto____40.invoke(system.clj:1)
    at epiphany.system__init.load(Unknown Source)
    at epiphany.system__init.<clinit>(Unknown Source)
    at java.lang.Class.forName0(Native Method) [rt.jar:1.8.0_102]
    at java.lang.Class.forName(Class.java:348) [rt.jar:1.8.0_102]
    at clojure.lang.RT.classForName(RT.java:2168)
    at clojure.lang.RT.classForName(RT.java:2177)
    at clojure.lang.RT.loadClassForName(RT.java:2196)
    at clojure.lang.RT.load(RT.java:443)
    at clojure.lang.RT.load(RT.java:419)
    at clojure.core$load$fn__5677.invoke(core.clj:5893)
    at clojure.core$load.invokeStatic(core.clj:5892)
    at clojure.core$load.doInvoke(core.clj:5876)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.core$load_one.invokeStatic(core.clj:5697)
    at clojure.core$load_one.invoke(core.clj:5692)
    at clojure.core$load_lib$fn__5626.invoke(core.clj:5737)
    at clojure.core$load_lib.invokeStatic(core.clj:5736)
    at clojure.core$load_lib.doInvoke(core.clj:5717)
    at clojure.lang.RestFn.applyTo(RestFn.java:142)
    at clojure.core$apply.invokeStatic(core.clj:648)
    at clojure.core$load_libs.invokeStatic(core.clj:5774)
    at clojure.core$load_libs.doInvoke(core.clj:5758)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at clojure.core$apply.invokeStatic(core.clj:648)
    at clojure.core$require.invokeStatic(core.clj:5796)
    at clojure.core$require.doInvoke(core.clj:5796)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at immutant.internal.util$require_resolve.invokeStatic(util.clj:58)
    at immutant.internal.util$require_resolve.invoke(util.clj:55)
    at immutant.wildfly$init_deployment.invokeStatic(wildfly.clj:90)
    at immutant.wildfly$init_deployment.invoke(wildfly.clj:83)
    at clojure.core$eval70.invokeStatic(NO_SOURCE_FILE:0)
    at clojure.core$eval70.invoke(NO_SOURCE_FILE)
    at clojure.lang.Compiler.eval(Compiler.java:6927)
    at clojure.lang.Compiler.eval(Compiler.java:6917)
    at clojure.lang.Compiler.eval(Compiler.java:6917)
    at clojure.lang.Compiler.eval(Compiler.java:6890)
    at clojure.core$eval.invokeStatic(core.clj:3105)
    at clojure.core$eval.invoke(core.clj:3101)
    at clojure.lang.Var.invoke(Var.java:379)
    at org.projectodd.wunderboss.c
@danielcompton
Copy link
Collaborator

Can you upgrade the version of Netty that Wildly is using?

@ckirkendall
Copy link
Author

ckirkendall commented Feb 7, 2017 via email

@danielcompton
Copy link
Collaborator

Can you share as much of your dependency tree as possible? It's a bit hard to debug further without more details on the Netty versions. Also, is it possible that there are two Netty JARs on the classpath under different artifact ID's (e.g. netty-handler and netty-all)? Looking around at this error, it seems like that sometimes is the cause (though I can imagine other reasons for the problem too).

@ckirkendall
Copy link
Author

ckirkendall commented Feb 7, 2017

The issue come from Jboss and wildfly providing Netty and a specific version of netty. The only netty files are coming in from the rethinkdb.clj deps. The version of Widfly that I am trying to deploy to has 4.0.15-Final being provided. This normally wouldn't be an issue but aleph is highly coupled with specific versions of netty and rethinkdb is highly coupled with current release of aleph due to ssl support. Note: widlfly and jboss both use Netty as base module.

deps tree root for rethinkdb:

[com.apa512/rethinkdb "0.15.24" :exclusions [[cljsjs/react]]]
   [aleph "0.4.1"]
     [byte-streams "0.2.2"]
       [clj-tuple "0.2.2"]
       [primitive-math "0.1.5"]
     [io.aleph/dirigiste "0.1.3"]
     [io.netty/netty-all "4.1.0.CR3"]
     [manifold "0.1.4"]
       [riddley "0.1.12"]
     [potemkin "0.4.3"]
   [com.google.protobuf/protobuf-java "3.0.0-alpha-3.1"]
   [gloss "0.2.5"]
   [org.clojure/tools.logging "0.3.1"]
   [rethinkdb-protobuf "2.2.0.1"]

@ckirkendall
Copy link
Author

@danielcompton - Is there a reason you don't just wrap the java api? The RQL api seems like it matches your setup very closely. It feels like you could support almost everything you have with very little code around the official java api.

@danielcompton
Copy link
Collaborator

The last time I looked, the Java driver didn't have any async support which means each changefeed would tie up a whole thread (I think). It's not uncommon to have thousands of changefeeds open, so this is a problem. I'd love to switch to using the Java driver, and it would make a lot of things a lot easier, but we need some sort of async support.

Have you tried excluding io.netty/netty-all from the rethinkdb dependency?

I don't have time to look into switching to the Java driver at the moment, or adding async support to the official Java driver, but if you wanted to provide a patch, I'd be happy to review it.

Alternatively, if you're able to provide a patch to clj-rethinkdb which make the problematic netty code path optional, I'd be happy to remove that too. Are you trying to use SSL in your cluster?

@ckirkendall
Copy link
Author

@danielcompton - thanks for the reply. That is a bummer. It would simplify so many things. I notice there is suggestion to use Netty. Probably should link this issue there also. I don't know how to reference this on that thread. rethinkdb/rethinkdb#4802

@danielcompton
Copy link
Collaborator

Totally agree, it's a bit of a bummer. Now that there's a bit more certainty around RethinkDB, people (including me) may be able to invest more time into the Java driver.

If you can find a way to exclude the rethinkdb.ssl namespace unless it is needed, that could be a good option, but I'm not sure how (without doing dirty hacks).

@ckirkendall
Copy link
Author

@danielcompton - I wrote a version of query that wraps the java driver to get around the issue. I did my best to make it a drop in replacement of query that way when async comes to the java api it can be added without issue. Do you think it would be good to open up a light version like this?

@danielcompton
Copy link
Collaborator

Not quite sure what you mean by open up, but I'd be happy to see it as a PR, or as a separate project?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants