Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error running deca cnv : java.lang.OutOfMemoryError: GC overhead limit exceeded #36

Open
lakhujanivijay opened this issue Feb 29, 2020 · 2 comments
Open

Error running deca cnv : java.lang.OutOfMemoryError: GC overhead limit exceeded #36

lakhujanivijay opened this issue Feb 29, 2020 · 2 comments

Comments

@lakhujanivijay
Copy link

My command

date && time deca-submit cnv -L Exome_Target_hg19.bed -o $(pwd) -I S23.bam

Error message

Sat Feb 29 12:29:55 IST 2020
Using SPARK_SUBMIT=/home/bioinfo/spark-2.1.0-bin-hadoop2.7//bin/spark-submit
2020-02-29 12:29:56 WARN  NativeCodeLoader:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2020-02-29 12:29:56 WARN  Utils:66 - Your hostname, bioinfo-pc resolves to a loopback address: 127.0.1.1; using 192.168.0.84 instead (on interface eno1)
2020-02-29 12:29:56 WARN  Utils:66 - Set SPARK_LOCAL_IP if you need to bind to another address
2020-02-29 12:32:37 ERROR Executor:91 - Exception in task 7.0 in stage 1.0 (TID 23)
java.lang.OutOfMemoryError: GC overhead limit exceeded
	at scala.reflect.ManifestFactory$$anon$9.newArray(Manifest.scala:117)
	at scala.reflect.ManifestFactory$$anon$9.newArray(Manifest.scala:115)
	at breeze.linalg.DenseVector$.zeros$mIc$sp(DenseVector.scala:254)
	at org.bdgenomics.deca.Coverage$.fragmentOverlap(Coverage.scala:119)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$5$$anonfun$apply$3.apply$mcVI$sp(Coverage.scala:221)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$5$$anonfun$apply$3.apply(Coverage.scala:220)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$5$$anonfun$apply$3.apply(Coverage.scala:220)
	at scala.collection.mutable.HashSet.foreach(HashSet.scala:78)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$5.apply(Coverage.scala:220)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$5.apply(Coverage.scala:219)
	at scala.Option.foreach(Option.scala:257)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1.apply(Coverage.scala:219)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1.apply(Coverage.scala:211)
	at scala.collection.Iterator$class.foreach(Iterator.scala:893)
	at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2.apply$mcV$sp(Coverage.scala:211)
	at scala.util.control.Breaks.breakable(Breaks.scala:38)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1.apply(Coverage.scala:210)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1.apply(Coverage.scala:173)
	at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:796)
	at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:796)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96)
	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
	at org.apache.spark.scheduler.Task.run(Task.scala:99)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
2020-02-29 12:32:40 ERROR SparkUncaughtExceptionHandler:91 - Uncaught exception in thread Thread[Executor task launch worker-4,5,main]
java.lang.OutOfMemoryError: GC overhead limit exceeded
	at scala.reflect.ManifestFactory$$anon$9.newArray(Manifest.scala:117)
	at scala.reflect.ManifestFactory$$anon$9.newArray(Manifest.scala:115)
	at breeze.linalg.DenseVector$.zeros$mIc$sp(DenseVector.scala:254)
	at org.bdgenomics.deca.Coverage$.fragmentOverlap(Coverage.scala:119)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$5$$anonfun$apply$3.apply$mcVI$sp(Coverage.scala:221)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$5$$anonfun$apply$3.apply(Coverage.scala:220)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$5$$anonfun$apply$3.apply(Coverage.scala:220)
	at scala.collection.mutable.HashSet.foreach(HashSet.scala:78)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$5.apply(Coverage.scala:220)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$5.apply(Coverage.scala:219)
	at scala.Option.foreach(Option.scala:257)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1.apply(Coverage.scala:219)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1.apply(Coverage.scala:211)
	at scala.collection.Iterator$class.foreach(Iterator.scala:893)
	at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2.apply$mcV$sp(Coverage.scala:211)
	at scala.util.control.Breaks.breakable(Breaks.scala:38)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1.apply(Coverage.scala:210)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1.apply(Coverage.scala:173)
	at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:796)
	at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:796)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96)
	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
	at org.apache.spark.scheduler.Task.run(Task.scala:99)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
2020-02-29 12:32:43 WARN  TaskSetManager:66 - Lost task 7.0 in stage 1.0 (TID 23, localhost, executor driver): java.lang.OutOfMemoryError: GC overhead limit exceeded
	at scala.reflect.ManifestFactory$$anon$9.newArray(Manifest.scala:117)
	at scala.reflect.ManifestFactory$$anon$9.newArray(Manifest.scala:115)
	at breeze.linalg.DenseVector$.zeros$mIc$sp(DenseVector.scala:254)
	at org.bdgenomics.deca.Coverage$.fragmentOverlap(Coverage.scala:119)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$5$$anonfun$apply$3.apply$mcVI$sp(Coverage.scala:221)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$5$$anonfun$apply$3.apply(Coverage.scala:220)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$5$$anonfun$apply$3.apply(Coverage.scala:220)
	at scala.collection.mutable.HashSet.foreach(HashSet.scala:78)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$5.apply(Coverage.scala:220)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$5.apply(Coverage.scala:219)
	at scala.Option.foreach(Option.scala:257)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1.apply(Coverage.scala:219)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1.apply(Coverage.scala:211)
	at scala.collection.Iterator$class.foreach(Iterator.scala:893)
	at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2.apply$mcV$sp(Coverage.scala:211)
	at scala.util.control.Breaks.breakable(Breaks.scala:38)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1.apply(Coverage.scala:210)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1.apply(Coverage.scala:173)
	at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:796)
	at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:796)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96)
	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
	at org.apache.spark.scheduler.Task.run(Task.scala:99)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

2020-02-29 12:32:44 ERROR TaskSetManager:70 - Task 7 in stage 1.0 failed 1 times; aborting job
2020-02-29 12:32:45 ERROR LiveListenerBus:70 - SparkListenerBus has already stopped! Dropping event SparkListenerStageCompleted(org.apache.spark.scheduler.StageInfo@1fe62c66)
2020-02-29 12:32:45 ERROR LiveListenerBus:70 - SparkListenerBus has already stopped! Dropping event SparkListenerJobEnd(1,1582959765130,JobFailed(org.apache.spark.SparkException: Job aborted due to stage failure: Task 7 in stage 1.0 failed 1 times, most recent failure: Lost task 7.0 in stage 1.0 (TID 23, localhost, executor driver): java.lang.OutOfMemoryError: GC overhead limit exceeded
	at scala.reflect.ManifestFactory$$anon$9.newArray(Manifest.scala:117)
	at scala.reflect.ManifestFactory$$anon$9.newArray(Manifest.scala:115)
	at breeze.linalg.DenseVector$.zeros$mIc$sp(DenseVector.scala:254)
	at org.bdgenomics.deca.Coverage$.fragmentOverlap(Coverage.scala:119)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$5$$anonfun$apply$3.apply$mcVI$sp(Coverage.scala:221)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$5$$anonfun$apply$3.apply(Coverage.scala:220)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$5$$anonfun$apply$3.apply(Coverage.scala:220)
	at scala.collection.mutable.HashSet.foreach(HashSet.scala:78)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$5.apply(Coverage.scala:220)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$5.apply(Coverage.scala:219)
	at scala.Option.foreach(Option.scala:257)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1.apply(Coverage.scala:219)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1.apply(Coverage.scala:211)
	at scala.collection.Iterator$class.foreach(Iterator.scala:893)
	at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2.apply$mcV$sp(Coverage.scala:211)
	at scala.util.control.Breaks.breakable(Breaks.scala:38)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1.apply(Coverage.scala:210)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1.apply(Coverage.scala:173)
	at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:796)
	at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:796)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96)
	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
	at org.apache.spark.scheduler.Task.run(Task.scala:99)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Driver stacktrace:))
Command body threw exception:
org.apache.spark.SparkException: Job aborted due to stage failure: Task 7 in stage 1.0 failed 1 times, most recent failure: Lost task 7.0 in stage 1.0 (TID 23, localhost, executor driver): java.lang.OutOfMemoryError: GC overhead limit exceeded
	at scala.reflect.ManifestFactory$$anon$9.newArray(Manifest.scala:117)
	at scala.reflect.ManifestFactory$$anon$9.newArray(Manifest.scala:115)
	at breeze.linalg.DenseVector$.zeros$mIc$sp(DenseVector.scala:254)
	at org.bdgenomics.deca.Coverage$.fragmentOverlap(Coverage.scala:119)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$5$$anonfun$apply$3.apply$mcVI$sp(Coverage.scala:221)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$5$$anonfun$apply$3.apply(Coverage.scala:220)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$5$$anonfun$apply$3.apply(Coverage.scala:220)
	at scala.collection.mutable.HashSet.foreach(HashSet.scala:78)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$5.apply(Coverage.scala:220)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$5.apply(Coverage.scala:219)
	at scala.Option.foreach(Option.scala:257)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1.apply(Coverage.scala:219)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1.apply(Coverage.scala:211)
	at scala.collection.Iterator$class.foreach(Iterator.scala:893)
	at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2.apply$mcV$sp(Coverage.scala:211)
	at scala.util.control.Breaks.breakable(Breaks.scala:38)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1.apply(Coverage.scala:210)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1.apply(Coverage.scala:173)
	at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:796)
	at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:796)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96)
	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
	at org.apache.spark.scheduler.Task.run(Task.scala:99)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Driver stacktrace:
2020-02-29 12:32:45 WARN  TaskSetManager:66 - Lost task 14.0 in stage 1.0 (TID 30, localhost, executor driver): TaskKilled (killed intentionally)
2020-02-29 12:32:45 WARN  TaskSetManager:66 - Lost task 4.0 in stage 1.0 (TID 20, localhost, executor driver): TaskKilled (killed intentionally)
2020-02-29 12:32:45 WARN  TaskSetManager:66 - Lost task 13.0 in stage 1.0 (TID 29, localhost, executor driver): TaskKilled (killed intentionally)
2020-02-29 12:32:45 WARN  TaskSetManager:66 - Lost task 15.0 in stage 1.0 (TID 31, localhost, executor driver): TaskKilled (killed intentionally)
Exception in thread "main" org.apache.spark.SparkException: Job aborted due to stage failure: Task 7 in stage 1.0 failed 1 times, most recent failure: Lost task 7.0 in stage 1.0 (TID 23, localhost, executor driver): java.lang.OutOfMemoryError: GC overhead limit exceeded
	at scala.reflect.ManifestFactory$$anon$9.newArray(Manifest.scala:117)
	at scala.reflect.ManifestFactory$$anon$9.newArray(Manifest.scala:115)
	at breeze.linalg.DenseVector$.zeros$mIc$sp(DenseVector.scala:254)
	at org.bdgenomics.deca.Coverage$.fragmentOverlap(Coverage.scala:119)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$5$$anonfun$apply$3.apply$mcVI$sp(Coverage.scala:221)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$5$$anonfun$apply$3.apply(Coverage.scala:220)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$5$$anonfun$apply$3.apply(Coverage.scala:220)
	at scala.collection.mutable.HashSet.foreach(HashSet.scala:78)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$5.apply(Coverage.scala:220)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$5.apply(Coverage.scala:219)
	at scala.Option.foreach(Option.scala:257)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1.apply(Coverage.scala:219)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1.apply(Coverage.scala:211)
	at scala.collection.Iterator$class.foreach(Iterator.scala:893)
	at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2.apply$mcV$sp(Coverage.scala:211)
	at scala.util.control.Breaks.breakable(Breaks.scala:38)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1.apply(Coverage.scala:210)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1.apply(Coverage.scala:173)
	at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:796)
	at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:796)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96)
	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
	at org.apache.spark.scheduler.Task.run(Task.scala:99)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Driver stacktrace:
	at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1435)
	at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1423)
	at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1422)
	at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
	at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
	at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1422)
	at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:802)
	at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:802)
	at scala.Option.foreach(Option.scala:257)
	at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:802)
	at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1650)
	at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1605)
	at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1594)
	at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
	at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:628)
	at org.apache.spark.SparkContext.runJob(SparkContext.scala:1918)
	at org.apache.spark.SparkContext.runJob(SparkContext.scala:1981)
	at org.apache.spark.rdd.RDD$$anonfun$reduce$1.apply(RDD.scala:1025)
	at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
	at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
	at org.apache.spark.rdd.RDD.withScope(RDD.scala:362)
	at org.apache.spark.rdd.RDD.reduce(RDD.scala:1007)
	at org.apache.spark.rdd.RDD$$anonfun$treeAggregate$1.apply(RDD.scala:1150)
	at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
	at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
	at org.apache.spark.rdd.RDD.withScope(RDD.scala:362)
	at org.apache.spark.rdd.RDD.treeAggregate(RDD.scala:1127)
	at org.apache.spark.mllib.linalg.distributed.RowMatrix.computeColumnSummaryStatistics(RowMatrix.scala:419)
	at org.bdgenomics.deca.Normalization$.filterColumns(Normalization.scala:44)
	at org.bdgenomics.deca.Normalization$$anonfun$normalizeReadDepth$1$$anonfun$5.apply(Normalization.scala:189)
	at org.bdgenomics.deca.Normalization$$anonfun$normalizeReadDepth$1$$anonfun$5.apply(Normalization.scala:188)
	at scala.Option.fold(Option.scala:158)
	at org.apache.spark.rdd.Timer.time(Timer.scala:48)
	at org.bdgenomics.deca.Normalization$$anonfun$normalizeReadDepth$1.apply(Normalization.scala:188)
	at org.bdgenomics.deca.Normalization$$anonfun$normalizeReadDepth$1.apply(Normalization.scala:185)
	at scala.Option.fold(Option.scala:158)
	at org.apache.spark.rdd.Timer.time(Timer.scala:48)
	at org.bdgenomics.deca.Normalization$.normalizeReadDepth(Normalization.scala:185)
	at org.bdgenomics.deca.cli.CNVer.run(CNVer.scala:139)
	at org.bdgenomics.utils.cli.BDGSparkCommand$class.run(BDGCommand.scala:55)
	at org.bdgenomics.deca.cli.CNVer.run(CNVer.scala:91)
	at org.bdgenomics.deca.cli.DecaMain$$anonfun$run$3.apply(DecaMain.scala:71)
	at org.bdgenomics.deca.cli.DecaMain$$anonfun$run$3.apply(DecaMain.scala:70)
	at scala.Option.fold(Option.scala:158)
	at org.bdgenomics.deca.cli.DecaMain.run(DecaMain.scala:70)
	at org.bdgenomics.deca.cli.DecaMain$.main(DecaMain.scala:26)
	at org.bdgenomics.deca.cli.DecaMain.main(DecaMain.scala)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
	at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
	at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
	at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
	at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
	at scala.reflect.ManifestFactory$$anon$9.newArray(Manifest.scala:117)
	at scala.reflect.ManifestFactory$$anon$9.newArray(Manifest.scala:115)
	at breeze.linalg.DenseVector$.zeros$mIc$sp(DenseVector.scala:254)
	at org.bdgenomics.deca.Coverage$.fragmentOverlap(Coverage.scala:119)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$5$$anonfun$apply$3.apply$mcVI$sp(Coverage.scala:221)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$5$$anonfun$apply$3.apply(Coverage.scala:220)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$5$$anonfun$apply$3.apply(Coverage.scala:220)
	at scala.collection.mutable.HashSet.foreach(HashSet.scala:78)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$5.apply(Coverage.scala:220)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1$$anonfun$apply$5.apply(Coverage.scala:219)
	at scala.Option.foreach(Option.scala:257)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1.apply(Coverage.scala:219)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2$$anonfun$apply$mcV$sp$1.apply(Coverage.scala:211)
	at scala.collection.Iterator$class.foreach(Iterator.scala:893)
	at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1$$anonfun$apply$2.apply$mcV$sp(Coverage.scala:211)
	at scala.util.control.Breaks.breakable(Breaks.scala:38)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1.apply(Coverage.scala:210)
	at org.bdgenomics.deca.Coverage$$anonfun$sortedCoverageCalculation$1.apply(Coverage.scala:173)
	at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:796)
	at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:796)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96)
	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
	at org.apache.spark.scheduler.Task.run(Task.scala:99)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

real	2m49.621s
user	32m37.424s
sys	0m3.242s
@heuermh
Copy link
Member

heuermh commented Feb 29, 2020

Hello @lakhujanivijay, thank you for submitting this issue!

What Spark config settings were you running with, and what does your cluster look like?

@lakhujanivijay
Copy link
Author

Thanks @heuermh for the response.

I don't know where to find that information about spark config settings. I just know the command I am running. And I am running this command on a workstation which has 16 threads , 32 Gb RAM.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@heuermh @lakhujanivijay