Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Caught a NaN error : 3 #286

Open
yolic66 opened this issue Apr 8, 2024 · 0 comments
Open

Caught a NaN error : 3 #286

yolic66 opened this issue Apr 8, 2024 · 0 comments

Comments

@yolic66
Copy link

yolic66 commented Apr 8, 2024

EPOCH 0
Caught a NaN error :
3
Graph execution error:

Detected at node 'loss/mul' defined at (most recent call last):
File "c:\Users\dell\Desktop\RandLA-Net-master\main_Semantic3D.py", line 324, in
model = Network(dataset, cfg)
File "c:\Users\dell\Desktop\RandLA-Net-master\RandLANet.py", line 80, in init
self.loss = self.get_loss(valid_logits, valid_labels, self.class_weights)
File "c:\Users\dell\Desktop\RandLA-Net-master\RandLANet.py", line 266, in get_loss
weights = tf.reduce_sum(class_weights * one_hot_labels, axis=1)
Node: 'loss/mul'
Detected at node 'loss/mul' defined at (most recent call last):
File "c:\Users\dell\Desktop\RandLA-Net-master\main_Semantic3D.py", line 324, in
model = Network(dataset, cfg)
File "c:\Users\dell\Desktop\RandLA-Net-master\RandLANet.py", line 80, in init
self.loss = self.get_loss(valid_logits, valid_labels, self.class_weights)
File "c:\Users\dell\Desktop\RandLA-Net-master\RandLANet.py", line 266, in get_loss
weights = tf.reduce_sum(class_weights * one_hot_labels, axis=1)
Node: 'loss/mul'
2 root error(s) found.
(0) INVALID_ARGUMENT: required broadcastable shapes
[[{{node loss/mul}}]]
[[optimizer/gradients/layers/Encoder_layer_0LFAmlp1/Conv2D_grad/Conv2DBackpropFilter/_1021]]
(1) INVALID_ARGUMENT: required broadcastable shapes
[[{{node loss/mul}}]]
0 successful operations.
0 derived errors ignored.
name: "loss/mul"
op: "Mul"
input: "loss/Const"
input: "loss/one_hot"
attr {
key: "T"
value {
type: DT_FLOAT
}
}

loss/mul
['loss/Const:0', 'loss/one_hot:0']
['loss/mul:0']
Traceback (most recent call last):
File "D:\software\Anaconda\envs\etmvsnet\lib\site-packages\tensorflow\python\client\session.py", line 1377, in _do_call
return fn(*args)
File "D:\software\Anaconda\envs\etmvsnet\lib\site-packages\tensorflow\python\client\session.py", line 1360, in _run_fn
return self._call_tf_sessionrun(options, feed_dict, fetch_list,
File "D:\software\Anaconda\envs\etmvsnet\lib\site-packages\tensorflow\python\client\session.py", line 1453, in _call_tf_sessionrun
return tf_session.TF_SessionRun_wrapper(self._session, options, feed_dict,
tensorflow.python.framework.errors_impl.InvalidArgumentError: 2 root error(s) found.
(0) INVALID_ARGUMENT: required broadcastable shapes
[[{{node loss/mul}}]]
[[optimizer/gradients/layers/Encoder_layer_0LFAmlp1/Conv2D_grad/Conv2DBackpropFilter/_1021]]
(1) INVALID_ARGUMENT: required broadcastable shapes
[[{{node loss/mul}}]]
0 successful operations.
0 derived errors ignored.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "c:\Users\dell\Desktop\RandLA-Net-master\RandLANet.py", line 162, in train
_, _, summary, l_out, probs, labels, acc = self.sess.run(ops, {self.is_training: True})
File "D:\software\Anaconda\envs\etmvsnet\lib\site-packages\tensorflow\python\client\session.py", line 967, in run
result = self._run(None, fetches, feed_dict, options_ptr,
File "D:\software\Anaconda\envs\etmvsnet\lib\site-packages\tensorflow\python\client\session.py", line 1190, in _run
results = self._do_run(handle, final_targets, final_fetches,
File "D:\software\Anaconda\envs\etmvsnet\lib\site-packages\tensorflow\python\client\session.py", line 1370, in _do_run
return self._do_call(_run_fn, feeds, fetches, targets, options,
File "D:\software\Anaconda\envs\etmvsnet\lib\site-packages\tensorflow\python\client\session.py", line 1396, in _do_call
raise type(e)(node_def, op, message) # pylint: disable=no-value-for-parameter
tensorflow.python.framework.errors_impl.InvalidArgumentError: Graph execution error:

Detected at node 'loss/mul' defined at (most recent call last):
File "c:\Users\dell\Desktop\RandLA-Net-master\main_Semantic3D.py", line 324, in
model = Network(dataset, cfg)
File "c:\Users\dell\Desktop\RandLA-Net-master\RandLANet.py", line 80, in init
self.loss = self.get_loss(valid_logits, valid_labels, self.class_weights)
File "c:\Users\dell\Desktop\RandLA-Net-master\RandLANet.py", line 266, in get_loss
weights = tf.reduce_sum(class_weights * one_hot_labels, axis=1)
Node: 'loss/mul'
Detected at node 'loss/mul' defined at (most recent call last):
File "c:\Users\dell\Desktop\RandLA-Net-master\main_Semantic3D.py", line 324, in
model = Network(dataset, cfg)
File "c:\Users\dell\Desktop\RandLA-Net-master\RandLANet.py", line 80, in init
self.loss = self.get_loss(valid_logits, valid_labels, self.class_weights)
File "c:\Users\dell\Desktop\RandLA-Net-master\RandLANet.py", line 266, in get_loss
weights = tf.reduce_sum(class_weights * one_hot_labels, axis=1)
Node: 'loss/mul'
2 root error(s) found.
(0) INVALID_ARGUMENT: required broadcastable shapes
[[{{node loss/mul}}]]
[[optimizer/gradients/layers/Encoder_layer_0LFAmlp1/Conv2D_grad/Conv2DBackpropFilter/_1021]]
(1) INVALID_ARGUMENT: required broadcastable shapes
[[{{node loss/mul}}]]
0 successful operations.
0 derived errors ignored.

Original stack trace for 'loss/mul':
File "c:\Users\dell\Desktop\RandLA-Net-master\main_Semantic3D.py", line 324, in
model = Network(dataset, cfg)
File "c:\Users\dell\Desktop\RandLA-Net-master\RandLANet.py", line 80, in init
self.loss = self.get_loss(valid_logits, valid_labels, self.class_weights)
File "c:\Users\dell\Desktop\RandLA-Net-master\RandLANet.py", line 266, in get_loss
weights = tf.reduce_sum(class_weights * one_hot_labels, axis=1)
File "D:\software\Anaconda\envs\etmvsnet\lib\site-packages\tensorflow\python\util\traceback_utils.py", line 150, in error_handler
return fn(*args, **kwargs)
File "D:\software\Anaconda\envs\etmvsnet\lib\site-packages\tensorflow\python\ops\math_ops.py", line 1406, in binary_op_wrapper
return func(x, y, name=name)
File "D:\software\Anaconda\envs\etmvsnet\lib\site-packages\tensorflow\python\ops\math_ops.py", line 1766, in _mul_dispatch
return multiply(x, y, name=name)
File "D:\software\Anaconda\envs\etmvsnet\lib\site-packages\tensorflow\python\util\traceback_utils.py", line 150, in error_handler
return fn(*args, **kwargs)
File "D:\software\Anaconda\envs\etmvsnet\lib\site-packages\tensorflow\python\util\dispatch.py", line 1082, in op_dispatch_handler
return dispatch_target(*args, **kwargs)
File "D:\software\Anaconda\envs\etmvsnet\lib\site-packages\tensorflow\python\ops\math_ops.py", line 529, in multiply
return gen_math_ops.mul(x, y, name)
File "D:\software\Anaconda\envs\etmvsnet\lib\site-packages\tensorflow\python\ops\gen_math_ops.py", line 6588, in mul
_, _, _op, _outputs = _op_def_library._apply_op_helper(
File "D:\software\Anaconda\envs\etmvsnet\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 740, in _apply_op_helper
op = g._create_op_internal(op_type_name, inputs, dtypes=None,
File "D:\software\Anaconda\envs\etmvsnet\lib\site-packages\tensorflow\python\framework\ops.py", line 3776, in _create_op_internal
ret = Operation(
File "D:\software\Anaconda\envs\etmvsnet\lib\site-packages\tensorflow\python\framework\ops.py", line 2175, in init
self._traceback = tf_stack.extract_stack_for_node(self._c_op)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "c:\Users\dell\Desktop\RandLA-Net-master\main_Semantic3D.py", line 325, in
model.train(dataset)
File "c:\Users\dell\Desktop\RandLA-Net-master\RandLANet.py", line 199, in train
a = 1 / 0
ZeroDivisionError: division by zero

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant