Coffee dies on assembling degree 8 bilinear form.

[dham]

COFFEE chokes on the following:

from firedrake import *

# Uncommenting this makes the breakage go away.
#parameters["coffee"] = {}

basemesh = UnitSquareMesh(1, 1, quadrilateral=True)
m = ExtrudedMesh(basemesh, 1)

fs = FunctionSpace(m, "Q", 8)
u = TrialFunction(fs)
v = TestFunction(fs)

form = inner(grad(u), grad(v)) * dx

m = assemble(form)

The error is:

Traceback (most recent call last):
  File "crash.py", line 12, in <module>
    m = assemble(form)
  File "/data/dham/src/firedrake/src/firedrake/firedrake/assemble.py", line 102, in assemble
    allocate_only=allocate_only)
  File "<decorator-gen-279>", line 2, in _assemble
  File "/data/dham/src/firedrake/src/firedrake/firedrake/utils.py", line 62, in wrapper
    return f(*args, **kwargs)
  File "/data/dham/src/firedrake/src/firedrake/firedrake/assemble.py", line 192, in _assemble
    kernels = tsfc_interface.compile_form(f, "form", parameters=form_compiler_parameters, inverse=inverse)
  File "/data/dham/src/firedrake/src/firedrake/firedrake/tsfc_interface.py", line 193, in compile_form
    number_map).kernels
  File "/data/dham/src/firedrake/src/PyOP2/pyop2/caching.py", line 200, in __new__
    obj = make_obj()
  File "/data/dham/src/firedrake/src/PyOP2/pyop2/caching.py", line 190, in make_obj
    obj.__init__(*args, **kwargs)
  File "/data/dham/src/firedrake/src/firedrake/firedrake/tsfc_interface.py", line 121, in __init__
    kernels.append(KernelInfo(kernel=Kernel(ast, ast.name, opts=opts),
  File "/data/dham/src/firedrake/src/PyOP2/pyop2/caching.py", line 200, in __new__
    obj = make_obj()
  File "/data/dham/src/firedrake/src/PyOP2/pyop2/caching.py", line 190, in make_obj
    obj.__init__(*args, **kwargs)
  File "/data/dham/src/firedrake/src/PyOP2/pyop2/base.py", line 3852, in __init__
    self._code = self._ast_to_c(self._ast, opts)
  File "/data/dham/src/firedrake/src/PyOP2/pyop2/sequential.py", line 73, in _ast_to_c
    ast_handler.plan_cpu(self._opts)
  File "/data/dham/src/firedrake/src/COFFEE/coffee/plan.py", line 121, in plan_cpu
    loop_opt.rewrite(rewrite)
  File "/data/dham/src/firedrake/src/COFFEE/coffee/optimizer.py", line 156, in rewrite
    self._recoil()
  File "/data/dham/src/firedrake/src/COFFEE/coffee/optimizer.py", line 492, in _recoil
    resource.RLIM_INFINITY))
ValueError: not allowed to raise maximum limit

[miklos1]

I cannot reproduce this.

[wence-]

Coffee's optimiser tries to call setrlimit if it thinks it made a kernel that allocated more stack space than 2MB. Possibly you don't have the correct permissions to make that call? Out of interest, what is the determined stack size?

[dham]

In [5]: resource.getrlimit(resource.RLIMIT_STACK)
Out[5]: (33554432, 33554432)

In [6]: resource.setrlimit(resource.RLIMIT_STACK, (resource.RLIM_INFINITY,
   ...:                                                            resource.RLIM_INFINITY))
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-6-e8a33e844aa9> in <module>()
      1 resource.setrlimit(resource.RLIMIT_STACK, (resource.RLIM_INFINITY,
----> 2                                                            resource.RLIM_INFINITY))

ValueError: not allowed to raise maximum limit

[wence-]

So this code does:

        decls = visit(self.header, info_items=['decls'])['decls']

        # Assume the size of a C type double is 8 bytes
        c_double_size = 8
        # Assume the stack size is 1.7 MB (2 MB is usually the limit)
        stack_size = 1.7*1024*1024

        decls = [d for d in decls.values() if d.size]
        size = sum([reduce(operator.mul, d.sym.rank) for d in decls])

        if size * c_double_size > stack_size:
            # Increase the stack size if the kernel's stack size seems to outreach
            # the space available
            try:
                resource.setrlimit(resource.RLIMIT_STACK, (resource.RLIM_INFINITY,
                                                           resource.RLIM_INFINITY))
            except resource.error:
                warn("Stack may blow up, could not increase its size.")

So you have a machine where both the soft and hard stack limits are 32MB. Without root, you can't up the stack size above the hard limit (which this code is doing).

On mine, and I think @miklos1's, the hard limit is unlimited, so we're fine.

This code is well sketchy anyway. It should at least ask for the current stack limit before deciding to change it...

You could band-aid by also catching ValueError and just emitting the warning (it may work).

[dham]

The bandaid solution appears to work. However....

[miklos1]

Related: firedrakeproject/tsfc#127.