Memray not working on generated binary file

[Partho]

Hello,
I have created a FastAPI middleware to perform memory profiling, the code looks like the following:

def register_memory_profiler(app: FastAPI):
    @app.middleware("http")
    async def profile_request(request: Request, call_next):
        """
            Profile the memory usage of the current request
        """
        memray_file_path = memray.FileDestination('/tmp/output.bin', overwrite=True)
        with memray.Tracker(destination=memray_file_path, trace_python_allocators=True):
            response = await call_next(request)
        subprocess.run(["memray", "summary", "/tmp/output.bin"])
        return response

Now, after this function is being called, the /tmp/output.bin gets generated.
However, when I try to run the following command on terminal:

$ memray summary /tmp/output.bin

the system complains the following message:

Failed to parse allocation records in /tmp/output.bin
Reason: The provided input file does not look like a binary generated by memray.

I ran ls, and I get this:

dsci@dsciclassifierapilumalee1-6b4c686d7-bddp6:~/classifier-api/app$ ls -al /tmp
total 8
drwxrwxrwt 1 root root    4096 Apr 24 19:46 .
drwxr-xr-x 1 root root    4096 Apr 24 19:33 ..
-rw-r--r-- 1 dsci science    0 Apr 24 19:46 output.bin

Any idea why I am getting this message and why the byte written is 0? It was working fine in my local docker image without issues but causing failure in our k8s pod in QA. How can I resolve this issue?

[godlygeek]

Well, 0 bytes is definitely wrong... after the with block ends, that file should definitely not be empty (though it can be while tracking is still running).

Perhaps a new request came in and a new Tracker started, overwriting the file before it could be read??

memray.Tracker sets up global state - there can only be one instance of it active at a time. If a second request comes in before that first one has finished, nothing good is going to happen - I think the failure mode might be that it successfully overwrites the file before it eventually fails to activate tracking, resulting in an empty file being left around.

Something like this (untested) might do the trick:

class MaybeTracker:
    def __init__(self):
        self.tracker = None
        self.counter = 0

    def __enter__(self):
        if self.tracker is None:
            self.counter += 1
            memray_file_path = memray.FileDestination(f"/tmp/output.bin.{self.counter}")
            self.tracker = memray.Tracker(destination=memray_file_path, trace_python_allocators=True)
            self.tracker.__enter__()

    def __exit__(self, *args):
        if self.tracker is not None:
            self.tracker.__exit__(*args)
            self.tracker = None
            subprocess.run(["memray", "summary", f"/tmp/output.bin.{self.counter}"])


maybe_tracker = MaybeTracker()

def register_memory_profiler(app: FastAPI):
    @app.middleware("http")
    async def profile_request(request: Request, call_next):
        """
            Profile the memory usage of the current request
        """
        memray_file_path = memray.FileDestination('/tmp/output.bin', overwrite=True)
        with maybe_tracker:
            response = await call_next(request)
        return response

That's got two important changes relative to your version - first, it doesn't attempt to activate tracking reentrantly. Once one coroutine starts tracking, nothing else will touch the tracker until that coroutine deactivates it. Secondly, instead of overwriting the capture files, it writes sequentially numbered ones.

Hopefully those two changes do the trick.