Limit RAM usage to python program

I’m trying to limit the RAM usage from a Python program to half so it doesn’t totally freezes when all the RAM is used, for this I’m using the following code which is not working and my laptop is still freezing:

import sys
import resource

def memory_limit():
    rsrc = resource.RLIMIT_DATA
    soft, hard = resource.getrlimit(rsrc)
    soft /= 2
    resource.setrlimit(rsrc, (soft, hard))

if __name__ == '__main__':
    memory_limit() # Limitates maximun memory usage to half
    try:
        main()
    except MemoryError:
        sys.stderr.write('MAXIMUM MEMORY EXCEEDED')
        sys.exit(-1)

I’m using other functions which I call from the main function.

What am I doing wrong?

Thanks in advance.

PD: I already searched about this and found the code I’ve put but it’s still not working…

Answers:

Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.

Method 1

I’ve done some research and found a function to get the memory from Linux systems here: Determine free RAM in Python and I modified it a bit to get just the free memory available and set the maximum memory available as its half.

Code:

import resource
import sys

def memory_limit():
    soft, hard = resource.getrlimit(resource.RLIMIT_AS)
    resource.setrlimit(resource.RLIMIT_AS, (get_memory() * 1024 / 2, hard))

def get_memory():
    with open('/proc/meminfo', 'r') as mem:
        free_memory = 0
        for i in mem:
            sline = i.split()
            if str(sline[0]) in ('MemFree:', 'Buffers:', 'Cached:'):
                free_memory += int(sline[1])
    return free_memory

if __name__ == '__main__':
    memory_limit() # Limitates maximun memory usage to half
    try:
        main()
    except MemoryError:
        sys.stderr.write('nnERROR: Memory Exceptionn')
        sys.exit(1)

The line to set it to the half is resource.setrlimit(resource.RLIMIT_AS, (get_memory() * 1024 / 2, hard)) where get_memory() * 1024 / 2 sets it to the half (it’s in bytes).

Method 2

I modify the answer of @Ulises CT. Because I think to change too much original function is not so good, so I turn it to a decorator. I hope it helps.

import resource
import platform
import sys

def memory_limit(percentage: float):
    """
    只在linux操作系统起作用
    """
    if platform.system() != "Linux":
        print('Only works on linux!')
        return
    soft, hard = resource.getrlimit(resource.RLIMIT_AS)
    resource.setrlimit(resource.RLIMIT_AS, (get_memory() * 1024 * percentage, hard))

def get_memory():
    with open('/proc/meminfo', 'r') as mem:
        free_memory = 0
        for i in mem:
            sline = i.split()
            if str(sline[0]) in ('MemFree:', 'Buffers:', 'Cached:'):
                free_memory += int(sline[1])
    return free_memory

def memory(percentage=0.8):
    def decorator(function):
        def wrapper(*args, **kwargs):
            memory_limit(percentage)
            try:
                return function(*args, **kwargs)
            except MemoryError:
                mem = get_memory() / 1024 /1024
                print('Remain: %.2f GB' % mem)
                sys.stderr.write('nnERROR: Memory Exceptionn')
                sys.exit(1)
        return wrapper
    return decorator

@memory(percentage=0.8)
def main():
    print('My memory is limited to 80%.')

Method 3

due to https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=34e431b0ae398fc54ea69ff85ec700722c9da773

it is better to use :

if str(sline[0]) == 'MemAvailable:':
            free_memory = int(sline[1])
            break

the answer code provided me with 8 GB of available mem on a machine with 1TB of RAM

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0