How to solve “OSError: telling position disabled by next() call”

I am creating a file editing system and would like to make a line based tell() function instead of a byte based one. This function would be used inside of a “with loop” with the open(file) call. This function is part of a class that has:

self.f = open(self.file, 'a+')
# self.file is a string that has the filename in it

The following is the original function
(It also has a char setting if you wanted line and byte return):

def tell(self, char=False):
    t, lc = self.f.tell(), 0
    self.f.seek(0)
    for line in self.f:
        if t >= len(line):
            t -= len(line)
            lc += 1
        else:
            break
    if char:
        return lc, t
    return lc

The problem I’m having with this is that this returns an OSError and it has to do with how the system is iterating over the file but I don’t understand the issue. Thanks to anyone who can help.

Answers:

Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.

Method 1

I don’t know if this was the original error but you can get the same error if you try to call f.tell() inside of a line-by-line iteration of a file like so:

with open(path, "r+") as f:
  for line in f:
    f.tell() #OSError

which can be easily substituted by the following:

with open(path, mode) as f:
  line = f.readline()
  while line:
    f.tell() #returns the location of the next line
    line = f.readline()

Method 2

I have an older version of Python 3, and I’m on Linux instead of a Mac, but I was able to recreate something very close to your error:

IOError: telling position disabled by next() call

An IO error, not an OS error, but otherwise the same. Bizarrely enough, I couldn’t cause it using your open('a+', ...), but only when opening the file in read mode: open('r+', ...).

Further muddling things is that the error comes from _io.TextIOWrapper, a class that appears to be defined in Python’s _pyio.py file… I stress “appears”, because:

1. The TextIOWrapper in that file has attributes like _telling that I can’t access on the whatever-it-is object calling itself _io.TextIOWrapper.
2. The TextIOWrapper class in _pyio.py doesn’t make any distinction between readable, writable, or random-access files. Either both should work, or both should raise the same IOError.

Regardless, the TextIOWrapper class as described in the _pyio.py file disables the tell method while the iteration is in progress. This seems to be what you’re running into (comments are mine):

def __next__(self):
    # Disable the tell method.
    self._telling = False
    line = self.readline()
    if not line:
        # We've reached the end of the file...
        self._snapshot = None
        # ...so restore _telling to whatever it was.
        self._telling = self._seekable
        raise StopIteration
    return line

In your tell method, you almost always break out of the iteration before it reaches the end of the file, leaving _telling disabled (False):

One other way to reset _telling is the flush method, but it also failed if called while the iteration was in progress:

IOError: can't reconstruct logical file position

The way around this, at least on my system, is to call seek(0) on the TextIOWrapper, which restores everything to a known state (and successfully calls flush in the bargain):

def tell(self, char=False):
    t, lc = self.f.tell(), 0
    self.f.seek(0)
    for line in self.f:
        if t >= len(line):
            t -= len(line)
            lc += 1
        else:
            break
    # Reset the file iterator, or later calls to f.tell will
    # raise an IOError or OSError:
    f.seek(0)
    if char:
        return lc, t
    return lc

If that’s not the solution for your system, it might at least tell you where to start looking.

PS: You should consider always returning both the line number and the character offset. Functions that can return completely different types are hard to deal with — it’s a lot easier for the caller to just throw away the value her or she doesn’t need.

Method 3

Just a quick workaround for this issue:

As you are iterating over the file from the beginning anyways, just keep track of where you are with a dedicated variable:

file_pos = 0
with open('file.txt', 'rb') as f:
    for line in f:
        # process line
        file_pos += len(line)

Now file_pos will always be, what file.tell() would tell you. Note that this only works for ASCII files as tell and seek work with byte positions. Working on a line-basis it’s easy though to convert strings from byte to unicode-strings.

Method 4

I had the same error: OSError: telling position disabled by next() call, and solved it by adding the ‘rb’ mode while opening the file.

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0