removing newlines from messy strings in pandas dataframe cells?

I’ve used multiple ways of splitting and stripping the strings in my pandas dataframe to remove all the ‘n’characters, but for some reason it simply doesn’t want to delete the characters that are attached to other words, even though I split them. I have a pandas dataframe with a column that captures text from web pages using Beautifulsoup. The text has been cleaned a bit already by beautifulsoup, but it failed in removing the newlines attached to other characters. My strings look a bit like this:

“hands-onndevelopment of games. We will study a variety of software technologiesnrelevant to games including programming languages, scriptingnlanguages, operating systems, file systems, networks, simulationnengines, and multi-media design systems. We will also study some ofnthe underlying scientific concepts from computer science and relatednfields including”

Is there an easy python way to remove these “n” characters?

Thanks in advance!

Answers:

Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.

Method 1

EDIT: the correct answer to this is:

df = df.replace(r'n',' ', regex=True)

I think you need replace:

df = df.replace('n','', regex=True)

Or:

df = df.replace('n',' ', regex=True)

Or:

df = df.replace(r'\n',' ', regex=True)

Sample:

text = '''hands-onndev nologiesnrelevant scriptingnlang
'''
df = pd.DataFrame({'A':[text]})
print (df)
                                                   A
0  hands-onndev nologiesnrelevant scriptingnla...

df = df.replace('n',' ', regex=True)
print (df)
                                                A
0  hands-on dev nologies relevant scripting lang

Method 2

df.replace(to_replace=[r"\t|\n|\r", "t|n|r"], value=["",""], regex=True, inplace=True)

worked for me.

Source:

https://gist.github.com/smram/d6ded3c9028272360eb65bcab564a18a

Method 3

To remove carriage return (r), new line (n) and tab (t)

df = df.replace(r'r+|n+|t+','', regex=True)

Method 4

in messy data it might to be a good idea to remove all whitespaces df.replace(r's', '', regex = True, inplace = True).

Method 5

   df = 'Sarah Marie Wimberly So so beautiful!!!nAbram Staten You guys look good man.nTJ Sloan I miss you guysn'

   df = df.replace(r'\n',' ', regex=True)

This worked for the messy data I had.

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0