Pandas DataFrame to List of Lists

It’s easy to turn a list of lists into a pandas dataframe:

import pandas as pd
df = pd.DataFrame([[1,2,3],[3,4,5]])

But how do I turn df back into a list of lists?

lol = df.what_to_do_now?
print lol
# [[1,2,3],[3,4,5]]

Answers:

Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.

Method 1

You could access the underlying array and call its tolist method:

>>> df = pd.DataFrame([[1,2,3],[3,4,5]])
>>> lol = df.values.tolist()
>>> lol
[[1L, 2L, 3L], [3L, 4L, 5L]]

Method 2

If the data has column and index labels that you want to preserve, there are a few options.

Example data:

>>> df = pd.DataFrame([[1,2,3],[3,4,5]], 
       columns=('first', 'second', 'third'), 
       index=('alpha', 'beta')) 
>>> df
       first  second  third
alpha      1       2      3
beta       3       4      5

The tolist() method described in other answers is useful but yields only the core data – which may not be enough, depending on your needs.

>>> df.values.tolist()
[[1, 2, 3], [3, 4, 5]]

One approach is to convert the DataFrame to json using df.to_json() and then parse it again. This is cumbersome but does have some advantages, because the to_json() method has some useful options.

>>> df.to_json()
{
  "first":{"alpha":1,"beta":3},
  "second":{"alpha":2,"beta":4},"third":{"alpha":3,"beta":5}
}

>>> df.to_json(orient='split')
{
 "columns":["first","second","third"],
 "index":["alpha","beta"],
 "data":[[1,2,3],[3,4,5]]
}

Cumbersome but may be useful.

The good news is that it’s pretty straightforward to build lists for the columns and rows:

>>> columns = [df.index.name] + [i for i in df.columns]
>>> rows = [[i for i in row] for row in df.itertuples()]

This yields:

>>> print(f"columns: {columns}nrows: {rows}") 
columns: [None, 'first', 'second', 'third']
rows: [['alpha', 1, 2, 3], ['beta', 3, 4, 5]]

If the None as the name of the index is bothersome, rename it:

df = df.rename_axis('stage')

Then:

>>> columns = [df.index.name] + [i for i in df.columns]
>>> print(f"columns: {columns}nrows: {rows}") 

columns: ['stage', 'first', 'second', 'third']
rows: [['alpha', 1, 2, 3], ['beta', 3, 4, 5]]

Method 3

I wanted to preserve the index, so I adapted the original answer to this solution:

list_df = df.reset_index().values.tolist()

Now you can paste it somewhere else (e.g. to paste into a Stack Overflow question) and latter recreate it:

pd.Dataframe(list_df, columns=['name1', ...])
pd.set_index(['name1'], inplace=True)

Method 4

I don’t know if it will fit your needs, but you can also do:

>>> lol = df.values
>>> lol
array([[1, 2, 3],
       [3, 4, 5]])

This is just a numpy array from the ndarray module, which lets you do all the usual numpy array things.

Method 5

I had this problem: how do I get the headers of the df to be in row 0 for writing them to row 1 in the excel (using xlsxwriter)? None of the proposed solutions worked, but they pointed me in the right direction. I just needed one line more of code

# get csv data
df = pd.read_csv(filename)

# combine column headers and list of lists of values
lol = [df.columns.tolist()] + df.values.tolist()

Method 6

Maybe something changed but this gave back a list of ndarrays which did what I needed.

list(df.values)

Method 7

Note: I have seen many cases on Stack Overflow where converting a Pandas Series or DataFrame to a NumPy array or plain Python lists is entirely unecessary. If you’re new to the library, consider double-checking whether the functionality you need is already offered by those Pandas objects.

To quote a comment by @jpp:

In practice, there’s often no need to convert the NumPy array into a list of lists.

If a Pandas DataFrame/Series won’t work, you can use the built-in DataFrame.to_numpy and Series.to_numpy methods.

Method 8

“df.values” returns a numpy array. This does not preserve the data types. An integer might be converted to a float.

df.iterrows() returns a series which also does not guarantee to preserve the data types. See: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.iterrows.html

The code below converts to a list of list and preserves the data types:

rows = [list(row) for row in df.itertuples()]

Method 9

If you wish to convert a Pandas DataFrame to a table (list of lists) and include the header column this should work:

import pandas as pd
def dfToTable(df:pd.DataFrame) -> list:
    return [list(df.columns)] + df.values.tolist()

Usage (in REPL):

>>> df = pd.DataFrame(
             [["r1c1","r1c2","r1c3"],["r2c1","r2c2","r3c3"]]
             , columns=["c1", "c2", "c3"])
>>> df
     c1    c2    c3
0  r1c1  r1c2  r1c3
1  r2c1  r2c2  r3c3
>>> dfToTable(df)
[['c1', 'c2', 'c3'], ['r1c1', 'r1c2', 'r1c3'], ['r2c1', 'r2c2', 'r3c3']]

Method 10

1. The solutions presented so far suffer from a “reinventing the wheel” approach. Quoting @AMC:

If you’re new to the library, consider double-checking whether the functionality you need is already offered by those Pandas objects.

2. If you convert a dataframe to a list of lists you will lose information – namely the index and columns names.

My solution: use to_dict()

dict_of_lists = df.to_dict(orient='split')

This will give you a dictionary with three lists: index, columns, data. If you decide you really don’t need the columns and index names, you get the data with

dict_of_lists['data']

Method 11

Not quite relate to the issue but another flavor with same expectation

converting data frame series into list of lists to plot the chart using create_distplot in Plotly

    hist_data=[]
    hist_data.append(map_data['Population'].to_numpy().tolist())

Method 12

We can use the DataFrame.iterrows() function to iterate over each of the rows of the given Dataframe and construct a list out of the data of each row:

# Empty list 
row_list =[] 

# Iterate over each row 
for index, rows in df.iterrows(): 
    # Create list for the current row 
    my_list =[rows.Date, rows.Event, rows.Cost] 

    # append the list to the final list 
    row_list.append(my_list) 

# Print 
print(row_list)

We can successfully extract each row of the given data frame into a list

Method 13

This is very simple:

import numpy as np

list_of_lists = np.array(df)

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0