Access a list within a dataframe pandas

I'm working on dataframes with PANDAS in Python. I read a csv file with pandas.read_csv(...). In this csv file the column 'possible_stops' contains lists (each cells in this column looks like : [ ] or [str0,str1]).

I want to iterate over the rows with itertuples() and for each row i want to access the list inside 'possible_stops' column. However, the dtype of this column is object and when i look for the type of a specific cell (type(row.possible_stops)) it returns str. I think there is something to do with dtype = .. inside the read_csv method but i don't exactly know what to do.

How can i convert cells inside this column into lists ?

Thanks very much!

EDIT : the resulting df looks like :

latitude longitude gps_speed possible_stops    
1.05869 1574942547  0.0 [['Bapeaume', 2]]
1.05862 1573910439  0.0 [['Bapeaume', 2]]
1.05862 1573910441  0.0 [['Bapeaume', 2]]
1.05862 1573910443  0.0 [['Bapeaume', 2]]
1.05862 1573910445  0.0 [['Bapeaume', 2]]
1.05862 1573910447  0.0 [['Bapeaume', 2]]

2 answers

  • answered 2020-03-25 14:34 Rens

    It is not possible for an item in a column in a Pandas dataframe to have the data type 'list'. See a list of possible datatypes for a dataframe column here: https://pbpython.com/pandas_dtypes.html.

    So instead of it having the type 'list' when stored in a dataframe a list is inserted as a string object. However, it is still possible to enter the items in the lists in the dataframe with a double for-loop. For example to print each item:

    import pandas as pd
    
    data = {'possible_stops':[['str0', 'str1'], ['str2', 'str3']], 'Age':[[20, 21], [19, 18]]} 
    df = pd.DataFrame(data) 
    for row in df.itertuples():
        for item in row.possible_stops:
            print(item)
    

    This will print the names in the created dataset one by one.

  • answered 2020-03-25 14:49 Pi-R

    I found a way to convert the cells into lists with ast.literal_eval()

    for row in selection_traces.itertuples():
    l=ast.literal_eval(row.possible_stops)
    

    The problem has been already solved on this forum... Sorry and thanks you !