Error in merging split and loaded files

Asked 2 months ago, Updated 2 months ago, 4 views

I read the file separately to handle a huge csv file, but the subsequent merge results in an error.

The following code

import pandas as pd
fname = '.../train.csv'
reader=pd.read_csv(fname, chunksize=10000)
df = pd.concat(r for in reader), ignore_index = True)

The following error message

TypeError: first argument must be an interactive of pandas objects, you passed an object of type "DataFrame"

Please tell me the cause and countermeasures.Thank you for your cooperation.

python python3 pandas

2022-09-30 11:05

1 Answers

There seems to be no problem with the code itself, so try limiting the version and number of lines of the pandas to see if the csv file is loading well as follows:

import pandas as pd

print(pd.__version__)
fname = '.../train.csv'
df=pd.read_csv(fname,nrows=10000)
print(df.info())
print(df.head())

Also, if there is no problem, you can change the code as follows to see if there is an error.If the error occurs, insert a debug code to see where the error is occurring.

import pandas as pd
fname = '.../train.csv'
df = None
for rin pd.read_csv(fname, chunksize=10000):
    if df is None:
        df = r
    else:
        df=df.append(r,ignore_index=True)


2022-09-30 11:05

If you have any answers or tips


© 2022 OneMinuteCode. All rights reserved.