Pandas

655 guides

keyboard_arrow_down

Other math topics

Dagster

Pandas

NumPy

Matplotlib

PySpark

MySQL

chevron_leftCreating DataFrames Cookbook

Combining multiple Series into a DataFrame Combining multiple Series to form a DataFrame Converting a Series to a DataFrame Converting list of lists into DataFrame Converting list to DataFrame Converting percent string into a numeric for read_csv Converting scikit-learn dataset to Pandas DataFrame Converting string data into a DataFrame Creating a DataFrame from a string Creating a DataFrame using lists Creating a DataFrame with different type for each column Creating a DataFrame with empty values Creating a DataFrame with missing values Creating a DataFrame with random numbers Creating a DataFrame with zeros Creating a MultiIndex DataFrame Creating a Pandas DataFrame Creating a single DataFrame from multiple files Creating empty DataFrame with only column labels Filling missing values when using read_csv Importing Dataset Importing tables from PostgreSQL as Pandas DataFrames Initialising a DataFrame using a constant Initialising a DataFrame using a dictionary Initialising a DataFrame using a list of dictionaries Inserting lists into a DataFrame cell Keeping leading zeroes when using read_csv Parsing dates when using read_csv Preventing strings from getting parsed as NaN for read_csv Reading data from GitHub Reading file without header Reading large CSV files in chunks Reading n random lines using read_csv Reading space-delimited files Reading specific columns from file Reading tab-delimited files Reading the first few lines of a file to create DataFrame Reading the last n lines of a file Reading URL using read_csv Reading zipped csv file as a DataFrame Removing Unnamed:0 column Resolving ParserError: Error tokenizing data Saving DataFrame as zipped csv Skipping rows without skipping header for read_csv Specifying data type for read_csv Treating missing values as empty strings rather than NaN for read_csv

check_circle

Mark as learned

thumb_up

thumb_down

chat_bubble_outline

Comment

auto_stories Bi-column layout

settings

Creating a single DataFrame from multiple files in Pandas

schedule Aug 10, 2023

Last updated

local_offer

Python●Pandas

When files contain columns

Consider the following my_data_one.txt file:

And my_data_two.txt file:

To read and combine the two files into a single DataFrame:


        
        
            
                
                
                    df_one = pd.read_csv("my_data_one.txt")
df_two = pd.read_csv("my_data_two.txt")
pd.concat([df_one, df_two], axis=1)
                
            
               A  B  C   D
0  3  4  10  11
1  5  6  12  13

Here, axis=1 indicates that we want to concatenate the DataFrames horizontally, as opposed to vertically.

When files contain rows

Consider the following my_data_one.txt file:

And my_data_two.txt:

To construct a single DataFrame from the two files:


        
        
            
                
                
                    df_one = pd.read_csv("my_data_one.txt")
df_two = pd.read_csv("my_data_two.txt")
df = pd.concat([df_one, df_two], axis=0, ignore_index=True)
df
                
            
               A  B
0  3  4
1  5  6
2  10  11
3  12  13

Note the following:

axis=0 for concat(~) means that we want to stack the DataFrames vertically, as opposed to horizontally.
ignore_index=True for concat(~) resets the index of the resulting DataFrame to the default integer indices ([0,1,2,3]). Without this parameter, we would end up with duplicate index values ([0,1,0,1]),