Pandas

655 guides

keyboard_arrow_down

Other math topics

Dagster

Pandas

NumPy

Matplotlib

PySpark

MySQL

chevron_leftCreating DataFrames Cookbook

Combining multiple Series into a DataFrame Combining multiple Series to form a DataFrame Converting a Series to a DataFrame Converting list of lists into DataFrame Converting list to DataFrame Converting percent string into a numeric for read_csv Converting scikit-learn dataset to Pandas DataFrame Converting string data into a DataFrame Creating a DataFrame from a string Creating a DataFrame using lists Creating a DataFrame with different type for each column Creating a DataFrame with empty values Creating a DataFrame with missing values Creating a DataFrame with random numbers Creating a DataFrame with zeros Creating a MultiIndex DataFrame Creating a Pandas DataFrame Creating a single DataFrame from multiple files Creating empty DataFrame with only column labels Filling missing values when using read_csv Importing Dataset Importing tables from PostgreSQL as Pandas DataFrames Initialising a DataFrame using a constant Initialising a DataFrame using a dictionary Initialising a DataFrame using a list of dictionaries Inserting lists into a DataFrame cell Keeping leading zeroes when using read_csv Parsing dates when using read_csv Preventing strings from getting parsed as NaN for read_csv Reading data from GitHub Reading file without header Reading large CSV files in chunks Reading n random lines using read_csv Reading space-delimited files Reading specific columns from file Reading tab-delimited files Reading the first few lines of a file to create DataFrame Reading the last n lines of a file Reading URL using read_csv Reading zipped csv file as a DataFrame Removing Unnamed:0 column Resolving ParserError: Error tokenizing data Saving DataFrame as zipped csv Skipping rows without skipping header for read_csv Specifying data type for read_csv Treating missing values as empty strings rather than NaN for read_csv

check_circle

Mark as learned

thumb_up

thumb_down

chat_bubble_outline

Comment

auto_stories Bi-column layout

settings

Parsing dates when using read_csv in Pandas

schedule Aug 12, 2023

Last updated

local_offer

Python●Pandas

Parsing columns as datetime

Consider the following my_data.txt file:


        
        
            
                
                
                    A,B
2020/12/25,7
2020/12,8
2020,9

To parse column A as a datetime when using read_csv(~):


        
        
            
                
                
                    df = pd.read_csv("my_data.txt", parse_dates=["A"])
df.dtypes
                
            
            A    datetime64[ns]
B             int64
dtype: object

Parsing index as datetime

Consider the following my_data.txt file:


        
        
            
                
                
                    A
2020/12/25,7
2020/12,8
2020,9

To parse the index as datetime:


        
        
            
                
                
                    df = pd.read_csv("my_data.txt", parse_dates=True)
df
                
            
                        A
2020-12-25  7
2020-12-01  8
2020-01-01  9

Here, the index is of type DatetimeIndex:


        
        
            
                
                
                    df.index
                
            
            DatetimeIndex(['2020-12-25', '2020-12-01', '2020-01-01'], dtype='datetime64[ns]', freq=None)

Combining multiple columns to form a single datetime column

Consider the following my_data.txt file:

Using a nested list

To combine columns Year and Month to form a single datetime column:


        
        
            
                
                
                    df = pd.read_csv("my_data.txt", parse_dates=[["Year","Month"]])
df
                
            
               Year_Month
0  2020-07-01
1  2020-08-01
2  2020-09-01

To confirm its data type:


        
        
            
                
                
                    df.dtypes
                
            
            Year_Month    datetime64[ns]
dtype: object

Using a dictionary

To combine columns Year and Month to form a single datetime column:


        
        
            
                
                
                    df = pd.read_csv("my_data.txt", parse_dates={"A":["Year","Month"]})
df
                
            
               A
0  2020-07-01
1  2020-08-01
2  2020-09-01

Using a dictionary is more flexible than using a nested list because:

you can specify a label to the combined column (e.g. "A" in this case)
you can specify multiple groups of columns to combine as a single date column.

Pandas | read_csv method

Reads a file, and parses its content into a DataFrame.

chevron_right

Published by Isshin Inada

Edited by 0 others

Did you find this page useful?

thumb_up

thumb_down

Comment

Citation

Ask a question or leave a feedback...

thumb_up

thumb_down

chat_bubble_outline

settings

Enjoy our search

Hit / to insta-search docs and recipes!

Parsing dates when using read_csv in Pandas

Parsing columns as datetime

Parsing index as datetime

Combining multiple columns to form a single datetime column

Using a nested list

Using a dictionary

Related