Writing Pandas DataFrame to SQLite
Start your free 7-days trial now!
To create and interact with the SQLite database, we make use of the sqlite3
library.
Solution
Consider the following DataFrame:
import pandas as pd
"A":[3,4], "B":[5.0,6.0], "C":["c","c"], "D":[True,False],
df
A B C D E0 3 5.0 c True 2021-12-251 4 6.0 c False 2021-12-26
To store this as a table in SQLite:
import sqlite3
# Create a connection to the SQLite database# Doesn't matter if the database does not yet existconn = sqlite3.connect('test_db.sqlite')df.to_sql('employees', conn, if_exists='replace', index=False)conn.close()
Here, note the following:
Running this script will create a new file called
test_db.sqlite
in the same directory as this script.Columns A to D will have the correct type derived in the SQLite database, but column E, which is of
datetime
type, will have typeunknown
in SQLite since SQLite does not supportdatetime
.index=False
is usually what you want becauseto_sql(~)
will automatically add the DataFrame index as a new column.
Reading SQLite table as a Pandas DataFrame
To read back this SQLite table as a DataFrame:
conn = sqlite3.connect('test_db.sqlite')df = pd.read_sql_query('SELECT * FROM employees', conn)conn.close()df
A B C D E0 3 5.0 c 1 2021-12-25 00:00:001 4 6.0 c 0 2021-12-26 00:00:00
Note the following:
There exists a method called
read_sql_table(~)
, but this is only relevant when you're using SQLAlchemy.Column E is derived to be of type
object
instead ofdatetime
- this is because SQLite does not supportdatetime
. To parse column E asdatetime
, set theparse_dates
arguments:conn = sqlite3.connect('test_db.sqlite')df = pd.read_sql_query('SELECT * FROM employees', conn, parse_dates=["E"])conn.close()df.dtypesindex int64A int64B float64C objectD int64E datetime64[ns]dtype: object