Pandas
keyboard_arrow_down 655 guides
chevron_leftCreating DataFrames Cookbook
Combining multiple Series into a DataFrameCombining multiple Series to form a DataFrameConverting a Series to a DataFrameConverting list of lists into DataFrameConverting list to DataFrameConverting percent string into a numeric for read_csvConverting scikit-learn dataset to Pandas DataFrameConverting string data into a DataFrameCreating a DataFrame from a stringCreating a DataFrame using listsCreating a DataFrame with different type for each columnCreating a DataFrame with empty valuesCreating a DataFrame with missing valuesCreating a DataFrame with random numbersCreating a DataFrame with zerosCreating a MultiIndex DataFrameCreating a Pandas DataFrameCreating a single DataFrame from multiple filesCreating empty DataFrame with only column labelsFilling missing values when using read_csvImporting DatasetImporting tables from PostgreSQL as Pandas DataFramesInitialising a DataFrame using a constantInitialising a DataFrame using a dictionaryInitialising a DataFrame using a list of dictionariesInserting lists into a DataFrame cellKeeping leading zeroes when using read_csvParsing dates when using read_csvPreventing strings from getting parsed as NaN for read_csvReading data from GitHubReading file without headerReading large CSV files in chunksReading n random lines using read_csvReading space-delimited filesReading specific columns from fileReading tab-delimited filesReading the first few lines of a file to create DataFrameReading the last n lines of a fileReading URL using read_csvReading zipped csv file as a DataFrameRemoving Unnamed:0 columnResolving ParserError: Error tokenizing dataSaving DataFrame as zipped csvSkipping rows without skipping header for read_csvSpecifying data type for read_csvTreating missing values as empty strings rather than NaN for read_csv
check_circle
Mark as learned thumb_up
2
thumb_down
0
chat_bubble_outline
0
Comment auto_stories Bi-column layout
settings
Reading the last n lines of a file in Pandas
schedule Aug 10, 2023
Last updated local_offer
Tags Python●Pandas
tocTable of Contents
expand_more Master the mathematics behind data science with 100+ top-tier guides
Start your free 7-days trial now!
Start your free 7-days trial now!
The method read_csv(~) has no parameter that allows you to directly read the last n lines.
Solution
An indirect way of reading only the last n lines is to first fetch the total number of lines in the file, and then use the skiprows parameter.
Consider the following my_data.txt file:
A,B,C1,2,34,5,67,8,9
To read the last 2 lines of the file:
def get_num_lines(fname): with open(fname) as f: for i, _ in enumerate(f): pass return i + 1
num_lines = get_num_lines("my_data.txt")
n = 2df = pd.read_csv("my_data.txt", skiprows=range(1,num_lines-n))df
A B C0 4 5 61 7 8 9
Note the following:
we first begin by fetching the total number of lines in the file. In this case,
num_lines=4.since the first row of the file represents the column labels, we skip rows starting from line
1, as indicated byrange(1,_). If your file does not contain a header row, then simply setskiprows=num_lines-n.
Related
Pandas | read_csv method
Reads a file, and parses its content into a DataFrame.
Published by Isshin Inada
Edited by 0 others
Did you find this page useful?
thumb_up
thumb_down
Comment
Citation
Ask a question or leave a feedback...
thumb_up
2
thumb_down
0
chat_bubble_outline
0
settings
Enjoy our search
Hit / to insta-search docs and recipes!
