How to open large csv files?

[thefuzzyrich]

Recently got my hands on a databreach but I'm not exactly sure how to open it. It's 9gb and 150m rows long. Tried using something csv viewer but it just freezes. Not sure if it's because my VM cant handle it, or if the program just sucks. How should I open the file? Any help is much appreciated!

 

[jrrvrl]

Try klogg

 

[bili]

Recently got my hands on a databreach but I'm not exactly sure how to open it. It's 9gb and 150m rows long. Tried using something csv viewer but it just freezes. Not sure if it's because my VM cant handle it, or if the program just sucks. How should I open the file? Any help is much appreciated!

Click to shrink...

Opening large CSV files can be tricky because they may be too big for standard spreadsheet programs like Excel. Here is the method to handle large CSV files:

Pandas is a powerful library in Python that can read large CSV files in chunks. This method is ideal for handling large files efficiently.

Python

import pandas as pd

# Read the CSV file in chunks
chunk_size = 100000 # Adjust chunk size as needed
chunks = pd.read_csv('large_file.csv', chunksize=chunk_size)

for chunk in chunks:
# Process each chunk
print(chunk.head()) # Example: display first few rows of the chunk

Click to shrink...