How to save a pandas DataFrame

To save a DataFrame in pandas, call one of its to_* methods. For most work, use Parquet:

df.to_parquet("sales.parquet")
df = pd.read_parquet("sales.parquet")

Which one to choose depends on what has to survive the trip, and who reads the file next.

The formats compared

Format	Save with	Keeps types	Good for
Parquet	`df.to_parquet("sales.parquet")`	Yes	Your default: compact and fast, and read by most data tools
CSV	`df.to_csv("sales.csv", index=False)`	No	Anything that needs plain text
Excel	`df.to_excel("sales.xlsx", index=False)`	Partly	Handing a file to someone who opens it in a spreadsheet
Pickle	`df.to_pickle("sales.pkl")`	Yes	A short-lived cache in your own code
SQL	`df.to_sql("sales", engine)`	Mapped to the database’s types	Data that belongs in a database you already have

to_parquet needs pyarrow installed, and to_excel needs openpyxl.

The mistakes to look out for

CSV loses your types. Dates come back as text unless you pass parse_dates, IDs like 007 come back as the number 7, and a whole-number column with a missing value comes back as floats. If you must use CSV, say what you expect when you read it:

df = pd.read_csv("sales.csv", dtype={"id": "string"}, parse_dates=["sold_at"])

to_csv writes the index by default. Without index=False, reading the file back gives you an extra Unnamed: 0 column.

Only unpickle what you trust. Reading a pickle can run code hidden in the file, so never open one from someone else. Pickles can also fail to load in another version of pandas, so don’t use them to keep data for long.

Large files. Parquet is already compressed. For CSV, a name ending in .csv.gz is enough to make pandas compress it:

df.to_csv("sales.csv.gz", index=False)

When a file isn’t enough

A file is fine until the DataFrame changes, or someone else needs it. Then you get sales_final_v2.parquet in a shared folder, and nobody knows which one is right.

rho keeps a DataFrame as a table with its history, with one line of Python:

import rho

rho.put(df, "research/sales")                  # a new version; columns keep their types
df = rho.get("research/sales")                 # the latest, or version=3 for an earlier one
rho.restore("research/sales", version=3)       # undo, as a new version
url = rho.share("research/sales")              # a link anyone can open

Data travels as Parquet, so whole numbers stay whole and timestamps stay timestamps. Your AI agents can read and query the same table. The Python SDK guide has the rest.