Polars (software)

Last updated

Polars
Original author(s) Ritchie Vink
Developer(s) Community
Repository github.com/pola-rs/polars/
Written in Rust
Operating system Cross-platform
Type Technical computing
License MIT License
Website pola.rs

Polars is an open-source software library for data manipulation. Polars is built with an OLAP query engine implemented in Rust using Apache Arrow Columnar Format as the memory model. Although built using Rust, there are Python, Node.js, R, and SQL API interfaces to use Polars.

Contents

As of September 2025, Polars has over 24 million monthly downloads [1] [2] and over 250 million downloads in total [3] .

History

The first code to be committed was made on June 23, 2020. [4] Polars started as a "pet project" by Ritchie Vink, who was motivated to by the limitations of the software tool pandas that is used to organize and work with data. Vink aimed to address those limitations with a data processing library written in the Rust programming language. [5] [2]

Ritchie Vink and Chiel Peters co-founded a company of the same name to develop Polars, after working together at the company Xomnia for five years. In 2023, Vink and Peters successfully closed a seed round of approximately $4 million, which was led by Bain Capital Ventures. [2] [6]

In September 2025, Vink and Peters raised € 18,000,000 (about US$ 21,000,000) in a Series A round led by Accel, along with Bain Capital Partners and angel investors. [2]

Vink and Peters have also developed other services like Polars Cloud and Polars Distributed that are built around Polars. [2]

Features

The core object in Polars is the DataFrame, similar to other data processing software libraries. [7] Contexts and expressions are important concepts to Polars' syntax. A context is the specific environment in which an expression is evaluated. Meanwhile, an expression refers to computations or transformations that are performed on data columns.

Polars has three main contexts:

Polars was also designed to be "intuitive and [have] concise syntax for data processing tasks". [8]

Compared with other data processing software

Compared to pandas

Feature differences

Given that Polars was designed to work on a single machine, this prompts many comparisons with the similar data manipulation software, pandas. [9] One big advantage that Polars has over pandas is performance, where Polars is 5 to 10 times faster than pandas on similar tasks. Additionally, pandas requires around 5 to 10 times as much RAM as the size of the dataset, which compares to the 2 to 4 times needed for Polars. These performance increases may be due to Polars being written in Rust and supporting parallel operations. [10]

Polars is also designed to use lazy evaluation (where a query optimizer will use the most efficient evaluation after looking at all steps) compared with pandas using eager evaluation (where steps are performed immediately). Some research on comparing pandas and Polars completing data analysis tasks show that Polars is more memory-efficient than pandas, [11] where "Polars consumes 63% of the energy needed by pandas on the TPC-H benchmark and uses eight times less energy than pandas on synthetic data". [8]

Polars does not have an index for the DataFrame object, which contrasts pandas' use of an index. [10]

Syntax differences

Polars and pandas have similar syntax for reading in data using a read_csv() method, but have different syntax for calculating a rolling mean. [12]

Code using pandas:

importpandasaspd# Read in datadf_temp=pd.read_csv("temp_record.csv",index_col="date",parse_dates=True,dtype={"temp":int})# Explore dataprint(df_temp.dtypes)print(df_temp.head())# Calculate rolling meandf_temp.rolling(2).mean()

Code using Polars:

importpolarsaspl# Read in datadf_temp=pl.read_csv("temp_record.csv",try_parse_dates=True,dtypes={"temp":int}).set_sorted("date")# Explore dataprint(df_temp.dtypes)print(df_temp.head())# Calculate rolling averagedf_temp.rolling("date",period="2d").agg(pl.mean("temp"))

Compared to Dask

Dask is a Python package for applying parallel computation using NumPy, pandas, and scikit-learn, and is used for datasets that are larger than what can fit in memory. Polars is for single-machine use, while Dask is more for distributed computing. [8]

Compared to DuckDB

DuckDB is an in-process SQL OLAP database system for efficient analytical queries on structured data. Both DuckDB and Polars offer excellent analytical performance, but DuckDB is more SQL-centric for running queries, while Polars is Python-centric. [8]

Compared to Spark

Apache Spark has a Python API, PySpark, for distributed big data processing. Similar to Dask, Spark is focused on distributed computing, while Polars is for single-machine use. So Polars has an advantage when processing data on a single machine, while Spark may be preferred for larger datasets that don't fit on a single machine. [8]

See also

References

  1. ↑ "PyPI Download Stats". pypistats.org. Retrieved 29 September 2025.
  2. 1 2 3 4 5 Heim, Anna (29 September 2025). "The startup behind open source tool Polars raises $21M from Accel". TechCrunch. Retrieved 29 September 2025.
  3. ↑ "Pepy.tech Total downloads stats". pepy.tech. Retrieved 1 October 2025.{{cite web}}: CS1 maint: url-status (link)
  4. ↑ "Company announcement". www.pola.rs. Retrieved 13 May 2025.
  5. ↑ "Seed funding of US $4M for Polars, one of the fastest DataFrame libraries in Python & Rust - Xomnia". 16 May 2025. Retrieved 30 May 2025.
  6. ↑ "Polars DataFrame Library Announces Seed Funding from Bain Capital Ventures". Business Wire . Retrieved 29 September 2025.
  7. ↑ Python, Real. "Python Polars: A Lightning-Fast DataFrame Library – Real Python". realpython.com. Retrieved 13 May 2025.
  8. 1 2 3 4 5 "1. Introducing Polars - Python Polars: The Definitive Guide [Book]". O'Reilly Media . Retrieved 30 May 2025.
  9. ↑ "Polars vs. pandas: What's the Difference? | The PyCharm Blog". The JetBrains Blog. 4 July 2024. Retrieved 13 May 2025.
  10. 1 2 "Using the Polars DataFrame Library". CODE. Retrieved 30 May 2025.
  11. ↑ Nahrstedt, Felix; Karmouche, Mehdi; Bargieł, Karolina; Banijamali, Pouyeh; Nalini Pradeep Kumar, Apoorva; Malavolta, Ivano (18 June 2024). "An Empirical Study on the Energy Usage and Performance of Pandas and Polars Data Analysis Python Libraries". Proceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering. EASE '24. New York, NY, USA: Association for Computing Machinery. pp. 58–68. doi:10.1145/3661167.3661203. ISBN   979-8-4007-1701-7.
  12. ↑ "How to Move From pandas to Polars | The PyCharm Blog". The JetBrains Blog. 19 June 2024. Retrieved 13 May 2025.

Further reading

This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.