Blog — bodo.ai

Our Blog_

Bodo Engineering Blog

The latest and greatest benchmarks, technical overviews, and product updates from the Bodo engineering team.

Fully Open Source Trustworthy Text-to-Analytics Stack for Iceberg Data with PyDough and BodoSQL

July 13, 2026

•

We demonstrate the first fully open source text-to-analytics stack for Apache Iceberg data in a simple workflow. This stack brings together three open source technologies: PyDough, BodoSQL, and Apache Iceberg.

New Interactive BodoSQL for the AI Era

June 24, 2026

•

We introduce the latest version of BodoSQL, designed for high-performance interactive query execution on massive datasets. We introduce a new C++-based execution backend that reduces query startup latency while preserving scalability and performance.

Building a Native GPU Reader for Apache Iceberg

June 4, 2026

•

Here is a deep look into how we designed our GPU-native Iceberg source operator, the architectural tradeoffs we weighed, and the specific edges we encountered while making the storage layer safe for hardware acceleration.

Building a Native GPU Iceberg Writer for Apache Iceberg

May 28, 2026

•

Here is a look at the internals of how we built our native GPU Iceberg writer. By building Iceberg write support directly into Bodo’s native execution engine, the result is not simply a faster Parquet write path—but a GPU-native Iceberg sink that preserves the throughput of the broader distributed pipeline.

Bodo vs Dask-CuDF: TPC-H on Distributed GPU clusters

May 21, 2026

•

Benchmarking all 22 TPC-H queries on a multi-node GPU cluster and show an overall speedup of more than 3× over Dask-CuDF across the full benchmark suite. These results highlight the advantages of an MPI-based SPMD execution model for communication-intensive analytical workloads.

Scaling GPU DataFrames: Why Bodo’s SPMD Architecture Outperforms Task-Based Engines

May 5, 2026

•

By combining a database-grade optimizer, streaming execution, and an efficient HPC-style distributed runtime, Bodo optimizes and executes entire workloads across nodes and devices, maximizing performance while minimizing overhead.

Building PyDough: Moving Correctness Upstream with a Knowledge Graph

March 10, 2026

•

In PyDough, accuracy does not depend primarily on prompt tuning or semantic-layer completeness. For relational reasoning to be reliable—especially in the context of large language models—it must sit on top of an explicit structural model of meaning. In PyDough, that model is a knowledge graph.

Why Python Must Be Considered a First Class Programming Platform

October 20, 2021

•

Earlier this month, ZDNet reported that Python is challenging C and Java to become the most popular programming language. We’d like to see that happen. But maybe not why you think.

Using LLMs at Scale: A Pandas-Native Approach

September 4, 2025

•

Using LangChain Agents + Bodo DataFrames on Large Datasets

October 21, 2025

•

We’ll show how to use langchain_bodo—Bodo’s integration with LangChain, a popular framework that lets you easily build AI agents—as well as the NYC Taxi dataset to create an agent capable of answering complex questions on real world data, such as estimating the average fare between Newark airport and Manhattan.

Using Bodo to Read and Write Amazon S3 Tables in Python

February 12, 2025

•

AWS S3 Tables is a new service that simplifies Iceberg open table format usage by providing setup and maintenance of Iceberg tables automatically. This includes built-in table maintenance and optimization, which simplifies managing large datasets. However, the main option currently for using S3 Tables is Apache Spark, which requires complex configuration and a Java-based engine...

The Economics of Cloud Computing: How to Reduce Costs and Maximize Value

•

The allure of the cloud is undeniable. It promises unprecedented scalability, on-demand resources, and the flexibility to adapt to ever-changing data requirements. However, the journey to cloud utopia comes with some complexities, particularly regarding cost management: How do you strike the perfect balance between reaping the benefits of cloud resources and optimizing costs?

Simplify and Accelerate Data Science at Scale with Bodo

July 27, 2021

•

To be production-ready often requires rewriting a significant portion of the native Python source code in another architecture. But what if you can scale linearly and reliably, in vanilla Python?

Scaling Amazon S3 Vectors Workflows Effortlessly in Python with Bodo

August 11, 2025

•

Run any Python library in parallel with @bodo.wrap_python

January 30, 2025

•

Python’s ecosystem is filled with powerful libraries, and the last few years have made it the go-to language for machine learning and AI. However, running custom operations from these libraries in parallel is challenging because it is a hassle to use anything but built-in operations in most parallel engines. This complexity often leads to tedious workarounds or suboptimal performance.

Robustness and Resilience of Bodo Execution in Data Pipelines

April 22, 2022

•

Overheads of master-executor systems like Spark have been justified as a “necessary evil” for achieving resilience. However, we have shown that Bodo can achieve much higher resilience without extra overheads.

Rethinking DataFrames: Easy as Pandas, Fast as a Data Warehouse

May 13, 2025

•

DataFrames have become the go-to abstraction for working with structured data in Python—and for good reason. Lightweight, intuitive, and versatile, libraries like Pandas have empowered millions of developers to explore and transform data easily. But as the scale and complexity of workloads have grown, a new question has emerged...

Bodo Dataframe Library: First Look

May 19, 2025

•

Reduce Snowballing Cloud Costs with Bodo

August 18, 2023

•

Why cloud costs can spiral when processing and storing big data and how to minimize compute costs with Bodo’s efficient compute engine.

PyTorch Conference 2025: Recap and Key Takeaways

November 4, 2025

•

Last week, some of us had the chance to attend PyTorch Conference 2025. We wanted to share some of our key takeaways from the conference; from distributed training to efficient inference, as well as highlight one aspect that is absolutely crucial but received less attention: data infrastructure.

Python DataFrames (Bodo, Daft, Polars, PySpark, Dask, Modin/Ray) Compete for Your NYC Taxi Fare

September 17, 2025

•

In this post, we run the full NYC Taxi workload and compare Bodo DataFrames with both the earlier systems and the Bodo JIT compiler.

Python data processing engine comparison with NYC Taxi trips (Bodo vs. Spark, Dask, Ray)

January 20, 2025

•

We compared popular Python data processing engines using the NYC Taxi dataset to evaluate performance and scalability. The results revealed significant differences, highlighting how the right engine can optimize both speed and cost for data-intensive workloads.

Performance and Cost Evaluation of Bodo vs. Spark, Dask, and Ray

December 1, 2021

•

Findings derived from the standard TPC-H benchmarks (22 in all) to compare Bodo’s economic and speed performance to Spark, Dask, and Ray for data processing workloads.

Parallel FASTA Aggregation with Bodo: A Python Example for Scalable Bioinformatics

February 20, 2025

•

In modern bioinformatics, large-scale genomic data is the norm, requiring efficient ways to read, process, and combine massive collections of sequence files for meaningful analyses. Reading multiple large FASTA files—a routine step in genomics—can become a bottleneck when datasets scale. This post shows how to efficiently parse and combine FASTA sequences in parallel using Bodo...

Pandas 3’s Native Integration with Bodo JIT and Iceberg

February 19, 2026

•

Pandas 3 takes a major step forward in performance and scalability by introducing native Bodo JIT integration for accelerating UDFs, and native Apache Iceberg support for reliable, scalable data management. We walk through both integrations and show how to get started.

Overcoming the Hidden Debt of Big-Data Analytics

August 24, 2021

•

What if we could improve analytics performance by 1,000x and and reduce aggregate operational expenses costs to 1/10 -- using the programming techniques and hardware already in use today? With our Series A funding, that’s what we at Bodo are committed to doing.

Making pd.DataFrame.apply Faster with Bodo

July 12, 2021

•

The issue with the Pandas apply function is that it can be unbearably slow when working with big data. Bodo makes up for Pandas’ lack of speed while staying equally powerful and user-friendly as Pandas.

Machine Learning Series: NYC Yellow Taxi Tips Prediction

May 17, 2022

•

Bodo allows machine learning practitioners to rapidly explore data and build complex pipelines. Using Bodo, developers can seamlessly scale their codes from using their own laptop to using Bodo's platform. In this series, we will...

Machine Learning Series: Credit Card Fraud Detection

May 17, 2022

•

In this post, we will run the code using Bodo. It means that the data will be distributed in chunks across processes. Bodo's documentation provides more information about the parallel execution model. If you want to run the example using Pandas only (without Bodo), simply...

Leveraging Bodo, PyTorch, and MPI for Efficient CPU-GPU Distributed Training in Large-Scale Machine Learning

April 9, 2025

•

Training deep learning models efficiently isn’t just about powerful GPUs—it starts with how you move and prepare your data. In this post, we’ll explore how to create a flexible system capable of scaling across heterogeneous CPU and GPU environments. Using Bodo and its autoparallelizing inferential compiler, we’ll preprocess data on CPUs at scale and send it in batches to GPUs for distributed training in PyTorch...

Key Takeaways from Iceberg Summit 2025

April 22, 2025

•

A few of us from Bodo recently attended the Iceberg Summit, and it was great to see how Iceberg is rapidly evolving to meet the demands of modern data infrastructure. As long-time champions of open table formats and scalable analytics, we were excited not just to attend—but also to contribute...

Introducing PyDough-CE: A simpler, safer path for natural language analytics

October 27, 2025

•

Today we’re releasing PyDough Community Edition (PyDough-CE), an open source project that finally makes natural language analytics both practical and safe.

Natively Integrated Python and SQL with Extreme Performance and Scale: Introducing BodoSQL

May 25, 2022

•

Python is the language of choice for AI and machine learning (ML), but SQL has been used...

Iceberg I/O performance comparison at scale (Bodo vs PyIceberg, Spark, Daft)

July 28, 2025

•

Iceberg I/O Optimizations in Compute Engines – Summit Recap

June 19, 2025

•

At this year’s Iceberg Summit, I had the opportunity to present a talk alongside my colleague Srinivas Lade on I/O optimizations in compute engines, particularly within the context of the Bodo engine.

Bodo & Iceberg: The Simple and Fast Open Data Warehouse of the Future

August 4, 2022

•

Bodo on Iceberg simplifies data management for users substantially by supporting native Python (in addition to SQL). The connector eliminates the need for an involved setup process and parameter tuning to make sure data engineers and data scientists are solving business problems instead of fighting data infrastructure.

How to Scale Your Python Application: Multiprocessing Library vs. Bodo

February 23, 2022

•

For a long time, the python multiprocessing library has been a solution for many data scientists and engineers to get faster results when processing time is a pain point. I want to show you a much faster solution: Bodo.

How to parallelize your LLM inference calls with Bodo

March 13, 2025

•

Large Language Models (LLMs) can be computationally intense and inference speed can quickly become a bottleneck—especially when you're sending multiple queries and waiting for responses. Slow response times don’t just lead to delays; they make real-time applications impractical, drive up compute costs, and create scalability challenges...

Bodo and Pandas: How to Accelerate Large Data Analytics on Your Laptop

August 4, 2021

•

Using only a single core for computation on a dataset with millions or even billions of rows can quickly become a disaster. The good news is Bodo has a solution for data scientists when Pandas does not meet their expectations.

From Iceberg to LoRa: A Unified LLM Fine-Tuning Pipeline with Bodo

November 6, 2025

•

We show how to use Bodo DataFrames to load and preprocess data directly from an Apache Iceberg table and feed it seamlessly into Bodo AI Toolkit to fine-tune a Llama 3.1 8B model using LoRa. We'll be training a chatbot on its own "liked" feedback to improve its responses.

Extending Parallel Python Computing to FPGAs

January 4, 2022

•

Today I’m happy to announce our collaboration with Xilinx, Inc., including their taking an investment in Bodo. This will be meaningful to more than just Xilinx and Bodo customers... It signifies another stage of our democratizing access to large-scale parallel computing using Python.

Moving Seamlessly from Development Environments to Production Realities

March 22, 2023

•

Bodo’s UI makes deploying, managing, and monitoring jobs running on notebooks or in production simple—enabling you to connect with terabytes of data and use job scheduling to turn your interactive notebooks into production jobs.

Bodo: Democratizing HPC for Data Analytics

September 2, 2020

•

Bodo is a novel analytics engine technology that empowers data scientists on all compute platforms (e.g. public or private cloud) to run Python workloads with the extreme performance and scalability of HPC without code rewrites.

Data Processing Engines Comparison Pt 2.

March 24, 2025

•

We will use the same workload and settings from the previous analysis and provide new implementations in both Daft and Polars. To recap, this workload summarizes monthly taxi and rideshare trips from New York City’s Taxi and Limousine Commission (TLC) and precipitation data from Central Park weather observations...

Creating ETL Pipelines With Bodo

July 7, 2022

•

In this post, we look at some of the considerations you may want to keep in mind when building an ETL pipeline similar to the data collection and exploration as seen...

Business Value of a Supercomputing Data Science Platform

November 10, 2020

•

According to Gartner, over 85% of data science projects fail. Bodo aims to solve this problem by eliminating key hurdles in the application development process.

Building PyDough: Why We Built a New Analytics Language

February 2, 2026

•

As analytics becomes more conversational, iterative, and collaborative, that gap matters more than ever. PyDough is our attempt to close it—not by generating better SQL, but by giving people a language that’s closer to how they actually think about their data. In this post, we walk through the specific limitations of SQL that led us to building PyDough, and show how PyDough approaches analytics from a different starting point.

Bodo’s Partnership with Snowflake

December 16, 2021

•

Today I’m thrilled to announce our partnership with Snowflake, accompanied by a strategic investment they have made in Bodo.

Bodo’s Approach to Open Platforms and Open Source

May 9, 2022

•

Bodo’s mission is to enable easy access to high-performance computing; to build a platform that makes working with petabyte-scale datasets as fast and straightforward as running pandas on small datasets using a laptop. We believe...

2022 at Bodo: A Year in Review

November 23, 2022

•

Bodo has accomplished so much this year, we wanted to take a moment to reflect on some of our highlights as 2022 comes to a close.

Bodo vs. Numba: Choosing the Right Tool for High-Performance Python

March 5, 2025

•

When it comes to efficient number crunching in Python, two powerful tools are Bodo and Numba. Both aim to speed up Python code using compilation techniques. While there is some overlap in the domains they support, there are notable performance differences, and each has unique areas of support that the other does not...

Bodo ♥️ PyIceberg: Scalable IO with the Simplicity of Python

February 26, 2025

•

Apache Iceberg is an open table format that provides simple, reliable, and scalable tables that are interoperable across various data tools such as compute engines. At Bodo, we believe that Iceberg is the future of data warehouse and data lake solutions due to its flexibility and open-source backing. However, using Iceberg at a large scale often requires Java-based tools which are difficult for developers to use and maintain.

Enterprise-Scale Data Engineering with Python: Evaluation of Bodo Derived from the TPCx-BB Q26 Benchmark

August 19, 2021

•

A large Fortune 10 enterprise evaluated Bodo for data engineering workloads in their new data infrastructure. They found that Bodo is much simpler to use and 10x faster than highly optimized Spark with the same cluster setup.

Bodo Plus Snowflake: Bringing Extreme Performance to Massive-Scale ETL Using Python

May 2, 2022

•

The Snowflake Data Cloud simplifies data management for data engineers at a near-unlimited scale, while the Bodo Platform brings extreme performance and scalability to large-scale Python data processing. Snowflake and Bodo have combined forces to give data teams...

Bodo Now Available on Saturn Cloud

June 1, 2022

•

Over the last few months, Bodo and SaturnCloud have worked together to provide a joint solution with Bodo software running within Saturn Cloud resources. Data scientists can now access...

Bodo Native Integration in PyIceberg 0.10: Bringing Scalability to PyIceberg with Pandas APIs

November 12, 2025

•

The PyIceberg 0.10 release introduces a new first-class integration: Table.to_bodo(). With this method, PyIceberg now plugs directly into Bodo’s high performance distributed DataFrame engine, all without leaving the Pandas API.

Bodo is Going Open Source – HPC Meets Python for data and AI

January 27, 2025

•

When we started Bodo, we had a clear mission: to make high-performance computing (HPC) as simple as possible. Along the way, we tackled some of the most challenging problems in parallel computing—achieving unprecedented performance and efficiency while maintaining ease of use. Today, we are proud to take that mission to the next level: The Bodo compute engine is now open source, bringing HPC to all data and AI workloads.

Bodo DataFrames vs PySpark and Dask on TPC-H Benchmarks

February 5, 2026

•

We compare Bodo DataFrames, Dask, and PySpark on a cluster of four Amazon EC2 instances (128 physical cores).

Bodo 2025.7: Turbo‑Charging DataFrames for Production Lakes

July 4, 2025

•

Bodo’s July release updates the best of the Bodo DataFrame library and adds new features—from the best way to interact with Iceberg, to making database-grade analytics now possible!

Bodo 2025.6 is Here: Supercharged Data Operations with Iceberg, Merge, and Sort

June 13, 2025

•

We are thrilled to announce the release of Bodo 2025.6, a major update focused on dramatically expanding the power and compatibility of our DataFrame library. This release brings some of the most requested and powerful data manipulation features to the Bodo dataframe libraryplatform, enabling you to do more with your data, faster than ever.

Benchmarking Bodo (Part II): Word Count

December 10, 2021

•

Previously, I’ve benchmarked Bodo using the popular example: The Monte Carlo approximation of Pi. In this post, I wanted to test how Bodo performs on another popular data analytics example benchmark: Word Count of Beer Review.

Benchmarking Bodo (Part I): Monte Carlo Pi

March 14, 2021

•

The Monte Carlo approximation of Pi happens to be a popular example as well as one of the first examples demonstrating the Spark RDD API. Trying out Bodo on this benchmark on the occasion of Pi Day seemed suitable, and that is what this blog post is about.

Accelerate Python with Bodo: Read Data From AWS S3

April 5, 2022

•

Amazon S3 is one of the most popular technologies that data engineers use to store data as a data lake. One of the typical applications is to read compressed parquet files as part of the extract process in an ETL (extract-transform-load) pipeline...

2025 Wrapped: Bodo’s Year in Review

December 18, 2025

•

2023 Review and Looking Ahead to 2024

February 13, 2024

•

2023 was a pivotal year for Bodo’s mission to bring High Performance Computing levels of efficiency, performance, and scalability to data workloads. From building a vectorized SQL engine to a new cloud service architecture, there are several milestones to celebrate. Let’s review some of the highlights of 2023 and look ahead to possibilities in 2024.

const next = await fetch("https://api.example.com/next-section");

_

Black and white grid pattern with black dots at the intersections, forming a repeating checkered design.

Get in touch

Have questions or want a personalized demo? Just give our team a shout—we’d love to chat!

const next = await fetch("https://api.example.com/next-section");

_

Black and white grid pattern with black dots at the intersections, forming a repeating checkered design.