Technology

Built for large-scale compute, not just small queries

Built from years of research by High Performance Computing (HPC) and compiler experts, Bodo is a revolutionary approach for efficient large-scale data processing. Data warehouses are traditionally designed for data storage and retrieval, making them relatively slow and expensive for complex data processing at large scale. The Bodo compute engine utilizes advanced compiler and HPC technologies for efficient parallel computing. Therefore, it is architecturally much faster and more efficient for data-heavy and compute-heavy workloads necessary for modern data engineering.

Solving the data warehouse compute problem with HPC efficiency

Bodo’s compiler optimization and parallel runtime system technologies bring HPC levels of performance and efficiency to large-scale data processing for the first time. Data warehouses focus on decades-old database techniques such as indexing, ensuring that a minimal amount of rows are scanned to match query filters that target small portions of the data. But modern queries that require heavy computation on large data need MPI parallelization and low-level code optimization techniques to run efficiently. The Bodo compute engine brings these optimization techniques to data engineering without any code change or tuning necessary.

BENCHMARKS

The Bodo Just-In-Time (JIT) compiler technology

The Bodo engine incorporates the first inferential compiler–it can infer the structure of the application to optimize and parallelize it automatically. This compiler is similar to an HPC expert re-writing code in a low-level paradigm, with the added bonuses of it being done transparently and in real time.

INPUT

Compiler Engine

ETL / Analytics Engine

Auto Parallelism

OUTPUT

Automatically:

Optimized

Parallelized

Vectorized

TOOLS

OPEN SOURCE LIBRARIES

HPC ARCHITECTURE

True parallel computing, not another distributed library

Bodo is the first compute engine to provide the full parallelism of SPMD (single program multiple data), a well-known parallel compute paradigm. In contrast, existing data platforms use distributed library backends to scale computation beyond a single CPU core, which are designed for web applications and not efficient parallel computation. By using SPMD, Bodo achieves maximum parallel efficiency and successfully avoids the bottlenecks and task overheads of distributed libraries.

An auto-parallelizing inferential compiler for native Python and SQL

Bodo's auto-parallelizing inferential compiler technology supports native Python as seamlessly as SQL. This allows the use of the two languages interchangeably without the need for complicated API layers like PySpark and hard-to-use database user defined functions (UDFs). Bodo's compiler parallelizes and optimizes Python code end-to-end into binary code without the interpreter overheads, combining Python’s expressive powers with HPC scaling and performance.