Built from years of research by High Performance Computing (HPC) and compiler experts, Bodo is a revolutionary approach for efficient large-scale data processing. Data warehouses are traditionally designed for data storage and retrieval, making them relatively slow and expensive for complex data processing at a large scale. The Bodo Compute Engine utilizes advanced compiler and HPC technologies for efficient parallel computing. Therefore, it is architecturally much faster and more efficient for data-heavy and compute-heavy workloads necessary for modern data engineering.
Bodo’s compiler optimization and parallel runtime system technologies bring HPC levels of performance and efficiency to large-scale data processing for the first time. Data warehouses focus on decades-old database techniques such as indexing—ensuring that a minimal amount of rows are scanned to match query filters that target small portions of the data. However, modern queries that require heavy computation on large data need MPI parallelization and low-level code optimization techniques to run efficiently. The Bodo Compute Engine brings these optimization techniques to data engineering without any code change or tuning necessary.
Bodo is the first compute engine to provide the full parallelism of SPMD (single program multiple data), a well-known parallel compute paradigm. In contrast, existing data platforms use distributed library backends, which are designed for web applications and not efficient parallel computation, to scale computation beyond a single CPU core. By using SPMD, Bodo achieves maximum parallel efficiency and successfully avoids the bottlenecks and task overheads of distributed libraries.
Bodo's auto-parallelizing inferential compiler technology supports native Python as seamlessly as SQL. This allows the use of the two languages interchangeably without the need for complicated API layers like PySpark and hard-to-use database user-defined functions (UDFs). Bodo's compiler parallelizes and optimizes Python code end-to-end into binary code without the interpreter overheads—combining Python’s expressive powers with HPC scaling and performance.
