A Brief Overview of MySQL, HeatWave & Lakehouse from a Technology Leader's Perspective
Nipun Agarwal having completed his MS degree in Computer Science from University of Virginia, Nipun is a seasoned technology professional with proven exce- llence in the areas of data processing, distributed sys- tems, machine learning, cloud computing and more. He has been associated with Oracle for close to three de- cades now and has been at the forefront of the com- pany’s technology journey so far.
From MySQL to HeatWave to Lakehouse, the journey has been remarkable. Tell us about this journey so far
MySQL is the world’s most popular open-source database, powering many of the most accessed applications such as Facebook, Booking.com, and Shopify. While it is renowned for its high performance for transactional applications, it’s comparatively slow at running analytics queries. HeatWave is an integrated in-memory, high-performance query accelerator that enables real-time analytics on data stored in MySQL. Additionally, it eliminates the need for a separate analytics database and the complex, time-consuming & costly ETL processes. Secondly, many customers want to apply machine learning on their data inside a database without having to move the data to another service. We introduced HeatWave AutoML which provides in-database support for machine learning and fully automates the machine learning training process which reduces the dependency on data scientists.
We then released MySQL HeatWave Lakehouse, letting users query as much as half a petabyte of data in object storage—in a variety of file formats such as CSV, Parquet, Avro, and files exported from other databases. The query processing is done entirely in the HeatWave engine, enabling customers to take advantage of HeatWave for non-MySQL workloads in addition to MySQL-compatible workloads. We also introduced Lakehouse specific automation in MySQL Autopilot that makes it easier for users and expedites data loading. With the addition of HeatWave Lakehouse, MySQL HeatWave provides one cloud database service for transaction processing, real-time analytics across data warehouses & data lakes, and machine learning—without ETL across cloud services.
With cloud becoming mainstream, how do you see the market of cloud database growing around the world?
We’re seeing an ever-stronger interest not only in running workloads on the cloud, but also in running workloads in different clouds. Thus, in addition to Oracle Cloud Infrastructure (OCI), we’ve made MySQL HeatWave available natively on AWS and Microsoft Azure, wherein customers can replace five services with one, reducing complexity and improving price-performance for analytics. The interest in using open-source technology in India is particularly strong which plays into our strength with MySQL. Since the desire to leverage machine learning is high in India, HeatWave AutoML includes everything users need to build, train, deploy, and explain machine learning models within MySQL HeatWave at no additional cost and without the need to hire data scientists. Also, many customers are looking to migrate out of AWS due to the high cost, exorbitant egress fees, and having to use multiple AWS services for their data processing and analytics needs.
How do you address concerns related to security with MySQL HeatWave Lakehouse?
With MySQL HeatWave, not only do customers eliminate the cost and complexity of having separate analytics database, lakehouse, ML, and ETL across cloud services, but also avoid the latency and security risks of data movement between data stores. Advanced security features such as asymmetric encryption with key generation & digital signatures, data masking & deidentification, and firewall monitoring help customers to protect data throughout its lifecycle and support compliance with regulatory requirements.
What is the future roadmap and opportunities with MySQL HeatWave?
We keep innovating with new capabilities for MySQL Autopilot and HeatWave AutoML, with JavaScript support in MySQL HeatWave, or with query acceleration for JSON documents. A key innovation is the support for generative AI with MySQL HeatWave vector store. Currently in private preview, the vector store will enable customers to leverage the power of large language models (LLMs) combined with their proprietary data to get answers that are more accurate and relevant than using models trained on only public data. With generative AI and vector store capabilities, customers will interact with MySQL HeatWave in natural language and efficiently search documents in various file formats in HeatWave Lakehouse.
From MySQL to HeatWave to Lakehouse, the journey has been remarkable. Tell us about this journey so far
MySQL is the world’s most popular open-source database, powering many of the most accessed applications such as Facebook, Booking.com, and Shopify. While it is renowned for its high performance for transactional applications, it’s comparatively slow at running analytics queries. HeatWave is an integrated in-memory, high-performance query accelerator that enables real-time analytics on data stored in MySQL. Additionally, it eliminates the need for a separate analytics database and the complex, time-consuming & costly ETL processes. Secondly, many customers want to apply machine learning on their data inside a database without having to move the data to another service. We introduced HeatWave AutoML which provides in-database support for machine learning and fully automates the machine learning training process which reduces the dependency on data scientists.
We’re seeing an ever-stronger interest not only in running workloads on the cloud, but also in running workloads in different clouds
We then released MySQL HeatWave Lakehouse, letting users query as much as half a petabyte of data in object storage—in a variety of file formats such as CSV, Parquet, Avro, and files exported from other databases. The query processing is done entirely in the HeatWave engine, enabling customers to take advantage of HeatWave for non-MySQL workloads in addition to MySQL-compatible workloads. We also introduced Lakehouse specific automation in MySQL Autopilot that makes it easier for users and expedites data loading. With the addition of HeatWave Lakehouse, MySQL HeatWave provides one cloud database service for transaction processing, real-time analytics across data warehouses & data lakes, and machine learning—without ETL across cloud services.
With cloud becoming mainstream, how do you see the market of cloud database growing around the world?
We’re seeing an ever-stronger interest not only in running workloads on the cloud, but also in running workloads in different clouds. Thus, in addition to Oracle Cloud Infrastructure (OCI), we’ve made MySQL HeatWave available natively on AWS and Microsoft Azure, wherein customers can replace five services with one, reducing complexity and improving price-performance for analytics. The interest in using open-source technology in India is particularly strong which plays into our strength with MySQL. Since the desire to leverage machine learning is high in India, HeatWave AutoML includes everything users need to build, train, deploy, and explain machine learning models within MySQL HeatWave at no additional cost and without the need to hire data scientists. Also, many customers are looking to migrate out of AWS due to the high cost, exorbitant egress fees, and having to use multiple AWS services for their data processing and analytics needs.
How do you address concerns related to security with MySQL HeatWave Lakehouse?
With MySQL HeatWave, not only do customers eliminate the cost and complexity of having separate analytics database, lakehouse, ML, and ETL across cloud services, but also avoid the latency and security risks of data movement between data stores. Advanced security features such as asymmetric encryption with key generation & digital signatures, data masking & deidentification, and firewall monitoring help customers to protect data throughout its lifecycle and support compliance with regulatory requirements.
What is the future roadmap and opportunities with MySQL HeatWave?
We keep innovating with new capabilities for MySQL Autopilot and HeatWave AutoML, with JavaScript support in MySQL HeatWave, or with query acceleration for JSON documents. A key innovation is the support for generative AI with MySQL HeatWave vector store. Currently in private preview, the vector store will enable customers to leverage the power of large language models (LLMs) combined with their proprietary data to get answers that are more accurate and relevant than using models trained on only public data. With generative AI and vector store capabilities, customers will interact with MySQL HeatWave in natural language and efficiently search documents in various file formats in HeatWave Lakehouse.
