Azure Information Explorer: Log and telemetry analytics benchmark | Azure Weblog and Updates


Azure Information Explorer (ADX), a element of Azure Synapse Analytics, is a extremely scalable analytics service optimized for structured, semi-structured, and unstructured information. It offers customers with an interactive question expertise that unlocks insights from the ocean of ever-growing log and telemetry information. It’s the excellent service to investigate excessive volumes of contemporary and historic information within the cloud by utilizing SQL or the Kusto Question Language (KQL), a robust and user-friendly question language.

Azure Information Explorer is a key enabler for Microsoft’s personal digital transformation. Just about all Microsoft services use ADX in a method or one other; this consists of troubleshooting, prognosis, monitoring, machine studying, and as an information platform for Azure providers similar to Azure Monitor, PlayFab, Sentinel, Microsoft 365 Defender, and plenty of others. Microsoft’s prospects and companions are utilizing ADX for a big number of situations from fleet administration, manufacturing, safety analytics options, bundle monitoring and logistics, IoT system monitoring, monetary transaction monitoring, and plenty of different situations. During the last years, the service has seen phenomenal development and is now operating on hundreds of thousands of Azure digital machine cores.

Final 12 months, the third technology of the Kusto engine (EngineV3) was launched and is at present supplied as a clear, in-place improve to all customers not already utilizing the newest model. The brand new engine contains a fully new implementation of the storage, cache, and question execution layers. Because of this, efficiency has doubled or extra in lots of mission-critical workloads.

Superior efficiency and cost-efficiency with Azure Information Explorer

To raised assist our customers assess the efficiency of the brand new engine and price benefits of ADX, we appeared for an present telemetry and logs benchmark that has the workload traits widespread to what we see with our customers:

  1. Telemetry tables that comprise structured, semi-structured, and unstructured information sorts.
  2. Information within the a whole lot of billions to check large scale.
  3. Queries that symbolize widespread diagnostic and monitoring situations.

As we didn’t discover an present benchmark to satisfy these wants, we collaborated with and sponsored GigaOm to create and run one. The brand new logs and telemetry benchmark is publicly out there on this GitHub repo. This repository features a information generator to generate datasets of 1GB, 1TB, and 100TB, in addition to a set of 19 queries and a take a look at driver to execute the benchmark.

The outcomes, now out there within the GigaOm report, present that Azure Information Explorer offers superior efficiency at a considerably decrease value in each single and high-concurrency situations. For instance, the next chart taken from the report shows the outcomes of executing the benchmark whereas simulating 50 concurrent customers:  

Study extra

For additional insights, we extremely suggest studying the full report. And don’t simply take our phrase for it. Use the Azure Information Explorer free providing to load your information and analyze it at excessive velocity and unmatched productiveness.

Take a look at our documentation to discover out extra about Azure Information Explorer and be taught extra about Azure Synapse Analytics. For deeper technical data, take a look at the brand new ebook Scalable Information Analytics with Azure Information Explorer by Jason Myerscough.