LOGIQ.AI Announces LogFlow Observability Data Pipeline as a Service (DPaaS) 

SAN FRANCISCO, Dec. 2, 2021 -- LOGIQ.AI, a leading provider of data management, analytics, and observability solutions for IT, DevOps, and SecOps teams, has today launched LogFlow, an Observability Data Pipeline as a Service (DPaaS). LogFlow is an innovative paradigm in machine data management that enables enterprises to unleash the true potential of machine data by connecting it to SMEs on-demand.

Global data creation and replication is growing at a CAGR of 23% (IDC), and 70% of organizations will shift focus to provide more context for data analytics (Gartner). New tooling is needed to optimize data volume and improve data quality. LogFlow solves these data challenges at both the core and the edge.

Greg O'Reilly, Observability Consultant at Visibility Platforms, said, "LogFlow enables our customers to take a whole new approach to observability data; one that helps regain control and unblock vendor or cost limitation. We're opening up discussions between ITOps and Security teams for the first time with a unified solution that keeps data secure, compliant, manageable, and readily available to those who need it on the front lines."

The Block and Drop problem

Contemporary data pipelines suffer from ingest-egress mismatches. "Enterprises have unfortunately been sold "block" and "drop" as intelligent features to counter back pressure and upstream unavailability in data pipelines", said Ranjan Parthasarathy, Co-founder, CEO of LOGIQ.AI. "Block and drop is data loss in disguise. Imagine losing a vital signature in your log stream that points to impending ransomware starting to spread. Don't introduce new business risks by buying into block and drop."

InstaStore

LogFlow eliminates block and drop by storing 100% of streaming data in InstaStore, a storage innovation that enables object storage as primary storage. In InstaStore, data is fully indexed and searchable in real-time. LogFlow also stores its indexes in InstaStore, giving a genuinely scalable platform with cleanly decoupled storage and compute. LogFlow ingests data even when upstream targets are down. Due to its indexing capabilities, it provides fine-grained data replays.

"InstaStore introduces a new paradigm for data agility that eliminates data loss and the need for storage tiering and data rehydration. Organizations can now unlock productivity, cost reduction, and compliance like never before," said Jay Swamidass, Head of Sales - APAC and EMEA at LOGIQ.AI.

Accelerating machine intelligence

LogFlow's native support for open standards makes it easy to collect machine data from any source. Similar to network flows, LogFlow manages data with its flow-level routing table. 

"LogFlow filters unwanted data and detects security events in-flight. Users can route streams, control EPS and run fine grained data replays.", said Tito George, Co-founder of LOGIQ.AI. "InstaStore's indexing and columnar data layouts enable faster querying, unlike archive formats like gzip."

Data routing isn't the real problem

Open-source tools like Fluent Bit and Logstash can already route data between various sources and target systems and allow routing raw archives to object stores. The complex problems to solve are:

• Controlling data volume and sprawl.
• Preventing data loss.
• Ensuring data reusability with fine-grained control.
• Business continuity during upstream failures.

Theodore Caroll, Head of Sales – Americas at LOGIQ.AI said, "There's no technical reason to accept anything less than 100% data availability. Your data is your only true fortress in responding to threats and adverse business events. Businesses need a system like LogFlow that ensures full data replay is continuously and infinitely available."

LogFlow's log management and SIEM capabilities provide built-in insurance against upstream failures. Businesses can run it in parallel with their existing systems. If upstream systems become unavailable, LogFlow can continue to provide crucial forensics.

Built for the enterprise

LogFlow's built-in "Rule Packs" have over 2000 rules that filter, tag, extract, and rewrite data for popular customer environments and workloads. LogFlow's SIEM Rule Packs also allow security event detection and tagging. 

LOGIQ.AI's LogFlow brings complete control over observability data pipelines and delivers high-value, high-quality data to teams that need it in real-time, all the time. For the first time, organizations can fully control data collection, consolidation, retention, manipulation, and upstream data flow management.               

About LOGIQ.AI

LOGIQ is the creator of the LOGIQ platform, a converged data analysis software that brings ingest, indexing, search, SQL, and streaming capabilities backed by any S3 compatible object-store. LOGIQ Log Insights is built on this converged architecture bringing scalability and simplicity to your IT operations analytics projects. The converged architecture enables rapid integration with AI platforms for gaining valuable business insights from your data. For further information visit: https://logiq.ai.

Source: LOGIQ