Keeping Elasticsearch data secure from attacks and exposure

In the last few years enterprises have seen an unprecedented amount of data lost from vulnerable Elasticsearch clusters. Tens of billions of records of highly sensitive data across financial services, healthcare, high-tech, retail, and hospitality segments have been exposed or stolen. A simple web search for Elasticsearch data breaches yields an unsettlingly large number of results going back a few years to as recently as a few hours ago.

This article breaks down the fundamental vulnerability with enterprise search platforms and describes how Titaniam provides a seamless plugin that delivers NIST FIPS 140-2 validated searchable encryption-in-use to eliminate this critical gap.

Though extremely powerful for search, Elasticsearch is vulnerable from a data security standpoint

Enterprise search platforms such as Elasticsearch take in enormous amounts of data and index them in many different ways to enable high-performance search and analytics. For this type of indexing to work towards full featured search, such as prefixes, suffixes, wildcards, ranges, etc. live data (i.e. that is no longer at-rest in underlying file systems) inside Elasticsearch that cannot be encrypted. If encryption is applied to Elasticsearch indexes, they will no longer be able to support search. With the primary purpose of the platform being search, this presents an unfortunate paradox.

In the last few years this tradeoff has been thoroughly exploited by ransomware actors who search for credentials into data rich Elasticsearch environments or crawl the web for misconfigured or inadvertently exposed clusters. Once inside they steal massive amounts of data and proceed to extort victims. In addition to external cyberattacks, Elasticsearch platforms are also vulnerable to insider attacks.

While the underlying platforms are able to encrypt data at-rest (in the file system while not in use) and also encrypt in transit as data moves between nodes (internode TLS), data still remains highly vulnerable inside the indexes themselves. This is the typical breach point.

The Importance of Encryption

Encryption is one of the best data security controls ever created. Since its invention in the late 1970s and with all the improvements since then, encryption is the one mechanism that can stand up to ransomware and other cyber attackers by making stolen data unreadable.

Titaniam utilizes encryption to secure data inside Elasticsearch by extending its umbrella of protection beyond data-at-rest and data-in-transit to include data-in-use. With this new defense in place, sensitive data inside Titaniam enables enterprise search platforms to become immune to compromise at the hands of external attackers, malicious insiders, or accidental exposure. Any data queried or otherwise exfiltrated retains encryption and cannot be used as leverage against the enterprise even if it is accessed using admin privileges.

Titaniam Plugin for Elasticsearch

Titaniam keeps data always encrypted while retaining search

Titaniam offers high performance real-time encryption-in-use for enterprise search platforms such as Elasticsearch. Titaniam’s protection stays in place even as data is actively indexed and searched by the platform.

The plugin enables the construction of the encrypted search index and also intercepts and transforms all queries against protected data. Both the datastore as well as any applications built on top of it do not need to be aware of the plugin’s existence as all operations are performed transparently and using native platform capabilities.

Titaniam releases data in multiple privacy preserving formats

In addition to retaining encryption within the platform and indices, Titaniam also supports traditional and format preserving encryption, tokenization, masking and redaction of sensitive data when it is released from the platform. The Titaniam plugin is a completely transparent mechanism and sensitive data can be protected without losing data usability, searchability, and without modifying either the Elasticsearch platform or applications built on top of it.

Titaniam supports regulatory compliance and audit

By utilizing NIST FIPS 140-2 validated encryption for data in all states (including data while it is actively being used) Titaniam helps protected Elasticsearch Search clusters support FEDRAMP, GDPR, CCPA, ITAR and Data Residency compliance. Titaniam is also an extremely efficient way to support least privilege access to data inside Elasticsearch and applications built on top of it.

Titaniam includes rich BYOK (bring your own key), HYOK (hold your own key) functionality and granular protection

 Titaniam offers rich BYOK (Bring your own key) and HYOK (Hold your own key) capabilities for all data secured by Titaniam inside a Titaniam protected Elasticsearch deployment. Titaniam protection is offered at all levels including index and field level protection. Titaniam protected Elasticsearch clusters never see clear text sensitive data as it is already encrypted by the time it reaches the index to be persisted. This makes them highly resistant to common attack patterns, data exfiltration, extortion, and insider attacks. This is further described in a separate section below.

Titaniam provides post attack visibility and evidence of encryption for regulatory compliance and audit

All Titaniam algorithms are NIST CAVP certified and our underlying engine is NIST FIPS 140-2 validated. This allows our customers to enjoy immediate compliance, efficiently apply data privacy controls across the enterprise, and most importantly, it allows Titaniam to provide granular field-level evidence of encryption in the event of an attack. CISOs can rely on Titaniam for visibility on what attackers observed, accessed, and exfiltrated and reporting that can be provided to auditors, regulators, and boards of directors.

Key Capabilities

• Encrypt your indexes with FIPS 140-2 encryption
• Perform full-featured search and analytics without any data decryption including in memory or in the reverse index
• Release data in nine secure and private formats including traditional and format preserving encrypted, tokenized, whole or partially masked, redacted, hashed, anonymized, or valet (where third parties can query the data but not decrypt it)
• Protects query parameters
• Grant clear text access on the basis of least privilege
• Deploy BYOK/HYOK at any granularity
• Become immune to breach, ransomware, double and triple extortion, insider attacks and misconfigurations
• Immediately support with data protection requirements for FEDRAMP, HIPAA, PCI, GDPR, CCPA, and other major regulations
• Deploy on all clouds, hybrid, or on-prem
• Supported on Elasticsearch and OpenSearch

How does it work?

For Secure Indexing and Search

The Titaniam Plugin for Elasticsearch enables sensitive data to be indexed and searched while always keeping the data in FIPS 140-2 validated encryption format:

• All sensitive data is preprocessed and encrypted prior to being indexed.
• Queries are intercepted and reformulated to execute in encrypted space without any data decryption whatsoever. Titaniam supports numerous types of queries including term, prefix, wildcard, match, match-phrase, match-phrase-prefix, range, term (CIDR) etc.
• By default, query results are released in encrypted form. Here is an example of query results:

• Titaniam plugin does not have a significant impact on ingest and search performance. The overhead is typically up to about 10% when ingesting data and 2-3% while searching.

This means that even if attackers find their way to your Elasticsearch deployment, the data they could exfiltrate would be encrypted and unusable.

For Secure Data Release

In order for a legitimate user to get clear text out of Titaniam protected Elasticsearch there are several controlled release processes including direct allow listing as well as controlled release via pre-integrated Titaniam proxy or Titaniam translation service. All release configurations are defined at the granular field-level, and different fieldscans be set up to behave differently.

View Titaniam’s Elasticsearch plugin in action by booking a demo here.

Digging deeper into customer-controlled keys for Elasticsearch (BYOK/HYOK)

The Business Problem

SaaS providers are often asked two basic data security questions:

1. Can you tell us how you intend to protect our data?
2. Can your devops staff see our data?

SaaS providers can point to the encryption built into the storage layer (AWS S3’s SSE-S3 or EBS volume encryption). These encryption solutions protect the data from being viewed by AWS datacenter employees but offer no security or privacy from the SaaS providers own devops team.

SaaS prospects know this very well, especially those in regulated industries such as banking, finance, healthcare. With ransomware attacks and data breaches on the rise, they demand more. This drives SaaS companies to either ship on-prem software or stand up dedicated instances. Neither is desirable from a SaaS standpoint.

Ideally SaaS companies should be able to respond to data protection queries with statements such as:

1. “We encrypt your data with the highest data protection standards (NIST FIPS 140-2).”
2. “Our devops team cannot see your data in clear text.”
3. “As our premium enterprise customer, you can supply and control the encryption keys. You can revoke them whenever you want.” aka BYOK – Bring Your Own Key or HYOK – Hold Your Own Key.

However, implementing BYOK/HYOK on Elasticsearch involves a massive engineering effort and an excellent understanding of specialized encryption.

Titaniam Solution

Titaniam can help.

Titaniam is the preeminent provider of encryption solutions protecting big data stores such as AWS S3 and search engines such as AWS OpenSearch. All Titaniam offerings come with consistently implemented BYOK/HYOK capabilities giving SaaS companies a simple and seamless key management experience.

Companies can roll out Titaniam and secure their SaaS backend in a matter of weeks with minimal engineering effort allowing SaaS engineering teams to focus on what they do the best – building and running their platform.

BYOK/HYOK in Titaniam for Elasticsearch is built further upon the existing strengths of searchable encryption. It is built using two simple premises – index specific encryption keys and ability to read those individual keys from any location.

As a SaaS operator it requires one thing – providers must maintain a separate search index (or a set of indices) for each customer. This enables independent upgrades, encryption key rotation, etc. Titaniam plugin includes a key registry to manage encryption key locations corresponding to customer specific indexes. Titaniam reads the encryption keys directly from customers keystores and supports common key stores such as AWS Secrets Manager, Hashicorp Vault etc. Adding a keystore integration is easy and we are continuously adding these to the solution.

Steps to enabling data protection with BYOK/HYOK for your Elasticsearch based application.

1. Install Titaniam Plugin for Elasticsearch.
2. Choose which fields to protect. In a typical Elasticsearch index, not all documents need to be secured and searched. You choose specific fields to secure.
3. Test out the front-end application and ensure it works seamlessly.
4. Create a key registry – specify which index should use which key and provide Titaniam with the credentials (in encrypted form) to access the corresponding keystores.
5. Start loading data and querying it.

Conclusion: No more tradeoffs between search and security

Titaniam enables Elasticsearch to retain its enormous power while making it the most secure search platform in the industry.

To book a demo of Titaniam’s Elasticsearch plugin, please schedule a demo today.

Schedule a Demo >