Solus

A distributed de-duper with TTL capability based on Bloom Filters

Overview

Solus is a high-performance, TTL-based deduplication library designed for (hundred+) billion scale operations. It uses probabilistic data structures (Bloom filters) with sharding to provide memory-efficient duplicate detection with configurable accuracy.

Features

• Massive Scale — Handle billions of unique entities using sharded Bloom filters
• TTL Support — Automatic expiration of entries based on time-to-live
• Pluggable Storage Backends — Aerospike and HBase out of the box
• Multi-Datacenter Support — DC (datacenter-local) and XDC (cross-datacenter) deduplication levels
• Configurable Accuracy — Tune false positive rates by adjusting hash functions, shards, and bits per shard
• Batch Operations — Efficient bulk add and check operations
• Atomic Operations — addIfAbsent for atomic check-and-add patterns

Documentation

Detailed documentation is available at https://phonepe.github.io/Solus

• Getting Started
• Usage Guide
• Deduplication Semantics
• Aerospike Backend
• HBase Backend

Getting Started

Maven

<dependency>
    <groupId>com.phonepe</groupId>
    <artifactId>solus</artifactId>
    <version>${solus.version}</version>
</dependency>

Gradle

implementation 'com.phonepe:solus:${solus.version}'

Requirements

• Java 17 or higher
• One of the supported storage backends:
  • Aerospike 6.x+
  • HBase 2.x+

Contributing

We welcome contributions! Please see our Contributing Guidelines for details.

License

Solus is licensed under the Apache License 2.0.

Copyright 2026 PhonePe India Pvt. Ltd.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.