Versions Compared

Old Version 32

changes.mady.by.user Xiaofan

Saved on

compared with

New Version 33

changes.mady.by.user Xiaofan

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Milvus Project Roadmap and Time Schedule


Experiment5Stable-RC17Stable

Time Note
2.22022.10.28Stable2.2.33.02023.28.823Stable2.32023.2.28
2.42023.12.30Under development
3.020232024.3.30Experiment3.0-GA2023.8.30

Under

Design

detailed features

Feature plans


Roadmap features

2.2

  1. Refactor Segment Assignment in Query coord
  2. Support Disk ANN Index
  3. Support RBAC 
  4. Support BulkInsert Data
  5. Support Rate Limiter → Memory protection, Insertion/Search throughput
  6. Support collection level data TTL
  7. Primary/Backup coordinator
  8. Refine and fully tested pymilvus, java and go SDK.
  9. Release birdwatcher, milvus meta inspect and hacking tool
  10. Better Monitoring, collection level DML metrics supported

2.2.3

       1. Rolling Upgrade

       2. Resource group  for querynode physical isolation

       3.  BulkInsert any sized data.

       4. Performance optimization

        5. Dynamic config change

2.3

  1. Refactor On Disk Data format 
  2. Performance Improving on both memory/disk index
  3. Faster failure recovery speed.
  4. Merge Index coordinator/ Data coordinator into 1 role
  5. Integrate with new vector search engine, Knowhere 2.0
  6. Support Range Search
  7. Refine Alias functionality
  8. Backup support
  9. Dynamic Partition load/release
  10. Incremental data subscription
  11. Milvus benchmark tool
  12. Fully tested Restful API
  13. SQL support - count
  14. Intergated with Huggingface, OpenAI, Paddle paddle

2.4

  1. Support GPU index
  2. Support ScaNN Index
  3. Refactor Scalar Execution engine with Velox
  4. Better Hybrid search speed
  5. Better streaming data search performance
  6. Better Query pagination
  7. Improve multi memory replica availability
  8. Integrate with more inference generation system
  9. Tracing
  10. Support offline deployment
  11. SQL support - search,insert, delete, query
  12. Segment mmap load for memory index 
  13. Index parameter optimization tool
  14. Compaction optimization

3.0 

  1. Full SQL Support 
  2. Modify collection schema dynamically - add column, remove column
  3. Support more datatype，map，list...
  4. Remove Datanode, all compaction/bulkload functionality moved into indexnode, streaming functionalities moved into log node
  5. Primary/Backup milvus cluster
  6. Support entity Update
  7. GPU index build
  8. Fully tested Cpp/Rust API
  9. New Milvus bootcamp

    

...

Category

Milvus 2.3.x (Achieved in recent releases)

Milvus 2.4.0(End of CY23 release)

Roadmap(3.0)

Scalability and Performance
World-class scalability and performance

Growing Index
Index for streaming-in data

ScaNN Index
20% faster than HNSW

Bulk Insert Optimazation
Easier import for larger data

Scalar Fields Index
Indexing for specific scalar fields

Holds More Collections/Partitions
Support 10000+ collections in small size cluster

Inverted Index for JSON

GPU Acceleration

Full test for 10 billion+ vectors

Ease of Use
Provide flexible usabilities and maintainance

Cluster Rolling Update
Update affected window <30s

Delete by Expression
Easy to delete useless data

Upsert
Update embeddings for ambiguous scenarios

Support More Datatypes
Datetime and various vector types(fp16, bf16)

Add/Delete Collection Fields

Support SQL Syntax

Upgrade SDK

Enterprise Offering
Features for production-ready

RBAC
Role based access control

Partition Key
Performance enhancement for large datasets

Accesslog Enhancement
Detailed info recorded for audit and tracing

Compaction Optimization
Improve system stability

CDC for Online/Cross-Cloud Migration

AI-augment Capability Designed features for AI applications

Dynamic Schema
Provide flexibility by schema-free

Group By for Grouping Requests
Aggregate split embeddings for one set

Multi-Vectors(beta)
Multiplex recall and reranking framework

Sparse Vector(beta)
Local feature extraction or keyword search

Scenario-oriented Vector Search Patterns

Spark Connector for large scale data processing

Multi-Vectors GA

Sparse Vector GA

...

VersionFeatureOwnerStatusComment2.1Integrates KafkaJaimein progress2.2Support meta storage interface (Meta in etcd, zookeeper and other user implemented logic)Jaimepending2.2Refine milvus bootcamp with Towhee and Hugging facehou jiepending2.2Data stored over local/distributed filesystemspendingLong TermIntegrates distributed KV stores such as HBase/TiKV/FoundationDBpending