You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 29 Next »

Milvus Project Roadmap and Time Schedule



Time Note
2.22022.10.28Experiment
2.32022.12.30Stable
2.42023.4.30Experiment
2.52023.6.30Stable
3.0-RC2023.9.30Experiment
3.0-GA2023.10.30Stable


Roadmap features

2.2

  1. Refactor Segment Assignment in Query coord
  2. Support Disk ANN Index
  3. Support RBAC 
  4. Support BulkInsert Data
  5. Support Rate Limiter → Memory protection, Insertion/Search throughput
  6. Support collection level data TTL
  7. Primary/Backup coordinator
  8. Refine and fully tested pymilvus, java and go SDK.
  9. Release birdwatcher, milvus meta inspect and hacking tool
  10. Better Monitoring, collection level DML metrics supported

     

2.3

  1. Refactor On Disk Data format 
  2. Performance Improving on both memory/disk index
  3. Faster failure recovery spped.
  4. Resource group for querynode physical isolation
  5. Merge Index coordinator/ Data coordinator into 1 role
  6. Integrate with new vector search engine, Knowhere 2.0
  7. Support Range Search
  8. Refine Alias functionality
  9. Backup support
  10. Rolling Upgrade
  11. Better Data Partition
  12. Incremental data subscription
  13. Milvus benchmark tool
  14. Fully tested Restful API
  15. Intergated with Huggingface, OpenAI, Paddle paddle


2.4

  1. Support GPU index
  2. ARM Support
  3. Refactor Scalar search engine with DuckDB/Velox?
  4. Better Hybrid search speed
  5. Remove Datanode, all compaction/bulkload functionality moved into indexnode, streaming functionalities moved into log node.
  6. Better Query pagination
  7. Primary/Backup milvus cluster
  8. Improve multi memory replica availability
  9. Intergate with more inference generation system
  10. Tracing
  11. Support offline deployment


2.5 ‘

  1. Support ScaNN Index
  2. Support more datatype，map，list...
  3. Compaction based on vector classification and scalar value
  4. Better streaming data search performance
  5. Segment LRU load for memory index 
  6. Index parameter optimization tool


3.0 

  1. SQL Support 
  2. Optimizer for vector database
  3. Modify collection schema
  4. Support entity Update
  5. GPU index build
  6. Fully tested Cpp/Rust API
  7. New Milvus bootcamp

    



VersionFeatureOwnerStatusComment
2.1Supports string data typesxi-ge16,dragondriverin progresshttps://wiki.lfaidata.foundation/display/MIL/MEP+11+--+Support+String+DataType?src=contextnavpagetreemode
2.1Username/Passwordawekjiangin progressMEP 27 -- Support Basic Authentication
2.2Supports Scalar bitmap/inverted Index for string and numeric data types
pending
2.2Collection alias atomic switch
pending
2.2Supports alter collection
pending
2.2Support auto data paritition
pending
3.0Support  list,set,json data type
pending



DML

VersionFeatureOwnerStatusComment
2.1support query/search result iteration
delayed to 2.2
2.1Support auto-expirationWritticin progressPartly finished. TBD next step.
2.2Supports search by id
pendingTBD
2.2Vector range search
pendingTBD
2.2Update support 
pending
2.2Delete performance optimization
pending
3.0Support complicated scalar operation with vectorization(DuckDB, Data fusion, Gandiva)
pending
Long termMulti vector support
pending
Long termSupports upsert/primary key deduplication
pending



Features

VersionFeatureOwnerStatusComment
2.1Multi tenant support and access control/Support ACLKe Jiangin progressMEP 29 -- Support Role-Based Access Control
2.1Data compressionyah01done
2.2query pagination
pending
2.2Segment LRU load
pending
2.2Change data capture
pending
3.0UDF support
pending
3.0User specified reranking and preprocessing in container
pending
3.0Adopts incremental backup
pending
Long TermSupports static data encryption
pending



Performance/Cost

VersionFeatureOwnerStatusComment
2.1performance benchmarkwangtingdone
2.1performance optimization and monitoringczs007in progress
2.1Multi memory replicasunbyin progress
2.1Data bulkloadyhmodoneMEP 24 -- Support bulk load
2.2Supports GPU Index building and embedding retrieval
pending
2.2Load balance optimizationcongqi xiapending
2.2Supports ScaNN Indexwalmart labsin progress
3.0Adopts cost-based query optimization algorithm to improve hybrid search efficiency
pending
Long TermSupports on-disk vector indexing
pending
Long TermAutomatic index optimization
pending



Stability

VersionFeatureOwnerStatusComment
2.1Chaos testzhuwenxingdonebasically works
2.xPressure testyanliang567pending
2.1Supports segment in memory replicassunbyin progress
2.2Coordinator Primary/Backup support
in progressMEP30
2.2Flow control && back pressure supportjaimepending
3.0Master/Slave cluster
pending
3.0Query node resource isolation
pending



Ease Of Use

VersionFeatureOwnerStatusComment
2.1Embedded Milvus that runs on laptopssoothing-rainin progress
2.1windows supportmatrix jiin progress
2.1 arm supportyusheng main progress
2.2Multi datacenter deployment and multi-cloud integration
pendingON AWS
2.2Cluster auto scalingLoveEachDaypending
2.2Milvus deploy with ansiblezwd1208in progress
Long TermOffline vector computation
pending



SDK

VersionFeatureOwnerStatusComment
2.1Supports Restful APIshaorenfsa, Anoshin progresscoding 
2.1Supports C++ SDKyhmo,matrixji,ArkaprabhaChakrabortyin progressalmost done
3.0Milvus Query Language(SQL like, with ANN,Range support)
pending
Long TermSupport GraphQL
pending
Long TermSupport Rust SDK
pending



Integration

VersionFeatureOwnerStatusComment
2.1Integrates KafkaJaimein progress
2.2Support meta storage interface (Meta in etcd, zookeeper and other user implemented logic)Jaimepending
2.2Refine milvus bootcamp with Towhee and Hugging facehou jiepending
2.2Data stored over local/distributed filesystems
pending
Long TermIntegrates distributed KV stores such as HBase/TiKV/FoundationDB
pending
  • No labels