*: add detailed intro to the TiDB X architecture #22245
Conversation
ti-chi-bot
bot
added
the
do-not-merge/work-in-progress
ti-chi-bot
bot
added
contribution
|
Hi @dbsid. Thanks for your PR. I'm waiting for a pingcap member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
ti-chi-bot
bot
added
the
missing-translation-status
Summary of ChangesHello @dbsid, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces detailed documentation for the new TiDB X architecture, outlining its fundamental shift towards a cloud-native, 'Share-Everything' design. It addresses limitations of the classic TiDB architecture by implementing innovations such as object storage as the single source of truth, a 'Compute and Compute' separation for workload isolation, and an LSM Forest storage engine, all aimed at enhancing scalability, stability, cost-effectiveness, and performance predictability. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
ti-chi-bot
bot
added
the
size/L
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a new documentation page for the TiDB X architecture. The content provides a good high-level overview of the new architecture, its motivations, and key innovations. My review focuses on improving clarity, correcting some typos and grammatical errors, and ensuring consistency in terminology, as per the repository's style guide. I've provided several suggestions to enhance readability and technical accuracy.
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Review CommentsTerminology Suggestion: Share-Everything → Shared-StorageThe term Share-Everything is imprecise and creates an awkward comparison (as @likidu noted - comparing a distributed database to an SOA is not apples-to-apples). Recommendation: Consider using Shared-Storage instead:
Alternative framing for the intro:
Structural Issues
Content Issues
PR DescriptionPlease fill in the required What is changed, added or deleted? section before moving out of draft. Overall the content is well-structured and provides good technical depth. The diagrams help illustrate the architectural differences clearly. |
ti-chi-bot
bot
removed
the
missing-translation-status
|
/ok-to-test |
|
@dbsid Please take a look at my suggestions. Thanks! |
Co-authored-by: Lilian Lee <lilin@pingcap.com>
There was a problem hiding this comment.
Note: Use sentence case capitalization for diagrams. I’ve highlighted what need to be updated in bold.
| Original | Suggested | Note |
|---|---|---|
| Classic (Share-Nothing) | Classic TiDB (shared-nothing) | |
| Raft Log | Raft log | In both left and right sides |
| Cloud-Native | cloud-native | |
| Shared-Storage | shared-storage | |
| RF Engine | RF engine | |
| WAL Chunk | WAL chunk | |
| Elastic Compute Pool TiKV Workers | Elastic compute pool TiKV workers | |
| Compact | Compaction | Use a consistent noun style |
| Load data | Data loading | Use a consistent noun style |
| LSM-Tree Engine | LSM-tree engine |
Ref: https://en.wikipedia.org/wiki/Shared-nothing_architecture
There was a problem hiding this comment.
Note: Use sentence case capitalization for diagrams. I’ve highlighted what need to be updated in bold.
| Original | Suggested | Note |
|---|---|---|
| Classic (Single LSM-tree) | Classic TiDB (single LSM-tree) | |
| Raft Log | Raft log | In both left and right sides |
| LSM-Forest | LSM-forest | |
| LSM-Tree Engine | LSM-tree engine |
Ref: https://en.wikipedia.org/wiki/Log-structured_merge-tree
There was a problem hiding this comment.
Note: Use sentence case capitalization for diagrams. I’ve highlighted what need to be updated in bold.
| Original | Suggested | Note |
|---|---|---|
| Isolated SQL Layer | Isolated SQL layer | |
| Shared Cache Layer | Shared cache layer | |
| Object Storage | Shared storage layer | Keep wording consistent with other layers |
| Shared Services | Shared services layer | |
| Row Engine shared resource | Row engine shared resource | |
| Columnar Engine shared resource | Columnar engine shared resource | |
| Shared Storage(object storage) | Object storage | Omit "shared" since it's used in the left |
| Analyze | Statistics collection | To make the operation clear and easy to understand |
Co-authored-by: Lilian Lee <lilin@pingcap.com>
Co-authored-by: Lilian Lee <lilin@pingcap.com>
|
|
||
| # TiDB X Architecture | ||
|
|
||
| TiDB X is a new distributed SQL architecture that makes cloud-native object storage the backbone of TiDB. This architecture enables elastic scalability, predictable performance, and optimized total cost of ownership (TCO) for AI-era workloads. |
There was a problem hiding this comment.
| TiDB X is a new distributed SQL architecture that makes cloud-native object storage the backbone of TiDB. This architecture enables elastic scalability, predictable performance, and optimized total cost of ownership (TCO) for AI-era workloads. | |
| TiDB X is a new distributed SQL architecture that makes cloud-native object storage the backbone of TiDB. Currently available in TiDB Cloud <CustomContent plan="starter,essential">Starter and Essential</CustomContent><CustomContent plan="premium">Starter, Essential, and Premium</CustomContent>, this architecture enables elastic scalability, predictable performance, and optimized total cost of ownership (TCO) for AI-era workloads. |
|
|
||
| ### Rapid elastic scalability | ||
|
|
||
| With data stored in shared object storage and each Region managed by an isolated LSM tree, TiDB X eliminates the need for physical data migration or large-scale compaction when adding or removing TiKV nodes. As a result, scaling operations are **5× to 10× faster** than in classic TiDB, while maintaining stable latency for online workloads. |
There was a problem hiding this comment.
How about adding a summary in the end?
Comparison summary
The following table summarizes the architectural transitions from classic TiDB to TiDB X and explains how TiDB X improves scalability, performance isolation, and cost efficiency.
| Feature | Classic TiDB | TiDB X | Primary benefit |
|---|---|---|---|
| Architecture | Shared-nothing (data stored on local disks) | Shared-storage (object storage as authoritative persistent storage) | Decouples compute from storage and enables cloud-native elasticity |
| Workload isolation | Foreground and background tasks share the same resources | Separation of compute and compute (elastic compute pools for heavy tasks) | Protects OLTP workloads from performance interference |
| Scaling mechanism | Physical data migration (SST file copying between nodes) | Logical attachment via metadata (no physical data movement) | 5×–10× faster scale-out and scale-in |
| Storage engine | Single LSM tree per TiKV node (RocksDB) | LSM forest (one independent LSM tree per Region) | Eliminates global mutex contention and reduces compaction interference |
| DDL execution | DDL competes with user traffic for local CPU and I/O | DDL offloaded to elastic compute resources | Faster schema changes with more predictable latency |
| Performance stability | Sensitive to compaction and rebalancing under heavy load | Background tasks isolated from online traffic | More stable latency under write-intensive or maintenance workloads |
| Cost model | Requires over-provisioning for peak workloads | Elastic TCO (pay-as-you-go) | Pay only for actual resource consumption |
| Backup and recovery | Data-volume dependent physical backup | Metadata-driven with object storage integration | Significantly faster backup and restore operations |
7 participants
First-time contributors' checklist
What is changed, added or deleted? (Required)
Which TiDB version(s) do your changes apply to? (Required)
Tips for choosing the affected version(s):
By default, CHOOSE MASTER ONLY so your changes will be applied to the next TiDB major or minor releases. If your PR involves a product feature behavior change or a compatibility change, CHOOSE THE AFFECTED RELEASE BRANCH(ES) AND MASTER.
For details, see tips for choosing the affected versions.
What is the related PR or file link(s)?
Do your changes match any of the following descriptions?