Why Your Cassandra Delete Queries Didn’t Free Up Disk Space (And How to Actually Reclaim It)

So you just ran a bunch of DELETE statements in Cassandra, expecting your disk usage to drop dramatically… and nothing happened.

Welcome to Apache Cassandra — where deleting data doesn’t actually mean deleting data.

Why Cassandra Doesn't Immediately Free Disk Space

In Cassandra, when you delete data, it doesn't get physically removed right away. Instead, Cassandra writes a special marker called a tombstone.

Think of tombstones like sticky notes saying:

"This data is deleted… but don't clean it up just yet."

These tombstones are necessary because Cassandra is a distributed system and needs to ensure deletes are properly propagated across replicas before physically removing anything.

So what actually happens?

• Your DELETE creates tombstones.
• Tombstones are written during regular writes or compaction workflows.
• Actual disk space is only reclaimed later during compaction.
• Depending on workload, this can take hours or even days.

You can make use of these tools provided by Cassandra to trigger cleanup sooner.

Step 1: Flush Memtables to Disk

nodetool flush <keyspace> <table>

What this does

This forces in-memory data (memtables) to be written into SSTables on disk. It ensures everything is persisted so compaction can work with it.

It was observed to usually take only a few seconds to execute even for tables around 350 GB in size.

Step 2: Trigger Garbage Collection via Compaction

nodetool garbagecollect <keyspace> <table>

What this does

This is essentially a manual compaction-triggered cleanup process that:

• Processes SSTables.
• Drops expired tombstones (based on the GC grace period).
• Rewrites data into new SSTables.
• Removes old SSTables when it is safe to do so.

The time taken to finish this process depends on how much data was deleted.

Important note: You will need a fair amount of free storage on the server for the garbagecollect command to work. During the process Cassandra rotates SSTables and rewrites data, which means it may temporarily consume more disk space before eventually reclaiming the deleted space.

You can monitor the progress using:

nodetool compactionstats

Do not be fooled by the reported completion percentage. Multiple compaction jobs can run one after another, so the first one reaching 100% does not necessarily mean the overall cleanup has finished.

Another Important Note

Avoid monitoring compaction using:

watch nodetool compactionstats

We once experienced a case where the watch command failed to exit even after pressing Ctrl + C. It continued running in the background and drove the server CPU utilization up to around 50–60%, whereas the server normally idled below 10%.

#cassandra #databases