CompactionManager (apache-cassandra API)
Date: 2018-02-14

org.apache.cassandra.db.compaction

## Class CompactionManager

- java.lang.Object
- - org.apache.cassandra.db.compaction.CompactionManager

- All Implemented Interfaces:
    CompactionManagerMBean

      public class CompactionManager
      extends java.lang.Object
      implements CompactionManagerMBean

  A singleton which manages a private executor of ongoing compactions.

  Scheduling for compaction is accomplished by swapping sstables to be compacted into a set via Tracker. New scheduling attempts will ignore currently compacting sstables.

- - ### Nested Class Summary

    Nested Classes
    Modifier and Type | Class and Description
    --- | ---
    `static class` | `CompactionManager.AllSSTableOpStatus`
    `static interface` | `CompactionManager.CompactionExecutorStatsCollector`

  - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `static int` | `GC_ALL`
    `static CompactionManager` | `instance`
    `static io.netty.util.concurrent.FastThreadLocal<java.lang.Boolean>` | `isCompactionManager`
    `static java.lang.String` | `MBEAN_OBJECT_NAME`
    `static int` | `NO_GC`

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `CompactionManager()`

  - ### Method Summary

    All Methods Static Methods Instance Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `static SSTableWriter` | `createWriter(ColumnFamilyStore cfs, java.io.File compactionFileLocation, long expectedBloomFilterSize, long repairedAt, SSTableReader sstable, LifecycleTransaction txn)`
    `static SSTableWriter` | `createWriterForAntiCompaction(ColumnFamilyStore cfs, java.io.File compactionFileLocation, int expectedBloomFilterSize, long repairedAt, java.util.Collection<SSTableReader> sstables, LifecycleTransaction txn)`
    `void` | `disableAutoCompaction()`
    `void` | `finishCompactionsAndShutdown(long timeout, java.util.concurrent.TimeUnit unit)`
    `void` | `forceCompactionForTokenRange(ColumnFamilyStore cfStore, java.util.Collection<Range<Token>> ranges)`
    `void` | `forceShutdown()` Shutdowns both compaction and validation executors, cancels running compaction / validation, and waits for tasks to complete if tasks were not cancelable.
    `void` | `forceUserDefinedCleanup(java.lang.String dataFiles)` Triggers the cleanup of user specified sstables.
    `void` | `forceUserDefinedCompaction(java.lang.String dataFiles)` Triggers the compaction of user specified sstables.
    `int` | `getActiveCompactions()`
    `javax.management.openmbean.TabularData` | `getCompactionHistory()` compaction history
    `java.util.List<java.util.Map<java.lang.String,java.lang.String>>` | `getCompactions()` List of running compaction objects.
    `java.util.List<java.lang.String>` | `getCompactionSummary()` List of running compaction summary strings.
    `long` | `getCompletedTasks()`
    `int` | `getCoreCompactorThreads()` Returns core size of compaction thread pool
    `int` | `getCoreValidationThreads()` Returns core size of validation thread pool
    `static int` | `getDefaultGcBefore(ColumnFamilyStore cfs, int nowInSec)`
    `int` | `getMaximumCompactorThreads()` Returns maximum size of compaction thread pool
    `int` | `getMaximumValidatorThreads()` Returns size of validator thread pool
    `int` | `getPendingTasks()`
    `com.google.common.util.concurrent.RateLimiter` | `getRateLimiter()` Gets compaction rate limiter.
    `long` | `getTotalBytesCompacted()`
    `long` | `getTotalCompactionsCompleted()`
    `void` | `interruptCompactionFor(java.lang.Iterable<CFMetaData> columnFamilies, boolean interruptValidation)` Try to stop all of the compactions for given ColumnFamilies.
    `void` | `interruptCompactionForCFs(java.lang.Iterable<ColumnFamilyStore> cfss, boolean interruptValidation)`
    `boolean` | `isCompacting(java.lang.Iterable<ColumnFamilyStore> cfses)`
    `static boolean` | `needsCleanup(SSTableReader sstable, java.util.Collection<Range<Token>> ownedRanges)` Determines if a cleanup would actually remove any data in this SSTable based on a set of owned ranges.
    `void` | `performAnticompaction(ColumnFamilyStore cfs, java.util.Collection<Range<Token>> ranges, Refs<SSTableReader> validatedForRepair, LifecycleTransaction txn, long repairedAt, java.util.UUID parentRepairSession)` Make sure the {validatedForRepair} are marked for compaction before calling this.
    `CompactionManager.AllSSTableOpStatus` | `performCleanup(ColumnFamilyStore cfStore, int jobs)`
    `CompactionManager.AllSSTableOpStatus` | `performGarbageCollection(ColumnFamilyStore cfStore, CompactionParams.TombstoneOption tombstoneOption, int jobs)`
    `void` | `performMaximal(ColumnFamilyStore cfStore, boolean splitOutput)`
    `CompactionManager.AllSSTableOpStatus` | `performScrub(ColumnFamilyStore cfs, boolean skipCorrupted, boolean checkData, boolean reinsertOverflowedTTL, int jobs)`
    `CompactionManager.AllSSTableOpStatus` | `performScrub(ColumnFamilyStore cfs, boolean skipCorrupted, boolean checkData, int jobs)`
    `CompactionManager.AllSSTableOpStatus` | `performSSTableRewrite(ColumnFamilyStore cfs, boolean excludeCurrentVersion, int jobs)`
    `CompactionManager.AllSSTableOpStatus` | `performVerify(ColumnFamilyStore cfs, boolean extendedVerify)`
    `CompactionManager.AllSSTableOpStatus` | `relocateSSTables(ColumnFamilyStore cfs, int jobs)`
    `java.util.List<SSTableReader>` | `runIndexSummaryRedistribution(IndexSummaryRedistribution redistribution)`
    `void` | `setConcurrentCompactors(int value)`
    `void` | `setCoreCompactorThreads(int number)` Allows user to resize maximum size of the compaction thread pool.
    `void` | `setCoreValidationThreads(int number)` Allows user to resize maximum size of the compaction thread pool.
    `void` | `setMaximumCompactorThreads(int number)` Allows user to resize maximum size of the compaction thread pool.
    `void` | `setMaximumValidatorThreads(int number)` Allows user to resize maximum size of the validator thread pool.
    `void` | `setRate(double throughPutMbPerSec)` Sets the rate for the rate limiter.
    `void` | `stopCompaction(java.lang.String type)` Stop all running compaction-like tasks having the provided `type`.
    `void` | `stopCompactionById(java.lang.String compactionId)` Stop an individual running compaction using the compactionId.
    `com.google.common.util.concurrent.ListenableFuture<?>` | `submitAntiCompaction(ColumnFamilyStore cfs, java.util.Collection<Range<Token>> ranges, Refs<SSTableReader> sstables, long repairedAt, java.util.UUID parentRepairSession)`
    `java.util.List<java.util.concurrent.Future<?>>` | `submitBackground(ColumnFamilyStore cfs)` Call this whenever a compaction might be needed on the given columnfamily.
    `java.util.concurrent.Future<?>` | `submitCacheWrite(AutoSavingCache.Writer writer)`
    `java.util.concurrent.Future<?>` | `submitIndexBuild(SecondaryIndexBuilder builder)` Is not scheduled, because it is performing disjoint work from sstable compaction.
    `java.util.List<java.util.concurrent.Future<?>>` | `submitMaximal(ColumnFamilyStore cfStore, int gcBefore, boolean splitOutput)`
    `java.util.concurrent.Future<?>` | `submitUserDefined(ColumnFamilyStore cfs, java.util.Collection<Descriptor> dataFiles, int gcBefore)`
    `java.util.concurrent.Future<?>` | `submitValidation(ColumnFamilyStore cfStore, Validator validator)` Does not mutate data, so is not scheduled.
    `java.util.concurrent.Future<?>` | `submitViewBuilder(ViewBuilder builder)`
    `void` | `waitForCessation(java.lang.Iterable<ColumnFamilyStore> cfss)`

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Field Detail

    - #### MBEAN_OBJECT_NAME

          public static final java.lang.String MBEAN_OBJECT_NAME

      See Also:
        Constant Field Values

    - #### instance

          public static final CompactionManager instance

    - #### NO_GC

          public static final int NO_GC

      See Also:
        Constant Field Values

    - #### GC_ALL

          public static final int GC_ALL

      See Also:
        Constant Field Values

    - #### isCompactionManager

          public static final io.netty.util.concurrent.FastThreadLocal<java.lang.Boolean> isCompactionManager

  - ### Constructor Detail

    - #### CompactionManager

          public CompactionManager()

  - ### Method Detail

    - #### getRateLimiter

          public com.google.common.util.concurrent.RateLimiter getRateLimiter()

      Gets compaction rate limiter. Rate unit is bytes per sec.
      Returns:
        RateLimiter with rate limit set

    - #### setRate

          public void setRate(double throughPutMbPerSec)

      Sets the rate for the rate limiter. When compaction_throughput_mb_per_sec is 0 or node is bootstrapping, this sets the rate to Double.MAX_VALUE bytes per second.
      Parameters:
        `throughPutMbPerSec` - throughput to set in mb per second

    - #### submitBackground

          public java.util.List<java.util.concurrent.Future<?>> submitBackground(ColumnFamilyStore cfs)

      Call this whenever a compaction might be needed on the given columnfamily. It's okay to over-call (within reason) if a call is unnecessary, it will turn into a no-op in the bucketing/candidate-scan phase.

    - #### isCompacting

          public boolean isCompacting(java.lang.Iterable<ColumnFamilyStore> cfses)

    - #### forceShutdown

          public void forceShutdown()

      Shutdowns both compaction and validation executors, cancels running compaction / validation, and waits for tasks to complete if tasks were not cancelable.

    - #### finishCompactionsAndShutdown

          public void finishCompactionsAndShutdown(long timeout,
                                                   java.util.concurrent.TimeUnit unit)
                                            throws java.lang.InterruptedException

      Throws:
        `java.lang.InterruptedException`

    - #### performScrub

          public CompactionManager.AllSSTableOpStatus performScrub(ColumnFamilyStore cfs,
                                                                   boolean skipCorrupted,
                                                                   boolean checkData,
                                                                   int jobs)
                                                            throws java.lang.InterruptedException,
                                                                   java.util.concurrent.ExecutionException

      Throws:
        `java.lang.InterruptedException`
        `java.util.concurrent.ExecutionException`

    - #### performScrub

          public CompactionManager.AllSSTableOpStatus performScrub(ColumnFamilyStore cfs,
                                                                   boolean skipCorrupted,
                                                                   boolean checkData,
                                                                   boolean reinsertOverflowedTTL,
                                                                   int jobs)
                                                            throws java.lang.InterruptedException,
                                                                   java.util.concurrent.ExecutionException

      Throws:
        `java.lang.InterruptedException`
        `java.util.concurrent.ExecutionException`

    - #### performVerify

          public CompactionManager.AllSSTableOpStatus performVerify(ColumnFamilyStore cfs,
                                                                    boolean extendedVerify)
                                                             throws java.lang.InterruptedException,
                                                                    java.util.concurrent.ExecutionException

      Throws:
        `java.lang.InterruptedException`
        `java.util.concurrent.ExecutionException`

    - #### performSSTableRewrite

          public CompactionManager.AllSSTableOpStatus performSSTableRewrite(ColumnFamilyStore cfs,
                                                                            boolean excludeCurrentVersion,
                                                                            int jobs)
                                                                     throws java.lang.InterruptedException,
                                                                            java.util.concurrent.ExecutionException

      Throws:
        `java.lang.InterruptedException`
        `java.util.concurrent.ExecutionException`

    - #### performCleanup

          public CompactionManager.AllSSTableOpStatus performCleanup(ColumnFamilyStore cfStore,
                                                                     int jobs)
                                                              throws java.lang.InterruptedException,
                                                                     java.util.concurrent.ExecutionException

      Throws:
        `java.lang.InterruptedException`
        `java.util.concurrent.ExecutionException`

    - #### performGarbageCollection

          public CompactionManager.AllSSTableOpStatus performGarbageCollection(ColumnFamilyStore cfStore,
                                                                               CompactionParams.TombstoneOption tombstoneOption,
                                                                               int jobs)
                                                                        throws java.lang.InterruptedException,
                                                                               java.util.concurrent.ExecutionException

      Throws:
        `java.lang.InterruptedException`
        `java.util.concurrent.ExecutionException`

    - #### relocateSSTables

          public CompactionManager.AllSSTableOpStatus relocateSSTables(ColumnFamilyStore cfs,
                                                                       int jobs)
                                                                throws java.util.concurrent.ExecutionException,
                                                                       java.lang.InterruptedException

      Throws:
        `java.util.concurrent.ExecutionException`
        `java.lang.InterruptedException`

    - #### submitAntiCompaction

          public com.google.common.util.concurrent.ListenableFuture<?> submitAntiCompaction(ColumnFamilyStore cfs,
                                                                                            java.util.Collection<Range<Token>> ranges,
                                                                                            Refs<SSTableReader> sstables,
                                                                                            long repairedAt,
                                                                                            java.util.UUID parentRepairSession)

    - #### performAnticompaction

          public void performAnticompaction(ColumnFamilyStore cfs,
                                            java.util.Collection<Range<Token>> ranges,
                                            Refs<SSTableReader> validatedForRepair,
                                            LifecycleTransaction txn,
                                            long repairedAt,
                                            java.util.UUID parentRepairSession)
                                     throws java.lang.InterruptedException,
                                            java.io.IOException

      Make sure the {validatedForRepair} are marked for compaction before calling this. Caller must reference the validatedForRepair sstables (via ParentRepairSession.getActiveRepairedSSTableRefs(..)).
      Parameters:
        `cfs` -
        `ranges` - Ranges that the repair was carried out on
        `validatedForRepair` - SSTables containing the repaired ranges. Should be referenced before passing them.
        `parentRepairSession` - parent repair session ID
      Throws:
        `java.lang.InterruptedException`
        `java.io.IOException`

    - #### performMaximal

          public void performMaximal(ColumnFamilyStore cfStore,
                                     boolean splitOutput)

    - #### submitMaximal

          public java.util.List<java.util.concurrent.Future<?>> submitMaximal(ColumnFamilyStore cfStore,
                                                                              int gcBefore,
                                                                              boolean splitOutput)

    - #### forceCompactionForTokenRange

          public void forceCompactionForTokenRange(ColumnFamilyStore cfStore,
                                                   java.util.Collection<Range<Token>> ranges)

    - #### forceUserDefinedCompaction

          public void forceUserDefinedCompaction(java.lang.String dataFiles)

      Description copied from interface: `CompactionManagerMBean`
      Triggers the compaction of user specified sstables. You can specify files from various keyspaces and columnfamilies. If you do so, user defined compaction is performed several times to the groups of files in the same keyspace/columnfamily.
      Specified by:
        `forceUserDefinedCompaction` in interface `CompactionManagerMBean`
      Parameters:
        `dataFiles` - a comma separated list of sstable file to compact. must contain keyspace and columnfamily name in path(for 2.1+) or file name itself.

    - #### forceUserDefinedCleanup

          public void forceUserDefinedCleanup(java.lang.String dataFiles)

      Description copied from interface: `CompactionManagerMBean`
      Triggers the cleanup of user specified sstables. You can specify files from various keyspaces and columnfamilies. If you do so, cleanup is performed each file individually
      Specified by:
        `forceUserDefinedCleanup` in interface `CompactionManagerMBean`
      Parameters:
        `dataFiles` - a comma separated list of sstable file to cleanup. must contain keyspace and columnfamily name in path(for 2.1+) or file name itself.

    - #### submitUserDefined

          public java.util.concurrent.Future<?> submitUserDefined(ColumnFamilyStore cfs,
                                                                  java.util.Collection<Descriptor> dataFiles,
                                                                  int gcBefore)

    - #### submitValidation

          public java.util.concurrent.Future<?> submitValidation(ColumnFamilyStore cfStore,
                                                                 Validator validator)

      Does not mutate data, so is not scheduled.

    - #### disableAutoCompaction

          public void disableAutoCompaction()

    - #### needsCleanup

          public static boolean needsCleanup(SSTableReader sstable,
                                             java.util.Collection<Range<Token>> ownedRanges)

      Determines if a cleanup would actually remove any data in this SSTable based on a set of owned ranges.

    - #### createWriter

          public static SSTableWriter createWriter(ColumnFamilyStore cfs,
                                                   java.io.File compactionFileLocation,
                                                   long expectedBloomFilterSize,
                                                   long repairedAt,
                                                   SSTableReader sstable,
                                                   LifecycleTransaction txn)

    - #### createWriterForAntiCompaction

          public static SSTableWriter createWriterForAntiCompaction(ColumnFamilyStore cfs,
                                                                    java.io.File compactionFileLocation,
                                                                    int expectedBloomFilterSize,
                                                                    long repairedAt,
                                                                    java.util.Collection<SSTableReader> sstables,
                                                                    LifecycleTransaction txn)

    - #### submitIndexBuild

          public java.util.concurrent.Future<?> submitIndexBuild(SecondaryIndexBuilder builder)

      Is not scheduled, because it is performing disjoint work from sstable compaction.

    - #### submitCacheWrite

          public java.util.concurrent.Future<?> submitCacheWrite(AutoSavingCache.Writer writer)

    - #### runIndexSummaryRedistribution

          public java.util.List<SSTableReader> runIndexSummaryRedistribution(IndexSummaryRedistribution redistribution)
                                                                      throws java.io.IOException

      Throws:
        `java.io.IOException`

    - #### getDefaultGcBefore

          public static int getDefaultGcBefore(ColumnFamilyStore cfs,
                                               int nowInSec)

    - #### submitViewBuilder

          public java.util.concurrent.Future<?> submitViewBuilder(ViewBuilder builder)

    - #### getActiveCompactions

          public int getActiveCompactions()

    - #### getCompactions

          public java.util.List<java.util.Map<java.lang.String,java.lang.String>> getCompactions()

      Description copied from interface: `CompactionManagerMBean`
      List of running compaction objects.
      Specified by:
        `getCompactions` in interface `CompactionManagerMBean`

    - #### getCompactionSummary

          public java.util.List<java.lang.String> getCompactionSummary()

      Description copied from interface: `CompactionManagerMBean`
      List of running compaction summary strings.
      Specified by:
        `getCompactionSummary` in interface `CompactionManagerMBean`

    - #### getCompactionHistory

          public javax.management.openmbean.TabularData getCompactionHistory()

      Description copied from interface: `CompactionManagerMBean`
      compaction history
      Specified by:
        `getCompactionHistory` in interface `CompactionManagerMBean`

    - #### getTotalBytesCompacted

          public long getTotalBytesCompacted()

    - #### getTotalCompactionsCompleted

          public long getTotalCompactionsCompleted()

    - #### getPendingTasks

          public int getPendingTasks()

    - #### getCompletedTasks

          public long getCompletedTasks()

    - #### stopCompaction

          public void stopCompaction(java.lang.String type)

      Description copied from interface: `CompactionManagerMBean`
      Stop all running compaction-like tasks having the provided `type`.
      Specified by:
        `stopCompaction` in interface `CompactionManagerMBean`
      Parameters:
        `type` - the type of compaction to stop. Can be one of: - COMPACTION - VALIDATION - CLEANUP - SCRUB - INDEX_BUILD

    - #### stopCompactionById

          public void stopCompactionById(java.lang.String compactionId)

      Description copied from interface: `CompactionManagerMBean`
      Stop an individual running compaction using the compactionId.
      Specified by:
        `stopCompactionById` in interface `CompactionManagerMBean`
      Parameters:
        `compactionId` - Compaction ID of compaction to stop. Such IDs can be found in the transaction log files whose name starts with compaction_, located in the table transactions folder.

    - #### setConcurrentCompactors

          public void setConcurrentCompactors(int value)

    - #### getCoreCompactorThreads

          public int getCoreCompactorThreads()

      Description copied from interface: `CompactionManagerMBean`
      Returns core size of compaction thread pool
      Specified by:
        `getCoreCompactorThreads` in interface `CompactionManagerMBean`

    - #### setCoreCompactorThreads

          public void setCoreCompactorThreads(int number)

      Description copied from interface: `CompactionManagerMBean`
      Allows user to resize maximum size of the compaction thread pool.
      Specified by:
        `setCoreCompactorThreads` in interface `CompactionManagerMBean`
      Parameters:
        `number` - New maximum of compaction threads

    - #### getMaximumCompactorThreads

          public int getMaximumCompactorThreads()

      Description copied from interface: `CompactionManagerMBean`
      Returns maximum size of compaction thread pool
      Specified by:
        `getMaximumCompactorThreads` in interface `CompactionManagerMBean`

    - #### setMaximumCompactorThreads

          public void setMaximumCompactorThreads(int number)

      Description copied from interface: `CompactionManagerMBean`
      Allows user to resize maximum size of the compaction thread pool.
      Specified by:
        `setMaximumCompactorThreads` in interface `CompactionManagerMBean`
      Parameters:
        `number` - New maximum of compaction threads

    - #### getCoreValidationThreads

          public int getCoreValidationThreads()

      Description copied from interface: `CompactionManagerMBean`
      Returns core size of validation thread pool
      Specified by:
        `getCoreValidationThreads` in interface `CompactionManagerMBean`

    - #### setCoreValidationThreads

          public void setCoreValidationThreads(int number)

      Description copied from interface: `CompactionManagerMBean`
      Allows user to resize maximum size of the compaction thread pool.
      Specified by:
        `setCoreValidationThreads` in interface `CompactionManagerMBean`
      Parameters:
        `number` - New maximum of compaction threads

    - #### getMaximumValidatorThreads

          public int getMaximumValidatorThreads()

      Description copied from interface: `CompactionManagerMBean`
      Returns size of validator thread pool
      Specified by:
        `getMaximumValidatorThreads` in interface `CompactionManagerMBean`

    - #### setMaximumValidatorThreads

          public void setMaximumValidatorThreads(int number)

      Description copied from interface: `CompactionManagerMBean`
      Allows user to resize maximum size of the validator thread pool.
      Specified by:
        `setMaximumValidatorThreads` in interface `CompactionManagerMBean`
      Parameters:
        `number` - New maximum of validator threads

    - #### interruptCompactionFor

          public void interruptCompactionFor(java.lang.Iterable<CFMetaData> columnFamilies,
                                             boolean interruptValidation)

      Try to stop all of the compactions for given ColumnFamilies. Note that this method does not wait for all compactions to finish; you'll need to loop against isCompacting if you want that behavior.
      Parameters:
        `columnFamilies` - The ColumnFamilies to try to stop compaction upon.
        `interruptValidation` - true if validation operations for repair should also be interrupted

    - #### interruptCompactionForCFs

          public void interruptCompactionForCFs(java.lang.Iterable<ColumnFamilyStore> cfss,
                                                boolean interruptValidation)

    - #### waitForCessation

          public void waitForCessation(java.lang.Iterable<ColumnFamilyStore> cfss)

Copyright © 2018 The Apache Software Foundation