Designing and Configuring Disk Stores

You define disk stores in your cache, then you assign them to your regions and queues by setting the disk-store-name attribute in your region and queue configurations.

Note: Besides the disk stores you specify, Pivotal GemFire has a default disk store that it uses when disk use is configured with no disk store name specified. By default, this disk store is saved to the application’s working directory. You can change its behavior, as indicated in Create and Configure Your Disk Stores and Modifying the Default Disk Store.

Design Your Disk Stores

Before you begin, you should understand GemFire Basic Configuration and Programming.
  1. Work with your system designers and developers to plan for anticipated disk storage requirements in your testing and production caching systems. Take into account space and functional requirements.
    • For efficiency, separate data that is only overflowed in separate disk stores from data that is persisted or persisted and overflowed. Regions can be overflowed, persisted, or both. Server subscription queues are only overflowed. Gateway sender queues are always overflowed and may be persisted. Assign them to overflow disk stores if you do not persist, and to persistence disk stores if you do.
    • When calculating your disk requirements, figure in your data modification patterns and your compaction strategy. GemFire creates each oplog file at the max-oplog-size, which defaults to 1 GB. Obsolete operations are only removed from the oplogs during compaction, so you need enough space to store all operations that are done between compactions. For regions where you are doing a mix of updates and deletes, if you use automatic compaction, a good upper bound for the required disk space is
      (1 / (1 - (compaction_threshold/100)) ) * data size
      where data size is the total size of all the data you store in the disk store. So, for the default compaction-threshold of 50, the disk space is roughly twice your data size. Note that the compaction thread could lag behind other operations, causing disk use to rise above the threshold temporarily. If you disable automatic compaction, the amount of disk required depends on how many obsolete operations accumulate between manual compactions.
  2. Work with your host system administrators to determine where to place your disk store directories, based on your anticipated disk storage requirements and the available disks on your host systems.
    • Make sure the new storage does not interfere with other processes that use disk on your systems. If possible, store your files to disks that are not used by other processes, including virtual memory or swap space. If you have multiple disks available, for the best performance, place one directory on each disk.
    • Use different directories for different members. You can use any number of directories for a single disk store.

Create and Configure Your Disk Stores

  1. In the locations you have chosen, create all directories you will specify for your disk stores to use. GemFire throws an exception if the specified directories are not available when a disk store is created. You do not need to populate these directories with anything.
  2. Choose disk store names that reflect how the stores should be used and that work for your operating systems. Disk store names are used in the disk file names:
    • Use disk store names that satisfy the file naming requirements for your operating system. For example, if you store your data to disk in a Windows system, your disk store names could not contain any of these reserved characters, < > : " / \ | ? *.
    • Do not use very long disk store names. The full file names must fit within your operating system limits. On Linux, for example, the standard limitation is 255 characters.
  3. Configure each disk store:
    1. Set the name. If you are modifying the default disk store configuration, use "DEFAULT".
      Example:
      name="serverOverflow"
    2. Configure the directory locations and the maximum space to use for the store.
      Example:
      <disk-dirs>
      	  <disk-dir>c:\overflow_data</disk-dir>
      	  <disk-dir dir-size="20480">d:\overflow_data</disk-dir>
      </disk-dirs>
    3. As needed, modify the store’s file compaction behavior. In conjunction with this, plan and program for any manual compaction.
      Example:
      compaction-threshold="40" 
      auto-compact="false" 
      allow-force-compaction="true" 
    4. As needed, modify the maximum size of a single oplog. When the current files reach this size, the system rolls forward to a new file. You get better performance with relatively small maximum file sizes.
      Example:
      max-oplog-size="512"
    5. As needed, modify queue management parameters for asynchronous queueing to the disk store. You can configure any region for synchronous or asynchronous queueing (region attribute disk-synchronous). Server queues and gateway sender queues always operate synchronously. When either the queue-size or time-interval is reached, enqueued data is flushed to disk. You can also synchronously flush unwritten data to disk through the DiskStore flushToDisk method.
      Example:
      queue-size="10000" 
      time-interval="15"
    6. As needed, modify the size of the buffer used for writing to disk.
      Example:
      write-buffer-size="65536"

    Complete disk store XML configuration example:

    <disk-store name="serverOverflow" compaction-threshold="40" 
    	       auto-compact="false" allow-force-compaction="true"
       	    max-oplog-size="512" queue-size="10000"  
       	    time-interval="15" write-buffer-size="65536">
    	   <disk-dirs>
    		      <disk-dir>c:\overflow_data</disk-dir>
    		      <disk-dir dir-size="20480">d:\overflow_data</disk-dir>
    	   </disk-dirs>
    </disk-store>

When you start your system, all defined disk stores will be used by GemFire as configured.

Related Topics
com.gemstone.gemfire.cache.DiskStore

Configuring Regions, Queues, and PDX Serialization to Use the Disk Stores

Example of using a named disk store for region persistence and overflow:
<region refid="PARTITION_PERSISTENT_OVERFLOW" 
    disk-store-name="persistOverflow1"/>
Example of using the default disk store for gateway sender queue persistence:
<cache>
  <gateway-sender id="persistedsender1" parallel="true" 
   remote-distributed-system-id="1"
   enable-persistence="true"
   disk-store-name="diskStoreA"
   maximum-queue-memory="100"/> 
   ... 
</cache>

Example of using the default disk store for server subscription queue overflow:

<cache-server port="40404">
	   <client-subscription eviction-policy="entry" 
        capacity="10000" disk-store-name="queueOverflow2"/>
</cache-server>

Example of using a named disk store for PDX serialization metadata:

<pdx read-serialized="true" 
    persistent="true" disk-store-name="SerializationDiskStore">
</pdx>