This guide describes DRBD version 8.4 and above. For 8.3 please look here.


17.2. Generation Identifiers

DRBD uses generation identifiers (GIs) to identify "generations"of replicated data.

This is DRBD’s internal mechanism used for

17.2.1. Data generations

DRBD marks the start of a new data generation at each of the following occurrences:

  • The initial device full sync,
  • a disconnected resource switching to the primary role,
  • a resource in the primary role disconnecting.

Thus, we can summarize that whenever a resource is in the Connected connection state, and both nodes' disk state is UpToDate, the current data generation on both nodes is the same. The inverse is also true.

Every new data generation is identified by a 8-byte, universally unique identifier (UUID).

17.2.2. The generation identifier tuple

DRBD keeps four pieces of information about current and historical data generations in the local resource meta data:

Current UUID. This is the generation identifier for the current data generation, as seen from the local node’s perspective. When a resource is Connected and fully synchronized, the current UUID is identical between nodes.

Bitmap UUID. This is the UUID of the generation against which the on-disk sync bitmap is tracking changes. As the on-disk sync bitmap itself, this identifier is only relevant while in disconnected mode. If the resource is Connected, this UUID is always empty (zero).

Two Historical UUIDs. These are the identifiers of the two data generations preceding the current one.

Collectively, these four items are referred to as the generation identifier tuple, or GI tuple" for short.

17.2.3. How generation identifiers change

17.2.3.1. Start of a new data generation

When a node loses connection to its peer (either by network failure or manual intervention), DRBD modifies its local generation identifiers in the following manner:

Figure 17.3. GI tuple changes at start of a new data generation

gi-changes-newgen

  1. A new UUID is created for the new data generation. This becomes the new current UUID for the primary node.
  2. The previous UUID now refers to the generation the bitmap is tracking changes against, so it becomes the new bitmap UUID for the primary node.
  3. On the secondary node, the GI tuple remains unchanged.

17.2.3.2. Start of re-synchronization

Upon the initiation of re-synchronization, DRBD performs these modifications on the local generation identifiers:

Figure 17.4. GI tuple changes at start of re-synchronization

gi-changes-syncstart

  1. The current UUID on the synchronization source remains unchanged.
  2. The bitmap UUID on the synchronization source is rotated out to the first historical UUID.
  3. A new bitmap UUID is generated on the synchronization source.
  4. This UUID becomes the new current UUID on the synchronization target.
  5. The bitmap and historical UUID’s on the synchronization target remain unchanged.

17.2.3.3. Completion of re-synchronization

When re-synchronization concludes, the following changes are performed:

Figure 17.5. GI tuple changes at completion of re-synchronization

gi-changes-synccomplete

  1. The current UUID on the synchronization source remains unchanged.
  2. The bitmap UUID on the synchronization source is rotated out to the first historical UUID, with that UUID moving to the second historical entry (any existing second historical entry is discarded).
  3. The bitmap UUID on the synchronization source is then emptied (zeroed).
  4. The synchronization target adopts the entire GI tuple from the synchronization source.

17.2.4. How DRBD uses generation identifiers

When a connection between nodes is established, the two nodes exchange their currently available generation identifiers, and proceed accordingly. A number of possible outcomes exist:

Current UUIDs empty on both nodes. The local node detects that both its current UUID and the peer’s current UUID are empty. This is the normal occurrence for a freshly configured resource that has not had the initial full sync initiated. No synchronization takes place; it has to be started manually.

Current UUIDs empty on one node. The local node detects that the peer’s current UUID is empty, and its own is not. This is the normal case for a freshly configured resource on which the initial full sync has just been initiated, the local node having been selected as the initial synchronization source. DRBD now sets all bits in the on-disk sync bitmap (meaning it considers the entire device out-of-sync), and starts synchronizing as a synchronization source. In the opposite case (local current UUID empty, peer’s non-empty), DRBD performs the same steps, except that the local node becomes the synchronization target.

Equal current UUIDs. The local node detects that its current UUID and the peer’s current UUID are non-empty and equal. This is the normal occurrence for a resource that went into disconnected mode at a time when it was in the secondary role, and was not promoted on either node while disconnected. No synchronization takes place, as none is necessary.

Bitmap UUID matches peer’s current UUID. The local node detects that its bitmap UUID matches the peer’s current UUID, and that the peer’s bitmap UUID is empty. This is the normal and expected occurrence after a secondary node failure, with the local node being in the primary role. It means that the peer never became primary in the meantime and worked on the basis of the same data generation all along. DRBD now initiates a normal, background re-synchronization, with the local node becoming the synchronization source. If, conversely, the local node detects that its bitmap UUID is empty, and that the peer’s bitmap matches the local node’s current UUID, then that is the normal and expected occurrence after a failure of the local node. Again, DRBD now initiates a normal, background re-synchronization, with the local node becoming the synchronization target.

Current UUID matches peer’s historical UUID. The local node detects that its current UUID matches one of the peer’s historical UUID’s. This implies that while the two data sets share a common ancestor, and the peer node has the up-to-date data, the information kept in the peer node’s bitmap is outdated and not usable. Thus, a normal synchronization would be insufficient. DRBD now marks the entire device as out-of-sync and initiates a full background re-synchronization, with the local node becoming the synchronization target. In the opposite case (one of the local node’s historical UUID matches the peer’s current UUID), DRBD performs the same steps, except that the local node becomes the synchronization source.

Bitmap UUIDs match, current UUIDs do not. The local node detects that its current UUID differs from the peer’s current UUID, and that the bitmap UUID’s match. This is split brain, but one where the data generations have the same parent. This means that DRBD invokes split brain auto-recovery strategies, if configured. Otherwise, DRBD disconnects and waits for manual split brain resolution.

Neither current nor bitmap UUIDs match. The local node detects that its current UUID differs from the peer’s current UUID, and that the bitmap UUID’s do not match. This is split brain with unrelated ancestor generations, thus auto-recovery strategies, even if configured, are moot. DRBD disconnects and waits for manual split brain resolution.

No UUIDs match. Finally, in case DRBD fails to detect even a single matching element in the two nodes' GI tuples, it logs a warning about unrelated data and disconnects. This is DRBD’s safeguard against accidental connection of two cluster nodes that have never heard of each other before.


This guide describes DRBD version 8.4 and above. For 8.3 please look here.