Resolve Scrub Errors on Ceph

Table of Contents

Saat memantau Cluster Ceph melalui dashboard. Ditemukan peringatan health: HEALTH_ERR 1 scrub errors/Possible data damage: 1 pg inconsistent.

Situation
#

Jalankan ceph -s mendapati output berikut

  cluster:
    id:     0260f99a-117e-4c7e-8fbe-86c483bcd7e9
    health: HEALTH_ERR
            1 scrub errors
            Possible data damage: 1 pg inconsistent

  services:
    : 3 daes, quorum mon01,mon02,mon03 (age 10w)
    mgr: mon01(active, since 7w), standbys: mon02, mon03
    mds: cephfs:1 {0=mds01=up:active} 2 up:standby
    osd: 285 osds: 285 up (since 43h), 285 in (since 2w)
    rgw: 3 daes active (cephrgw01, cephrgw02, cephrgw03)

  data:
    pools:   8 pools, 4328 pgs
    objects: 294.96M objects, 463 TiB
    usage:   694 TiB used, 1.3 PiB / 2.0 PiB avail
    pgs:     4320 active+clean
             7    active+clean+scrubbing+deep
             1    active+clean+scrubbing+deep+inconsistent

  io:
    client:   3.8 MiB/s rd, 188 MiB/s wr, 11 op/s rd, 732 op/s wr

Resolution
#

Jalankan ceph health detail untuk menemukan ID pg untuk pg yang tidak konsisten

HEALTH_ERR 1 scrub errors; Possible data damage: 1 pg inconsistent
OSD_SCRUB_ERRORS 1 scrub errors
PG_DAMAGED Possible data damage: 1 pg inconsistent
    pg 5.6f1 is active+clean+scrubbing+deep+inconsistent, acting [7,141,208,199,70,37,182,131,120,259]

Perbaiki pg dengan command ceph pg repair $pgid