Bad Disk Repair

Bad disk troubleshooting

When a disk failure occurs, it can be checked by the following methods:

  • In the Replica Server logs, an IO error was found for a certain disk
  • It is possible that the latency of a certain server is significantly higher than that of other servers. Continuing to investigate, if it is found that the IO wait of a certain disk is significantly higher, it basically proves that the disk is a slow disk

Bad disk blacklist

Pegasus supports disk black list, if you want to take a bad disk offline, firstly, define it in the disk black list file on the Replica Server where it is located, the file path is determined by the configuration:

    data_dirs_black_list_file = /home/work/.pegasus_data_dirs_black_list

Then log in to the corresponding server and edit the file, for example, disable ssd2 and ssd3:


Restart service

After marking the black list of bad disks, a restart is required to take effect. It is recommended to restart the Replica Server process on the corresponding server through High availability restart steps.

After restarting, the following records can be found in the server log, indicating that the disks marked in the black list have taken effect:

data_dirs_black_list_file[/home/work/.pegasus_data_dirs_black_list] found, apply it
black_list[1] = [/home/work/ssd2/]
black_list[2] = [/home/work/ssd3/]
Copyright © 2023 The Apache Software Foundation. Licensed under the Apache License, Version 2.0.

Apache Pegasus is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.

Apache Pegasus, Pegasus, Apache, the Apache feather logo, and the Apache Pegasus project logo are either registered trademarks or trademarks of The Apache Software Foundation in the United States and other countries.