[arch-devops] Broken disk on vostok

[Sven-Hendrik Haase]

On Sat, Jun 27, 2020, 19:37 Jelle van der Waa <jelle at vdwaa.nl> wrote:

> On 27/06/2020 04:01, Sven-Hendrik Haase via arch-devops wrote:
> > On 26.06.20 19:07, Sven-Hendrik Haase wrote:
> >> Hey all,
> >>
> >> During a small routine check I noticed that we have a broken disk on
> >> vostok. In fact, according to the log, we've had it for 7 months at
> >> least which is a bit embarrassing given that this is RAID1 only and it's
> >> also our primary backup box. This really goes to show that we need
> >> better monitoring/more meaningful alerts.
> >>
> >> Anyway, I had Hetzner replace the disk and the array is now rebuilding.
> >>
> >> Cheers,
> >> Sven
> >>
> >
> > The array is now resynced and appears to be happy. Yay!
>
> Thanks for handling and finding the issue sven! However how can our
> monitoring not catch this? Especially for gemini it would be nice to
> know :-)
>
> Greetings,
>
> Jelle van der Waa
>

Yeah, well I dunno. That's gonna get really embarrassing if we don't notice
for too long at some point.

>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.archlinux.org/pipermail/arch-devops/attachments/20200627/e2ec0bc4/attachment.htm>