Linux’owy dramat w 3 aktach

10:47 pm wszystko

Akt I

Scena I

# dmesg| tail

EXT3-fs error (device xvda): ext3_readdir: bad entry in directory #376470: directory entry across blocks - offset=0, inode=3085352006, rec_len=62128, name_len=235
Aborting journal on device xvda.
ext3_abort called.
EXT3-fs error (device xvda): ext3_journal_start_sb: Detected aborted journal
Remounting filesystem read-only

W tym momencie przestają działać wszystkie serwisy www stojące na serwerze.

Scena II

# fsck / -> wszysto ładnie i clean, tylko że dalej / jest zamontowany tylko do odczytu

Scena II

# reboot

(w czase rebootu zaczynam sie zastanawiać gdzie/czy jest backup i jak często był robiony.

Akt II

Scena I

>ping 1.2.3.4

Pinging 1.2.3.4  with 32 bytes of data:
Request timed out.
Request timed out.
Request timed out.
Request timed out.

Scena II (tragiczna)

(recovery console)

INIT: version 2.86 booting
mount: relocation error: /lib/libpthread.so.0: symbol _h_errno, version GLIBC_2.0 not defined in file libc.so.6 with link time reference
touch: relocation error: /lib/libpthread.so.0: symbol _h_errno, version GLIBC_2.0 not defined in file libc.so.6 with link time reference
mount: relocation error: /lib/libpthread.so.0: symbol _h_errno, version GLIBC_2.0 not defined in file libc.so.6 with link time reference
mount: relocation error: /lib/libpthread.so.0: symbol _h_errno, version GLIBC_2.0 not defined in file libc.so.6 with link time reference
* Filesystem type ‘devpts’ is not supported. Skipping mount.
mount: relocation error: /lib/libpthread.so.0: symbol _h_errno, version GLIBC_2.0 not defined in file libc.so.6 with link time reference
umount: relocation error: /lib/libpthread.so.0: symbol _h_errno, version GLIBC_2.0 not defined in file libc.so.6 with link time reference
Activating swap…failed.
mount: relocation error: /lib/libpthread.so.0: symbol _h_errno, version GLIBC_2.0 not defined in file libc.so.6 with link time reference
Setting the system clock..
date: relocation error: /lib/libpthread.so.0: symbol _h_errno, version GLIBC_2.0 not defined in file libc.so.6 with link time reference
System Clock set. Local time: .
/etc/rcS.d/S12mtab.sh: line 144: /proc/mounts: No such file or directory
Cleaning up ifupdown…rm: cannot remove `/etc/network/run/ifstate’: Read-only file system
Loading device-mapper support.
Checking file systems…fsck: relocation error: /lib/libpthread.so.0: symbol _h_errno, version GLIBC_2.0 not defined in file libc.so.6 with link time reference
fsck died with exit status 127
failed (code 127).
* File system check failed.

# fsck

fsck: relocation error: /lib/libpthread.so.0: symbol _h_errno, version GLIBC_2.0 not defined in file libc.so.6 with link time reference

Scena III

(Recovery Finnix Image)

# fsck /dev/costam

fsck: costam było źle, ale naprawiłem i jest ładnie

Scena IV

(patrz scena II)

Scena V;

srv://lost+found# ls |wc -l
1053

po przyjrzeniu się zawartości lost+fount okazało się, że jest tam śmiertelna mieszanka /lib /var/www i troszke z /usr. Bohater umarł.

Akt III

Błąd w Xen, coś z rozmiarem bloku ext3. Cały dzień zmarnowany.

Leave a Comment

Your comment

You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Please note: Comment moderation is enabled and may delay your comment. There is no need to resubmit your comment.