My ReadyNAS Pro 4 crashed after downloading or deleting big Logfiles! The result was a NAS stuck in the state „Booting …“
First check if the disks are bad! In my case everything was good!
If you still have warranty on your device ask the support for help! If not you can follow my instructions on your own risk! A little knowledge in Linux is necessary and a workstation running linux makes it easier because we need some tools to recover the system.
If you are lucky it is just a too big logfile in the backup directory. In my case that was not the only problem. Probably you should check this first, before you „Factory Reset“ everything :)
Boot in „Tech Support“-Mode and connect to your device like described below. Start your raid and mount the „sysroot“ (mount /dev/md0 /mnt) and run „du -csh /mnt/var/log/*“. If the frontview content is really big you can delete the files in the backup folder (/var/log/frontview/backup/*).
After a lot of investigation and trying i made it booting again. The Problem was a corrupted internal flash memory, which made my „Factory Reset“ procedure fail with a message „err: could not p“. It should say „ERR: Could not properly extract“ and means, that the firmware could not be properly extracted to the „sysroot“, because in my case the tar-Archive on the internal flash memory was corrupted.
The Recovery
First thing is the BootMenu:
In my Version of ReadyNAS i can reach the BootMenu by pressing and holding the „hidden“ reset button at the back of the device and power on the device. After a few seconds the display says „BootMenu“. You can cycle the Boot-Options with the „Backup“-Button at the front of the device (next to the USB-Port) and confirm your selection with another press to the reset button. From Netgear i have the following instructions:
- Power off your system.
- Using a straightened paper clip, press and hold the Reset button.
- Press the Power button to power on your system.
- Continue to press the Reset button until the status display screen shows a boot menu message.
- Press the Backup button to scroll through the boot mode options. The status display screen shows the current boot mode option.
- When the status display screen shows the boot mode you need, press and release the Reset button to confirm your boot menu selection.
The system boots in the selected boot mode.
These options are available:
- Normal. Initiates a normal boot process, just like booting using the Power button.
- Factory default. Initiates a short disk test that takes approximately 5 minutes, then begins a 10-minute time-out period waiting for RAIDar. During the 10-minute time-out period, the system can be powered off safely without causing any data loss. After 10 minutes, or if you select the factory default option with RAIDar during the time-out period, the factory default process begins.WARNING: The factory default reboot process resets the unit to factory settings, erases all data, resets all defaults, and reformats the disk to X-RAID2.
- OS reinstall. Reinstalls the firmware from the internal flash to the disks. Use the OS reinstall boot mode when the system crashes and corrupts some configuration files. OS reinstall boot mode also resets some settings on your storage system, such as Internet protocol settings and the administrator password, to defaults.
- Tech support. Boots into a low-level diagnostic mode. Use the tech support boot mode only when instructed to do so by a NETGEAR technical support representative.
- Skip volume check. After a system crash, the system tries to scan and fix the volume using a file system check. If several problems are found on a disk, this process can stall, causing the system not to boot. Use this option to skip the scan and let the system boot.
- WARNING: Use the skip volume check boot mode only when instructed to do so by a NETGEAR Technical Support representative, otherwise data loss might occur.
- Memory test. Performs a memory test. The pass or fail result is reported on the status display screen.
- Disk Test. Performs an offline full disk test. This can take 4 hours or more depending on the size of your disks. Any problems are reported on the status display screen or in the RAIDar discovery tool.
- You can try OS-Reinstall at this point and if you are lucky your device boots properly after that (but not in my case).
The first thing i recommend is to backup your data. I booted my device in „Tech support“-Mode, which starts an internal minimal environment (Linux with busybox). When the device is ready it gives you a „login-number“ for the NetgearSupport.
Now we want to connect to the device to backup our data. The device gets an IP address from your dhcp-server. If you don’t have a dhcp-server your device boots with a default IP address (i’m not sure what it is – google for it or let RAIDar find it for you). You can now telnet to your device. If you are a windows user you can use Putty.
telnet 192.168.1.2
You can login to your ReadyNAS with username „root“ and password „infr8ntdebug“
Old busybox
I tried a lot to get the data from the NAS to another device…and the old busybox version killed me a lot of times (it seems like nothing works) and i helped myself by copying a new busybox version from an usb-stick to /bin/ and everything worked fine. (Maybe copy it to /bin/busybox2 and issue every command on yourself is safer!)
To get access to your data you have to assemble the raid with the prepared script
start_raid.sh
Now the tool mdadm made your diskarray accessible in /dev.
If you want to check the system which the device will boot you can mount the sysroot. It depends on how your device was configured but in my case it was just /dev/md0 for the „sysroot“ and you can mount it to /sysroot in the temp fs.
mount /dev/md0 /sysroot/
Now you have to check where your data lives. It is really helpful if you now how Linux works…
Accessing your data
I’m not sure but in some cases it is using LVM and sometimes not (I had both cases on different devices)
LVM (Logical Volume Management)
If your data is stored somewhere in an LV you have to mount your LV. You can check this by issuing the command
vgdisplay
If you will find a volume group you have to make it active and than mount the LV somewhere to read or copy the data (In general it’s named „c“, i don’t know why).
vgchange -ay <VG> lvdisplay # or lvm lvs mount -o ro /dev/<lv-name-and-path> /data
If the name of the VolumeGroup is „c“:
vgchange -ay c lvm lvs mount -o ro /dev/c/c /data
SoftArray
If not your data could be in another partition of the Raid-Array. In my case it was /dev/md1 and you can mount it to /data or /mnt/data…
mount /dev/md1 /data # mount -o ro /dev/md1 /data (for Read-Only)
USB-Disk target or another NAS
If you want to copy your data to another NAS you probably have to replace your busybox executable in /bin with a newer version like i described above. Maybe the easiest way is to copy your data to an USB-Harddrive. First thing is to figure out which name your USB-Disk got by the environment. Normally it is the last device in the list. Try to mount it to /mnt and check the contents… in my case it was /dev/sdf, because it is a 4-Bay NAS and sda,sdb,sdc,sdd are the internal storage disks, sde is the internal flash and sdf the external hard disk.
mount /dev/sdf1 /mnt/
If you have found your external disk, copy your previously mounted data and your are fine. Be aware of FS-Types (you’ll need the right filesystem type on the target device)
cp -rp /data /mnt
Recover the firmware to make your device boot again
Ok! Now it’s time to get a working firmware image. In my case it’s RAIDiator_4.2.30 (download it from Netgear-Support). You’ll need binwalk, a tool to analyze firmware images or other binary files. We need binwalk to extract the files from the firmware image.
binwalk RAIDiator-x86-4.2.30
it gives me something like this:
DECIMAL HEXADECIMAL DESCRIPTION -------------------------------------------------------------------------------- 16384 0x4000 POSIX tar archive (GNU), owner user name: "root", owner group name: "root"
and we get the position (16384) of the file/s. With this information we can extract the archive with a tool like dd:
dd if=RAIDiator-x86-4.2.30 bs=1 skip=16384 of=firmware.tar
which will create a tar with the firmware inside. Copy this tar to an USB-Stick or find any other way to get the file to the NAS (wget, tftp, scp, etc…). To avoid mistakes, i removed all my data drives from the NAS, inserted one old disk (the device needs at least one drive) and rebooted it in „Tech support“-Mode. Than i copied the archive with wget from my Debian workstation and extracted the firmware to the internal flash. But step by step. First mount the internal flash, which should now be /dev/sdb1
mkdir /mnt/flash mount /dev/sdb1 /mnt/flash
Now download the extracted image to your NAS with wget or scp
mkdir /mnt/firmware cd /mnt/firmware wget 192.168.1.1/firmware.tar
192.168.1.1 is my Linux workstation and i installed apache and moved the firmware.tar to /var/www/html/, to get access to the file from my network. (There are many other ways to get that file to the NAS)
Now extract the content of the file and move/overwrite the files on the internal flash.
tar vxf firmware.tar
which should extract these files:
csums.md5 initrd.gz kernel kernel.up memtest root.tlz supplmnt.tlz syslinux.cfg version.txt
make sure that you are in the /mnt/firmware directory and that these files are the only one in place! Move them to the mounted internal flash and please double check everything – you will overwrite those files
mv * /mnt/flash
Now save reboot the system with unmounting all mounts, make sure all data is synced and stop the raid.
cd sync umount /mnt mdadm --stop --scan
Now its save to reboot the NAS
Recover the OS
This – now repaired – internal memory is the source of the „OS Reinstall“ or „Factory Reset“ procedure and should now copy the files to the disks. Try a reboot and choose „Factory Reset“ with the unused disk. Everything will be deleted and the storage should boot fine in „Normal Mode“. You are done… Reinsert your drives make an „OS Reinstall“ and your data should be there and you can login to your device again with default user/password (admin/netgear1)
Hope this helps
PS:
Install the new OS 6.X
If you want to, you can now install the new OS 6. There is a good description in the NTGR-Forum. You will loose all warranty but with the new version you can install bigger drives and the mentioned fan problem doesn’t appear with my Pro4 and the firmware version I appended.
Download the latest firmware here or the one i used R4toR6_latest.bin. And just upgrade the NAS in Frontview 4.2 by uploading it in the Webinterface. Reboot and Factory Default via boot menu and you are good.
No responses yet