Can't boot my iMac OS (high sierra) or Boot Camp / Windows!
So suddenly my iMac wouldn't boot at all one morning. The boot process stalled with only the iMac prohibitory symbol showing. When trying to boot up in Windows/Boot Camp, Windows blue-screened complaining about the AppleSSD.sys driver. I had hibernated Windows in Boot Camp the previous night like I always do. I suddenly started fearing the worst, SSD or HDD failure. This is my go-to machine for everything including software development and I didn't do a lot in the way of backups, but my source code for development was on an external drive, thankfully. I needed a repair or a workaround at the least.
So my goals in order were:
- Determine what went wrong in a non-destructive way.
- Make backups of everything I could before trying to come up with a solution that might change or even destroy data.
- Re-create a working system with as much data and os booting capabilities restored as possible.
- Do this all without cracking open my iMac (myself or at a shop).
What went wrong with this Fusion drive? Starting with some simple repair attempts
Resetting NVRAM
It seems plenty of suggestions on the internet for similar Mac drive issues start with resetting NVRAM. I went ahead and tried this because I hadn’t gotten very far debugging yet and it was a simple thing to try. Unfortunately, it didn't seem to do much at all.
Disktuil verifyVolume/repairVolume
Several solutions I found suggested running "diskutil verifyVolume" (or repairVolume). I tried this and it resulted in:
- incomplete or inconsistent CoreStorage Physical Volume set
- Problems were encountered during repair of the partition map
- Error: -69716: Storage system verify or repair failed
- Underlying error: 1: Operation not permitted
Disk utility GUI from recovery
Luckily recovery was still working. I booted into Mac recovery mode by holding down the Command (⌘) – R combination immediately upon hearing the startup chime, and then ran disk utility. Disk utility showed the Boot Camp partition but the Mac OS partition showed as unformatted. Also, if you select the “show all devices” option, the SSD device didn't show up at all.
Running First Aid on Apple HDD media in disk utility
So I ran First Aid in the disk utility from recovery on the main HDD. This resulted in the following:
- Found corruption that needs to be repaired. To repair the startup volume, run first aid from Recovery. (I was already running first aid from recovery). The Details window showed:
- Logical Volume...spans 2 devices (this is the SSD and HDD). Incomplete or inconsistent CoreStorage Physical Volume set....Problems were found with the partition map which might prevent booting.
At this point I'm thinking maybe the partition map can be repaired but I'm not sure how this would be done especially considering that it looked more and more like the SSD had failed.
Diskutil, fsck, and single user mode
diskutil list (terminal)
So back into terminal from recovery. I ran “diskutil list” and this showed the Apple_CoreStorage Macintosh HD and the BOOTCAMP volumes which looked fine.
diskutil cs list (terminal; show corestorage)
CoreStorage is what comprises the logical volume that spans the two physical volumes in the fusion drive, the HDD and the SSD. This is where it got weird. This showed two physical volumes (The SSD and HDD that the logical volume spans) BUT the first physical volume showed "no properties" while the second physical volume showed properties such as “disk: disk0s2”.
From what I’ve seen, the first physical volume should show properties as well. So where were the SSD (first physical volume) properties?
Single user mode (Start your Mac while holding down the Command (⌘) – S combination on your keyboard)
Another suggestion I found for repairing volumes was to run fsck in single user mode.
I tried booting in single user mode so I could run fsck, but on bootup, after showing some terminal output, it hung at “pci pause: SDXC”. SDXC stands for SD extended capacity.
So no luck with single user mode. At this point I could have run Apple diagnostics by holding the D key while booting up but didn’t know about this until later (after I completed a solution). I ran it later after I settled on another solution and it showed no problems at all (even though the SSD still isn't found by disk utility).
I did start thinking (and still wonder) if it could be as simple as a loose cable to the SSD. But I’m not cracking open my iMac (or paying someone else to do it) if I can avoid it. And I'm doubting it's just a loose cable anyway.
So it seemed my SSD had/has failed. Note that some suggestions out there to fix a SSD problem (in a fusion drive) are to “fix a split fusion drive”. A split fusion drive is when the SSD and HDD appear as different volumes. This just wasn't the case here. I didn't have a split fusion drive. The system couldn't see the SSD device at all anymore.
So how do I move forward? I assume I’ve lost some if not all my data and both OS’s. But since I can see the Boot Camp drive, my first goal was to recover the data from that if not actually restore it completely so I could boot from it. Unfortunately, at this point, I gave up on trying to save anything from the Mac OS volume.
Backup what I can
Image from Bootcamp option in disk utility
Running diskutil from recovery, I used the “Image from Bootcamp” option when you right click a volume in disktuil.
This creates a DMG file that is mountable by simply double clicking it in Finder. I marked it for read/write so I could make changes if necessary. After creating the image file I verified I could access all the files on the NTFS Boot Camp volume and could even copy them to another location. Although the DMG can be mounted directly from a Mac, it’s an NTFS volume, so it's mounted read-only and therefore can’t be modified without more work….
Modify and restore Boot Camp on fresh installation of Mojave
Using NTFS-3G to modify an NTFS Boot Camp volume on Mac
Since the original saved Boot Camp volume wouldn’t boot because it was complaining about AppleSSD.sys, I decided to try renaming AppleSSD.sys (and it’s backup) on the NTFS Boot Camp volume. I used an open-source utility to be able to mount the NTFS volume for modification. Instructions for installing and using NTFS-3G are here: https://github.com/osxfuse/osxfuse/wiki/NTFS-3G. Note that to install NTFS-3G with homebrew, I had to install homebrew first (in the ntfs-3g instructions). I also had to install osxfuse which wouldn't install with homebrew so I had to download it from https://osxfuse.github.io/ and install it. I also needed to gain access to write /usr/local/lib/pkgconfig (this was reported by the ntfs-3g install. This was done with this command: sudo chown -R $(whoami) /usr/local/lib/pkgconfig.
Note that the first time I tried to mount the NTFS volume using NTFS-3G, it reported:
- “The NTFS partition is in an unsafe state. Please resume and shutdown Windows fully (no hibernation or fast restarting), or mount the volume read-only with the 'ro' mount option”
Obviously, I couldn’t resume and shutdown this Boot Camp/Windows volume. The problem is that the last time I was booted in this Boot Camp/Windows volume, I hibernated Windows. The hibernation file needed to be removed (and possibly other cleanup) before NTFS-3G would mount it. Luckily, NTFS-3G can help with this. To fix this, I first used an old command that didn’t work: sudo ntfs-3g /dev/disk0s3 /Volumes/NTFS -o remove_hiberfile. That command is outdated. The following command worked (after unmounting the volume first): sudo ntfsfix /dev/disk0s3
Once the the ntfs volume was “fixed”, here are the terminal commands I used to mount the volume and rename the files (including the backup):
- sudo mkdir /Volumes/NTFS
- sudo /usr/local/bin/ntfs-3g /dev/disk0s3 /Volumes/NTFS -o local
- cd /Volumes/NTFS/Windows/System32/drivers
- mv AppleSSD.sys AppleSSD.sys.save
- cd ../DriverStore/FileRepository/applessd.inf_amd64_6dbad8971c604bbc/
- mv AppleSSD.sys AppleSSD.sys.save
Convert DMG to IMG using Hdituil
Now that the Boot Camp files were modified and ready to go, I needed to restore the image to a new Boot Camp installation. To prepare for that, I needed to convert the DMG file to a IMG file in order to use DD to restore the Boot Camp volume.
I first made a copy of the DMG file because I wasn’t sure the hdiutil utility would save the source. It does save the source, so a copy wasn’t necessary. And the copy took up a LOT of space (250GB or so). I then ran “hdiutil convert -format UDTO -o BOOTCAMP.img "BOOTCAMP copy.dmg". This gave me a IMG file that I could use with DD to restore Boot Camp.
Prepping the original HDD in the iMac and installing Mojave
Before doing the final command to restore the partition, I needed to create a new Mac OS Mojave installation along with a new Boot Camp installation.
I’m not going into details here for installing a fresh installation of Mojave since it’s easy to google the instructions for installing Mojave from scratch using internet recovery. It’s interesting that Mojave has no issues with the missing/failed SSD (it installed fine to the HDD), but Windows complains about the failed SSD with a applessd.sys driver bluescreen.
Installing Boot Camp after a clean install of Mojave
Again, I won’t go into details here since a quick google search brings up instructions for installing Boot Camp. But note that you need to make sure that the partition you create for Boot Camp needs to be bigger than the saved IMG that was created with disktuil (above). And you don’t need a Windows activation key since you won’t be using this installation. You’ll be restoring the original Boot Camp volume.
Restore using the Boot Camp IMG file
The restore of Boot Camp uses DD. This is a destructive command. Use it only if you know what you’re doing. This will restore the original Boot Camp volume to the new Boot Camp volume (the new Boot Camp volume will be destroyed!). Here’s the command I used: sudo dd bs=1m if="BOOTCAMP.img.cdr" of="/dev/rdisk0s3". Note that this takes quite a while and there is no output of any kind until it’s done. It took probably an hour and a half for me at least.
That’s it! Restored and bootable Boot Camp volume
After it was done I was able to boot my old Windows volume from Boot Camp on a fresh installation of Mojave. And it's a good thing too, because I watched a video on trying to replace the SSD that is part of a fusion drive setup in an iMac and it's not simple at all. The SSD is behind almost every component inside the iMac, as opposed to the HDD which is right up front when you take off the screen on the iMac.
Conclusions
- Don't get a iMac with a Fusion drive! It just seems like a disaster waiting to happen.
- I really don't think this SSD drive should have failed yet. This is a late 2015 iMac bought new in 2016. An SSD drive shouldn't fail in two years with the way I was using it.
- Why is the SSD part of the Fusion drive setup behind almost every other internal component? This would make for a very difficult end-user repair. And on that topic, why isn't the HDD and SSD user replaceable on an iMac?
- And finally, I probably should start using TimeMachine or some other backup solution. I lost everything on my iMac OS volume.
See other posts on Mac/OS:
Buy iMac 27in on Amazon: Apple iMac (27" Retina 5K display, 3.4GHz quad-core Intel Core i5, 8GB RAM, 1TB) - Silver (Latest Model)