OSX Fresh Install – Storage Analysis

Now that I have a “fresh” and mostly empty MacBook Pro (MBP) running OSX Sierra, I am super curious to see how storage changes with time.  To understand that story, I’ll need to take a baseline measure now, while the system is relatively fresh, and compare later, assuming things degrade.

As I continue to use, or not use, my MBP, it will receive a variety of OS patches and application upgrades.  I might get a wild hair and install some wacky toolings, try to hook it up to peripherals, and in general, change the underlying performance flavor.  At some point, things will be so far gone…I’ll wonder: what happened?

Plan

I’ll use a variety of tools — some provided by Apple, a third party App (Daisy Disk), and some BSD command line action–to strike a baseline.  Then, if I feel like my laptop is rebooting too slowly, or web  browsing is taking longer then usual, I can repeat this analysis and look for any outliers.    I did not take this approach initially, and towards the end, I had no idea where things went astray, or why–I had no performance baseline to compare back to.

Apple’s Storage Tools

Apple, out of the box, does a pretty good job at exposing storage information in an easy to understand and digest fashion.  The status bar in any finder window will reveal the amount of free storage remaining:

Screen Shot 2017-07-19 at 5.25.41 AM

Overall hard drive stats can be found via the Finder “Info” capability (Command I), which reveals little more then what I already knew.:

Screen Shot 2017-07-19 at 4.05.55 AM

Bringing up System Information reveals a bit more detail:

Screen Shot 2017-07-19 at 5.21.02 AM

No surprise here, a bunch of photos and some other..yellow…stuff.  Clicking the manage button reveals a nice screen w/a bit more detail:

Screen Shot 2017-07-19 at 5.27.35 AM

This tells a story,  I suppose, and for a quick glance, it would be good enough.

But…when my original 500GB spindle was filling up, I was at a loss.   Unneccessary and redundant files can be masked as “Data” or “Other”, and it can be really tough to figure out what happened to all the free disk space.

In those days, I used the Daisy Disk app to quickly figure out where the space was going.  Let’s see what Daisy has to say about Mac HD.  

/Mac HD

Mac HD is the name of  the newly installed SSD drive. 110.5GB used out of 500GB.  Appx…no days in…and “poof”. 20% is toast.

Screen Shot 2017-07-19 at 3.52.35 AM
/MacHD

The main offenders are /Users (Photos), /private and /System.

Before we dive into those areas, 3.1GB in smaller objects. Interesting!  Let’s dive into smaller objects a bit.

Screen Shot 2017-07-19 at 5.51.07 AM
/MacHD/ Smaller Objects

Smaller Objects: /Library

Screen Shot 2017-07-19 at 5.51.33 AM
/Mac HD/Library

/Library is a litany of miscellaneous system objects, used by various functions of Mac OS.  Nothing too offensive.  Application Support is a bunch of stuff Apple is trying to sell me.

Smaller Objects: /usr

Screen Shot 2017-07-19 at 5.58.49 AM
/MacHD/usr

/usr is an element of the BSD subsystem.  The bulk of /usr is in /usr/share, which like /Library, is used as a common repository for BSD metadata.  I moused over to zoom in. Tokenizer, whatever that is, is taking the lions share.  Emacs takes 3x the space as vim…typical!  /usr/standalone has 95MB, which is mostly firmware related, bin is 50MB, lib is 49.2MB, libexce is 33.6MB, sbin is 18.4MB and local is 36k.  No real shocks in those areas.  Some apache and PHP stuff which maybe could be vaporized if needed, along w/emacs, but all in all, pretty tight.

Smaller Objects: /Applications

Screen Shot 2017-07-19 at 6.53.56 AM
/Mac HD/Applications

/Applications is where all OSX apps live.  Applications themselves are actually bundles of executables, shared objects, and other app specific metadata files–one OSX application is equivalent to an entire C:\Program Files\<App> directory.  The applications directory is 1.8GB, and is almost all Apple stock.  Interestingly enough, CyberDuck is 220MB.  I’d love to find a new ftp client, as I’m not a fan of how slow and bloated it feels.  I’d settle for a command line / midnight commander style, but … also don’t want to get too custom.

/private

Screen Shot 2017-07-19 at 4.15.24 AM
/MacHD/private/var

/private seems to be swapdisk related…maybe a richman’s tmp? The bulk of vm (virtual memory, I’m guessing) is composed of two files: sleeepimage (4.3GB) and swapfile0 (1.1 GB).  That’s not too bad, sleep image will mirror whatever I had in RAM at the time I put my laptop to sleep, and swapfile is OS paging data.  Both files are temporary.

Var/db

Screen Shot 2017-07-19 at 4.21.37 AM
/Mac HD/private/var/db

/db is a slightly different story. Dyld I’m guessing is dnyamic (link) loader, a shared object (.so) cache.  Will be interesting to see how that grows with time.  The remaining items seem like they could be cleared out, if space got to be a premium. I’d guess OSX has some kind of space management feature to periodically clean up errant dir…but..I might be guessing wrong there.

System/

100% of the System folder is currently inside the Library folder.

Screen Shot 2017-07-19 at 3.52.17 AM
/Mac HD/System/Library

A note on daisy disk itself: the way to read DaisyDisk is from the inner circle, out.

  • Pink – Directories are pink
  • Light Grey – Single file
  • Dark Grey – Bunches of smaller files

What I look for are large slices of Light Grey–low hanging fruit.

Each ring represents a given sub path, including each file and directory.   By starting on the inside, say /Speech/, I can quickly mouse over from inner ring to outer ring to see what files are using storage.

If I see a ring that has more and more subdivided rings going outward, I know there are a ton of directories with  lots of little files.  If I see a large light grey ring, then I know a large data blob is taking up space.

Looking at the graph,  I see rings of the exact same size and the outer most ring is light grey.  This tells me that a single large file is pushed deep into some given directory hierarchy.   If space is at a premium, these capabilities would be prime canditates for removal.    And in this case, that capability … I mouse over…is speech.  500MB is not a lot storage, until it is. Noted.

Users/

Screen Shot 2017-07-19 at 4.48.47 AM
/Mac HD/Users

The bulk of Users is me (no surprise), and the bulk of me is photos (again, no surprise).  That pink spike…that is interesting.  It appears to be my library and…downloads?

Screen Shot 2017-07-19 at 4.52.34 AM
/Mac HD/Users/Me

91GB in photos, I’m cool with that. 500MB in downloads, some installers I need to clear out. 5MB in Music! Wth is that about.  I haven’t done anything w/Music. How could that be?

Music/

Screen Shot 2017-07-19 at 4.51.54 AM
/Mac HD/Users/Me/Music

I’m good. Music includes iTunes, and cloud purchases, and whatever that is.

Library/

Screen Shot 2017-07-19 at 3.52.57 AM
/Mac HD/Users/Me/Library

1GB in Library.  This is one to watch, I think.  The bulk of Library, right now, are caches, and the bulk of those caches are WebKit.

Storage Analysis

Some flavors of Linux fit the entire kit and kaboodle in under 400MB.  That’s not a fair comparison–renowned ‘lite weight’ Linux distribution PuppyOS is not even close to Apple in terms of usability.  Logging into wifi everytime you boot up? Yup, that’s Linux.

All in all, a solid 7GB  is dedicated to OSX Sierra.  Out of 500GB?  It’s not noticeable.  Compared to 400MB? It’s pretty big.  Still, what do I care? It’s not like these GB are doing anything for me, unused.

What I am thinking about…isn’t now, when the OS is fresh.  What will it look like in the future?

Building a file catalog

All the pictures above are nice and will help remind me of the story down the road…but the detail isn’t there.  Time to build a file catalog, which I’ll compress and whisk off to a NAS:

$ sudo find -x / -type file -ls -not \( -path "/Volumes/*" -prune \) > ~/MacHD_File_Catalog_20170719.txt &

I need to exclude /Volumes so I don’t re-scan the HD or scan remotely mounted drives.
To track progress, I run a tail loop to figure out where it is in the volume scan:

$ while [1]; do tail -n 1 MacHD_File_Catalog_20170719.txt | cut -d '/' -f2,3; sleep 10; done 
/Applications/blah
/Library/blah
[1]+ Done
/usr/standalone
/usr/standalone

I hit ctrl+c to stop the tail.    It only ran for a few minutes so I probably could have simplified the whole bit and avoided the & and tail loop.  Caveat emptor.

File Catalog Results

The resulting  file catalog is 96M in size and contains some 514k files.  It compresses down to 7MB.  I quickly scan through it but there isn’t anything that jumps out at me.  It’s good enough for now, looks relatively complete.

Later on, if things get dicey, I’ll rerun these commands and use a combination of grep & file difference utilities to see what has changed. Good times, eh?

 

 

Categories: OSX

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s