Ubuntu CD to FLAC

Oddly enough–or maybe not–Ubuntu doesn’t come pre-installed w/abcde (a better cd encoder), my favorite method to rip & encode music from a cd to my hard drive for discless playback. That’s cray cray!

Why abcde?

abcde is a command line bash script / wrapper around a variety of utilities that do the actual heavy lifting. It is very mature at this point–it just works, and works well.

It has a configuration file (.abcde.conf) that can be used to tweak outputs so that abcde rips & encodes just the way I want it to work, as well as custom hooks (like copying automatically to a NAS, for example). It’s not as pretty as other options, and isn’t the best for cds that don’t already have metadata; iTunes is killer when it comes to ‘perfect’ encoding in terms of track names & album art. Since Apple sells music, they want to know what you’re ripping, so they are more then happy to hook you up.

IMG_0291

But…iTunes has no FLAC! Ubunutu itself has other windowed apps that can act as a fall back for when the going gets rough (which it can, periodically).

Where abcde runs supreme is fire+forget simplicity: throw a cd in, drop to terminal, and one ‘abcde’ later…a discless, lossless version of the cd is ready for playback within a few minutes.  The disc ejects when it’s done…pop in another, hit the up arrow key, press enter…and repeat!

Why FLAC?

IMG_0288
MP3 compresses audio by a ridiculous degree, which is great for realtime streams, or 20 years ago, when 1GB was “woah”: a 600MB CD was a huuugge amount of space back then. An MP3 equivalent was around 60MB. The big drawback of MP3s is that they do so by changing the sound, betting we won’t notice…and…honestly, it turns out, we do.

However, most of time we don’t care, so it’s all good. These days, though, there’s really no reason to throw out part of the sound just to save on storage–especially if we are buying cds. FLAC compresses lossleslly…all the original sound data is stored, it’s just stored in a smaller package.  As a result, any FLAC track can be restored to it’s original cd form, back and forth, endlessly.

IMG_0296
MP3, like progressive jpgs, are garbage.  The beauty of the original is lost.

MP3s–not so. An MP3 encoding forever changes the original sound; the original wave ‘loses’ part of it’s data, replacing it with an algorithm that approximates what was there…and while true, the music track is now smaller–this approximation resulted in permanent changes to the original sounds, which is why MP3s are known as lossy encoding: part of the music is lost. Maybe we don’t hear it all the time (excellence vs perfection), but..you can run tests yourself. I have, and I can hear the difference–and my ears are not golden.

Note: if you have a track or album in MP3, don’t bother to re-encode from MP3 to FLAC…the damage has already been done by the MP3 algorithm, and it’s irreversible. Better to re-rip from CD to FLAC. If what you have is Mp3s, that might be good enough, esp for live or special events where maybe that’s all there is. It’s not that MP3s are evil. However…to take a CD recorded in a million dollar facility by professionals dedicated to their craft, and chop it up as MP3, it just seems counter productive: why not buy them as MP3s to begin with? To each their own though…

Storage is cheap–if I’m buying a cd, why would I want to mangle the sound just to listen to it? I could do that for free via youtube or other streaming services.

IMG_0290
Sci-fi: When robots are cheaper then USB cables.

So why compress at all, if storage is so cheap? Good question. There’s a ton of dead space in raw WAV forms. FLAC compresses WAV files @ a fraction of the speed of MP3s–an entire CD can be encoded as FLAC in very little time. So while space is cheap (no reason to encode), FLAC encoding can also be very fast (no reason not to encode).

All encoding is trading storage vs time vs quality.  For an entire CD, mild FLAC encoding is perfect/lossless, saves on average around 30% vs uncompressed, at an expense of around 15 seconds; I’m good with this tradeoff. Apple has its lossless encoding standard but…its just too hard to play back Apple lossless across *all* devices. FLAC is more universally accepted (by everyone except…Apple, of course…for now)–and can store metadata tags. Or, at least, store then more easily. Genre, album artwork, tag details–all the usual standard you’d expect. WAV…not so much.

Bottom line, if you rip 100 CDS for discless playback w/out FLAC, w/FLAC you can rip 130 and get the pretty pictures/genres….for an extra half hour of time.

What kind of computer will I need?

Some good news–not many horses will be needed. A raspberry pi could do the trick.   The flac ripping and encoding process is very lightweight–the bottleneck will be your cd-drive: there are physical limits to how fast the cd can spin, and those limits have been reached. For an audiophile product, it takes remarkably little. Now playing back those digital files…

Source Reading

abcde has bounced around a bit…latest home (as of 2017) is:
https://abcde.einval.com/wiki/

Ubuntu forums had a relatively decent step-by-step post:
https://askubuntu.com/questions/867593/is-there-a-way-in-ubuntu-to-rip-a-cd-to-flac-and-add-replaygain-tags-automatical

Great! Why not just stop here? A few things caught my eye, as they usual do.

A few gotcha’s & tweaks from the Ubunutu post:
No need for metaflac. The flac package apparently now includes metaflac, so no need to specify on command line.

Needs a cd text library. icedax can read metadata information right from the cd…which, as a fallback, is a good thing.

Too much work! For me, I need the rip and encode process to be lights out. Perhaps I’ll get some crazy metadata, or album covers that don’t make sense. All good, I’ll correct later.

ReplaGain isn’t for me. ReplayGain (https://en.m.wikipedia.org/wiki/ReplayGain) analyzes audio so that all tracks have the same average peak volume–leaving around 14db of headroom (on avg).  While I understand why this might be good, at some level it’s (albeit) non destructively changing the source material, and so for me, just not something I need. Plus…I’m not sure that my playback devices respect it.

FLAC encode value is too high. Measures via http://z-issue.com/wp/flac-compression-level-comparison/ reveal a conventional wisdom: FLAC offers lossless compression values ranging from 0 (least amount of compression) to 8 (most amount of compression). Compression comes at a cost–encode time. The apparent tipping point between compression and encode time is 3. In a “time is money” of world, not storage…the time savings do indeed outweigh the storage benefit (for me): it’s not worth 660% more time to save 1% more space…for me.

From the excellent z-issue site:
Uncompressed WAV: 128 seconds to rip – 233.6 MiB resulting file size

QualityEncode (sec)Size (MiB)Ratio (%)Enc + (sec)Enc + (%)

FLAC 0 3.531 174.6 74.7% 0.000 0.00%
FLAC 1 3.721 173.5 74.3% 0.190 5.38%
FLAC 2 4.658 173.2 74.1% 1.127 31.92%
FLAC 3 5.255 165.0 70.6% 1.724 48.82%
FLAC 4 6.584 163.8 70.1% 3.053 86.46%
FLAC 5 9.112 163.4 69.9% 5.581 158.06%
FLAC 6 9.130 163.4 69.9% 5.599 158.57%
FLAC 7 19.475 163.3 69.9% 15.944 451.54%
FLAC 8 28.846 163.1 69.8% 25.315 660.30%

Quality: The FLAC compression level used
Encode (sec): The time it took to encode the song
Size (MiB): The resulting FLAC file size (rounded to tenths of a Megabyte)
Ratio (%): FLAC file size as a percentage of the original uncompressed WAV
Enc + (sec): The additional time required to encode as compared to FLAC 0 (in seconds)
Enc + (%): The additional time required to encode as compared to FLAC 0 (as a percentage of increase)

abcde doesn’t fall back well.. I’ll need to hack on abcde so that it falls back from musicbrainz to cddb to cd text. If that doesn’t pan out..I want it to error out. Maybe beep beep beep and halt…because…untitled albums kinda suck. I’d rather know!

Install

Install is dead simple:

sudo apt-get install abcde cdparanoia flac imagemagick glyrc icedax

Breakdown of all the bits:

Package Description
abcde Our fearless champion of disc ripping & encoding; it includes tooling for CDDB/musicbrainz meta-data access too.
cdparanoia A best odds binary for reading WAV data off even your most scratched CDs. This was a much bigger deal in Linux 20 years ago…where exposing your cd drive to the operating system was a major feat unto itself.
flac A library for encoding/decoding WAV to FLAC.
imagemagick an image manipulation library–used to make sure album artwork is right size & image format.
glyrc Album artwork search tool.
icedax Used for cd text retreival when online databases suck

How this all fits together..

IMG_0292
Friend computer inspires a different kind of paranoia.

abcde uses cdparanoia to figure out what your cd is, asks musicbrainz for artist, album & track metadata, uses that metadata w/glyrc to find an image, imagemagick to make sure it’s the right size, then cdparanoia to rip…w/flac handling the encode. Everything happens in a cached directory off your home (~), and moved to your Music directory (~/Music) when done. You can ctrl+c to and restart, and abcde will resume, handy for those of us with aging cd drives that might drift off for a snooze mid rip.

Config

The secret sauce to making abcde run is its configuration file; after you install, copy and paste the below and you’re done. If you don’t, abcde will still work, it just takes more command line options to make it work…and who wants to remember what those are?  Note it’s all just bash!  Any command you need can be woven in, if you need to.

~/.abcde.conf

# -----------------$HOME/.abcde.conf----------------- #
#
# A sample configuration file to convert music cds to
#       FLAC using abcde version 2.7.2
#
# Modified for AskUbuntu http://askubuntu.com/q/867593/57576
#  to allow ReplayGain as well as Album Art retrieval...
#
# http://andrews-corner.org/linux/abcde/index.html
# -------------------------------------------------- #

# Encode tracks immediately after reading. Saves disk space, gives
# better reading of 'scratchy' disks and better troubleshooting of
# encoding process but slows the operation of abcde quite a bit:
LOWDISK=y

# Specify the method to use to retrieve the track information,
# the alternative is to specify 'musicbrainz':
CDDBMETHOD=musicbrainz

# Make a local cache of cddb entries and then volunteer to use
# these entries when and if they match the cd:
CDDBCOPYLOCAL="y"
CDDBLOCALDIR="$HOME/.cddb"
CDDBLOCALRECURSIVE="y"
CDDBUSELOCAL="y"

# Specify the encoder to use for FLAC. In this case
# flac is the only choice.
FLACENCODERSYNTAX=flac

# Specify the path to the selected encoder. In most cases the encoder
# should be in your $PATH as I illustrate below, otherwise you will
# need to specify the full path. For example: /usr/bin/flac
FLAC=flac

# Specify your required encoding options here. Multiple options can
# be selected as '--best --another-option' etc.
# Overall bitrate is about 880 kbs/s with level 8.
FLACOPTS='-s -e -V -3' 

# Output type for FLAC.
OUTPUTTYPE="flac"

# Options to pass to metaflac for ReplayGain:
#FLACGAINOPTS="--add-replay-gain"

# The cd ripping program to use. There are a few choices here: cdda2wav,
# dagrab, cddafs (Mac OS X only) and flac. New to abcde 2.7 is 'libcdio'.
CDROMREADERSYNTAX=cdparanoia            

# Give the location of the ripping program and pass any extra options,
# if using libcdio set 'CD_PARANOIA=cd-paranoia'.
CDPARANOIA=cdparanoia
CDPARANOIAOPTS="--never-skip=40"

# Give the location of the CD identification program:
CDDISCID=cd-discid            

# Give the base location here for the encoded music files.
OUTPUTDIR="$HOME/Music"               

# The default actions that abcde will take.
ACTIONS=cddb,getalbumart,playlist,read,encode,tag,move,clean

# Decide here how you want the tracks labelled for a standard 'single-artist',
# multi-track encode and also for a multi-track, 'various-artist' encode:
OUTPUTFORMAT='${OUTPUT}/${ARTISTFILE}-${ALBUMFILE}/${TRACKNUM}.${TRACKFILE}'
VAOUTPUTFORMAT='${OUTPUT}/Various-${ALBUMFILE}/${TRACKNUM}.${ARTISTFILE}-${TRACKFILE}'

# Decide here how you want the tracks labelled for a standard 'single-artist',
# single-track encode and also for a single-track 'various-artist' encode.
# (Create a single-track encode with 'abcde -1' from the commandline.)
ONETRACKOUTPUTFORMAT='${OUTPUT}/${ARTISTFILE}-${ALBUMFILE}/${ALBUMFILE}'
VAONETRACKOUTPUTFORMAT='${OUTPUT}/Various-${ALBUMFILE}/${ALBUMFILE}'

# Create playlists for single and various-artist encodes. I would suggest
# commenting these out for single-track encoding.
PLAYLISTFORMAT='${OUTPUT}/${ARTISTFILE}-${ALBUMFILE}/${ALBUMFILE}.m3u'
VAPLAYLISTFORMAT='${OUTPUT}/Various-${ALBUMFILE}/${ALBUMFILE}.m3u'

# This function takes out dots preceding the album name, and removes a grab
# bag of illegal characters. It allows spaces, if you do not wish spaces add
# in -e 's/ /_/g' after the first sed command.
mungefilename ()
{
  echo "$@" | sed -e 's/^\.*//' | tr -d ":><|*/\"'?[:cntrl:]"
}

# What extra options?
MAXPROCS=2                              # Run a few encoders simultaneously
PADTRACKS=y                             # Makes tracks 01 02 not 1 2
EXTRAVERBOSE=2                          # Useful for debugging
COMMENT='abcde version 2.7.2'           # Place a comment...
EJECTCD=y                               # Please eject cd when finished 🙂
INTERACTIVE=n                           # Run lights out

Modify abcde

First, let’s stash a copy and edit that:

$ sudo cp /usr/bin/abcde /usr/bin/my_abcde
$ sudo vi /usr/bin/my_abcde

Now, to improve abcde for my own fiendish needs. I like musicbrainz better, however it’s not perfect. So I’m gonna be a bit lazy here…I’ll specify musicbrainz in my .abcde.conf file, and then if it can’t find an album, I’ll invoke cddb. If the album can’t be found in cddb, abcde will automatically check the cd for metadata, so we’re good there.

#around line 4066...

if [ "$DOCDDB" = "y" ]; then
        # start with a sane default:
        CDDBLOCALSTATUS=notfound
        if [ $CDDBUSELOCAL = "y" ]; then
                do_localcddb
        fi
        if checkstatus cddb-choice > /dev/null; then
                :
        else
                if [ "$CDDBLOCALSTATUS" = "notfound" ] ; then
                        case "$CDDBMETHOD" in
                                cddb)
                                        do_cddbstat
                                        do_cddbquery
                                        do_cddbread
                                        ;;
                                musicbrainz)
                                        do_musicbrainz
                                        ;;
                        esac
                fi
                #---- copy below ----
                #const void 2017-07 begin
                #TODO add check for musicbrainz
                if [ "$UNKNOWNDISK" = "y" ] ; then
                        vecho "No musicbrainz, falling back to cddb."
                        UNKNOWNDISK=n
                        rm -f "$ABCDETEMPDIR/cddbchoices"
                        do_cddbstat
                        do_cddbquery
                        do_cddbread
                fi
                #const void* 2017-07 end                                                                                                                                                              
                #---- copy above ----

                CHOICE=$(checkstatus cddb-choice)
                if [ "$CHOICE" = 0 ] ; then
                        # We don't have any information at all; try to fall back
                        # to CD-Text for basic information
                        vecho "No CDDB information found, trying cdtext from the CD"
                        do_cdtext
                fi
        fi
        do_cddbedit

Execute

$ eject                                 # Systemically eject tray
$ my_abcde                                 # Rip & Encode

IMG_0297

Once abcde is done, you can find the results in ~/Music. It’s good to tweak the abcde.conf file to do what you need it do…that’s what makes Linux such an expressive operating system–automation tooling that does exactly what you want, exactly.

IMG_0294

uage

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s