My Ubuntu Evil Empire Escape (UbuntuMUEEE)

mediame -syncdevices

The mediame -syncdevices command synchronizes any connected devices to their staged device content.

Each device has a section within the mediame.ini file. The devicepath parameter defines the path to the directory to be synced. For a Cowon iAudio 9 on Ubuntu the path is "/media/iAUDIO 9".

This command does the following for each device:

• Identifies if the device is attached by checking the devicepath to see if it is a directory.
• Going thru each directory in the ~/mediame/staging/<devicename> directory. For each such directory it:
  1. Produces the set of .mp3 files under the directory.
  2. Checks whether the corresponding directory identified by the <contenttype>_location parameter exists on the device and produces the set of .mp3 files under that directory.
  3. Deletes files in 2 but not in 1 deleting empty directories deleting any directories that become empty.
  4. Copies files in 1 but not in 2 into the corresponding directory on the device creating directories as needed.
  5. The result being that exact copies of the staged content is set in place on the device without touching any other directories on the device.

No provision is currently in place, other than putting error messages in the trace log, for copy failures including device full copy failures.

MontyHigh, www.worldofwallstreet.us

 

mediame --podcastimport

The mediame --podcastimport command imports the podcast type .mp3 files that are sitting in the subdirectories of the ~/mediame/newpodcasts directory. The idea is that podcasts are (at least for now) manually downloaded and copied into the appropriate directory (one per podcast). The --podcastimport command then:

• Sets the Artist, Album and Genre .mp3 tags based on the mediame.ini file settings and the directory the file came from. The album title is the has the current year in the format YYYY_ prefixed to the directory name the file.
• Adjusts the title, prefixing the date of import to the title in YYYYMMDD_ format.
• Imports the adjusted file as if it were being imported with a --import command.
• Keeps a copy of the imported file in the subdirectory (named YYYMMDD) under the ~/mediame/recentpodcasts directory of that day that the --podcastimport command is executed. This directory is used by the --stage command for staging the most recent N days of podcasts.
• Removes /recentpodcasts subdirectories older than the mediame.ini [newpodcasts] recentpodcastdays configuration parameter.

The result is that keeping an mp3 player loaded with the most recent N days of podcasts becomes a simple matter of downloading the podcasts into the appropriate directory and runing a TBS command (--sync command?) which does the equivalent of a --podcastimport command, a --stage command and the synchronizing the .mp3 player to the files that have been staged.

The [newpodcasts] section of the mediame.ini file takes the following keyvalue pairs:

directories=<csv list of directory names>, mediame creates a directory under newpodcasts entries in this list (and removes any newpodcasts subdirectories which do not have an entry in the list. The idea is to have one directory for each different podcast.

artists=<csv list of podcast artists>, has one entry for each of the directories CSV list entries. An artists csv list entry provides the Artist tag value to be used for podcast .mp3 files imported from the corresponding directory.

genres=<csv list of podcast genres>, has one entry for each of the directories CSV list entries. A genres csv list entry provides the genre to be used for .mp3 files imported from that directory.

recentpodcastdays=<int>, has the number of recent podcast directories that should be maintained where older directories are purged to make room for newer directories.

MontyHigh, www.worldofwallstreet.us

"mediame -stage" Command

The "mediame - stage <devicename>" file creates a "staged" copy of the directory structure to be loaded into the specified device. The "staged" directory structure resides in the ~/mediame/staging/<devicename> directory. Device name must be alphanumeric.

The contents of the [alldevices] and [<device name>] sections of the ~/mediame/conf/mediame.ini define what mp3 files are to be staged. The superset of the two sets of files are staged. Initially, the following key value pairs are supported within these sections:

<content-class>_artists=<csv list of artists or '*'>, where <content-class> is one of: "popmusic", "podcast", "recentpodcast", "book", "classicalmusic". An '*' value indicates that all content of the class is to be staged. Otherwise a csv list of the artists (or authors or composers) to be staged is given where every album and every track by that artist is staged.

<content-class_genre_startswith=<csv list of strings>, where <content-class> is one of: "popmusic", "podcast", "recentpodcast", "book", "classicalmusic" and where any track whose genre starts with one of the entries in the csv list is a candidate for staging.

When a content-class has both an _artists ini parameter and a _genre_startswith ini parameter an mp3 file if it matches either the _artists criteria or the _genre_startswith criteria.

<content-class>_location=<relative path>, where <content-class> is one of: "popmusic", "podcast", "recentpodcast", "book", "classicalmusic" and where <relative path> defines where under the ~/mediame/staging/<devicename> the content-class's mp3 directory structure should be placed. The default value for a content-class is the content class-name, e.g. "book" artists are placed in the ~/mediame/staging/<devicename>/book directory.

<content-class>_genreoverride=<genre_name>, where <content-class> is one of: "popmusic", "podcast", "recentpodcast", "book", "classicalmusic" and where <genre_name> is the string that all of that content-class's tracks are assigned. This genreoverride mechanism provides a way for the Cowon iAudio9 to be given a command to shuffle all of the content-class's tracks. One selects the genre and shuffle and all the tracks in that genre shuffle. The reassignment occurs at the point of staging (prior to the tracks being sync'ed to the actual device). The default value of empty string ("") indicates that no override operation should take place.

One additional parameter controls how many directories of recent podcast days are copied to the device:

recentpodcastdays=<1..1000000> gives the number of recent podcast day directories to be staged. There is one directory for each of the most recent days that a mediame -P command is executed. This controls how much space is occupied by recent podcasts.

The initial values I'm implementing is for the following .ini content:

[alldevices]
popmusic_genre_startswith=MusicNPop
recentpodcast_artists=*
#popmusic_artists=*

[i9]
popmusic_location=Music
popmusic_genreoverride=MusicN
recentpodcast_location=PodcastRecent
recentpodcastdays=14
book_location=Book

MontyHigh, www.worldofwallstreet.us

"mediame -import" MediaMe 0.01 Media Import

The command line "mediame --import <directory path>" or "mediame -I <directory path>"imports all of the supported media (.mp3 files for now) from the provided directory (and its subdirectories) into the Canonical directory structure without overwriting duplicate files. "mediame --import" leaves in the logs directory a filenamed "importduplicates" with summary information for each duplicated file.

This command is useful for initially populating the Canonical directory structure and for importing groups of files which already have their "tags" (e.g. ID3 tags) properly set.

The basic classification of .mp3 files during a --import operation is defined by the ~/mediame/conf/mediame.ini file's [genreimport] and [artistimport] sections.

This [genreimport] section optionally contains the following keys:

• popmusic=<csv list of genre starts with strings> where any .mp3 whose genre begins with one of those strings is classified as a popmusic song.
• podcast=<csv list of genre stats with strings> where any .mp3 whose genre begins with one of those strings is classified as a podcast.
• book=<csv list of genre stats with strings> where any .mp3 whose genre begins with one of those strings is classified as part of a book.
• classicalmusic=<csv list of genre stats with strings> where any .mp3 whose genre begins with one of those strings is classified as part of a classical music composition.

This [artistimport] section optionally contains the following keys:

• popmusic=<csv list of artist starts with strings> where any .mp3 whose artist begins with one of those strings is classified as a popmusic song. A value of "*" causes an otherwise unclassified .mp3 file to be classified as a popmusic song. How an otherwise unclassified .mp3 file is classified is undetermined when more than one key has a value of "*".
• podcast=<csv list of artist starts with strings> where any .mp3 whose artist begins with one of those strings is classified as a podcast. A value of "*" causes an otherwise unclassified .mp3 file to be classified as a podcast. 
• book=<csv list of genre stats with strings> where any .mp3 whose artist begins with one of those strings is classified as part of a book. A value of "*" causes an otherwise unclassified .mp3 file to be classified as part of a book. 
• classicalmusic=<csv list of genre stats with strings> where any .mp3 whose artist begins with one of those strings is classified as part of a classical music composition. A value of "*" causes an otherwise unclassified .mp3 file to be classified as part of a classical music composition.

"mediame --import" skips the importing of files which are unclassified and leaves a file named "importunclassifiedlist.csv" with the summary information about files that were  skipped.

 I have used my own system for classifying music within itunes. To import my iTunes library these sections are set as follows:

[genreimport]

popmusic=Music

podcast=Podcast

book=z

[artistimport]

book=z

classicalmusic=x

MontyHigh, www.worldofwallstreet.us

MediaMe 0.01 Canonical Media Directory Structure

MediaMe 0.01's Canonical Media Directory Structure is as follows:

mediame - typically exists as the Home level of the directory structure. It holds the python executables and the following directories:

conf - where configuration files are held. The only configuration file currently in use is mediame.ini.

logs - each run of mediame creates a log file with the name mediame<YYYYMMDDHHMMSS>.txt where YYYYMMDDHHMMSS.txt has the date and time when mediame was started. The oldest files are purged as needed to keep the number of log files under a 100 file limit. Other operation specific log files are also stored in this directory.

newpodcasts - where podcasts are downloaded. Podcasts typically have to have their MP3 tags overwritten as part of the importing process. Each subdirectory in newpodcasts has a corresponding configuration file entry designating what the tags should be for files imported from the subdirectory. There is just one level of new podcast subdirectories.

recentpodcasts - where a copy of recent podcasts are kept (another copy of the files are found under the media/podcasts directory treee). recentpodcasts has a directory for a limited number of recent dates with the directory name formatted as YYYYMMDD. Each such directory holds the podcast files that were imported on corresponding day of the month. This facilitates the staging of the N most recent days's podcasts for synchronization with a device.

media - the directory structure into which media is imported. The directory tree underneath media is as follows:

popmusic - where all imported pop music resides.

<artist> - one directory for each pop music artist.

<album> - a directory for each of the artists albums. The individual "songs" aka tracks reside in these directories.

podcast - where all imported podcasts reside.

<artist> - one directory for each significant different artist. The artist is ordinarily the "host" of the podcast but selected "guests" have their own artist directories when they are interesting enough that its worth following their work directly. Pseudo-artists, e.g. "Misc Interview" are used for one-of-a-kind interviews where neither the interviewer or interviewee are significant enough to warrant their own directory.

<year>e.g. 2012 takes the place of the <album> for podcasts. The individual podcasts appear here with their title (beginning with the date encoded for alphabetical sorting of tracks by date) encoded in at the beginning the file name.

book - where all imported audio books reside.

<author> - holds the author of the book.

<album> - typically a CD's worth of material. Album names are organized so that the albums within a book appear in alphabetically sorted order. The track file names are similarly organized to begin with their track number so that track numbers appear within an album in sorted order.

classicalmusic - where all imported classical music resides.

<composer>

<album> Where individual media files appear. File names begin with the track number so that files appear within the file in sorted order. 

staging - the directory structure which has a subdirectory for each player. A player's subdirectory holds a complete image of what is to be synchronized into the player with files and subdirectories organized as the player requires.

MontyHigh, www.worldofwallstreet.us

MediaMe 0.01 Requirement Specification

MediaMe (Media My Ubuntu Evil Empire Escape), MediaMe is a Python command-line program that is intended to allow me to escape iTunes grasp. MediaMe version 0.01 is intended to provide the following:

• A standard (canonical) directory structure for storing all of the media for my portable media players. Version 0.01 supports just the Cowon iAudio 9 and .mp3 files.
• A convenient way for importing additional media files of the general varieties that interest me (in order of importance):
  • Podcasts - Individually downloaded spoken audio programs, interviews, etc. which may or may not follow the official Podcast RSS based retrieval protocol - my top sites sourcing this right now include King World News, Financial Sense Online, Solari Report, C-Realm Podcast and No Agenda show. I typically want all of the most recent podcasts on my player sorted in a fashion where I can easily select from among the most recent podcasts.
  • Audio Books - with no DRM, typically ripped CD-sourced (or even cassette tape sourced) audio books - the key things that differentiate a book from a podcast are:
    • a book typically extends across multiple MP3 files while a Podcast is self-contained in a single MP3 file.
    • I typically want the whole book (or multiple hours of the whole book) on a player at a time, while I typically only wants the most recent Podcasts on a player.
    • I have a large library of audio books and want to be able to conveniently select which books (or which CD-length sections of a book) are loaded into my players.
  • Pop Music - I usually like to be able to "shuffle" among pop music. The iAudio 9 has some special constraints that must be worked around to support this. I currently have players with 16 GB of storage or more and only 8 GB of pop music. I thus want to load all of pop music I have that I actually like into the players.
  • Classical Music - Classical music is similar to audio books in that compositions typically occupy multiple tracks (e.g. Handles Messiah) and I want to select which compositions get loaded into the player. When I listen to classical music I don't want to "shuffle" all tracks but rather listen to complete compositions.
• A convenient way to specify what media gets loaded into a player.
• A convenient way to synchronize a player to have it hold just its specified media. 

MontyHigh, www.worldofwallstreet.us

UbuntuMUEEE 005: Getting Started With The Ruby Programming Language

Here's the steps for getting Ruby installed on your Ubuntu PC. By the way, this entire blog is written assuming Ubuntu 11.10.

1. Open up a terminal window.
2. Enter "sudo apt-get install ruby1.9.1." and entire your password when prompted.
3. Load up the MP3 tag class by entering "sudo gem install mp3info".
4. Use the Ubuntu software center to install JEdit.
5. Start JEdit, click the Plugins menu item and install the Ruby plugin.

You are now ready to start some Ruby programming.

MontyHigh, www.worldofwallstreet.us

Download ZFridayTheRabitSleptLate_Kemelman01

 

Download ZFridayTheRabitSleptLate_Kemelman02

 

Download ZFridayTheRabitSleptLate_Kemelman03

Download ZFridayTheRabitSleptLate_Kemelman04

Download ZFridayTheRabitSleptLate_Kemelman05

 

 

.

UbuntuMUEEE 004: Moving The MP3 To Ubuntu

Having completed UbuntuEEE 003, I am now ready to move my MP3 audio files to Ubuntu. There may be other (better) ways to do this, but this post records how I did it.

I chose to use an external USB disk drive because I have such a large amount of MP3 audio and because I want to have a backup copy (you do backup your PC, don't you?).

1. Determine how many GB of content is to be moved:
   
   1. Launch Windows Explorer.
   2. Open My Music
   3. Open iTunes
   4. Right-click iTunes Music and select Properties. Here's what I got (that's a lot of podcasts).
   5. Record the size and number of files and number of folders (in my case 100GB, 20,042 and 2,383).
2. Verify that your Ubuntu PC has enough disk space for this amount of Music.
   1. TBS.
3. Plug your external USB disk drive into a USB 2.0 port on your PC. If you don't plug it into a USB 2.0 port the copying of files to your disk drive will be pretty slow: around 1 MB/sec or 3.6 GB / hour. For my 100 GB this means around 30 hours.
4. Verify that your USB disk drive has suffient space to hold your iTunes music. Open another windows explorer, open My Computer, click the USB drive, right click and select properties. Compare the available space with the recorded size. 
5. Wait until you can let your Windows PC run for an extended period of time.
6. Open the USB disk drive in its windows explorer and create a folder to hold the MP3 to be copied. I've named mine iTunesCopy. Open the folder.
7. Drag and drop the iTunes Music folder from the Windows Explorer from step 1 into the iTunesCopy folder in the external USB disk drive. Take a long coffee break (perhaps hours) while this runs.
8. Connect your USB disk drive to a USB 2.0 port on the Ubuntu PC, boot up the PC and log in.
9. Verify, using Ubuntu's Disk Usage Analyzer, that sufficient disk space is available.
10. Do a drag and drop to copy the iTunes Music folder from the USB disk driver into your Ubuntu login directory.

You've now got your entire iTunes music libary transferred to Ubuntu.

UbuntuMUEEE 003: Converting DRMed ITunes Store Music To MP3 Format

While I have something like 300GB of audio books, podcasts and music in MP3 format, I find that I have bought more than 300 songs from the iTunes store. These, being in Apple proprietary format and being protected by iTunes Digital Rights Management (DRM) cannot be played on non-Apple devices.

The first step for escaping the Evil Empires is to get all of one's content into a format that can be played by non-Evil Empire players.

This post provides the solution that I'vve used for converting those songs to MP3 format. There is probably a better way, but I could not easily google it. This way is pretty time-consuming. Leave a comment to help other fellow-travelers if you know a better way.

Here's the way I'm doing it:

1. Open iTunes.
2. Create a "Purchased" smart play-list if you don't already have one (see image below). Click File->New Smart Playlist.
3. Click File->Edit Preferences. Select the General tab and click on the Import Settings button which brings up the Import Settings dialog box. Set select the MP3 encoder and pick a nice high-quality (see figure below). I'm going with 192 kbps because I have big MP3 player and this only 300 songs and I want them to sound as close to what I bought from iTunes as possible.
4. Create a temporary ordinary playlist. Mine is named aaaburn. The aaa prefix makes it appear at the beginning of the iTunes list of playlists.
5. Drag and drop a group of around 12 songs from the Purchased play list into the aaaburn playlist.
6. Put a writeable CD into your PC's CD-ROM drive. I'm going to use up that old bunch of write-once CD-Rs I have lying around. You could probably get away with using a single read-writable CD (CD-RW), but I don't have one so I'm going to use what I have.
7. Right click on the aaaburn playlist and select "Burn Playlist To Disc".You'll see dialog box come like the following. Set it up as shown and click ok.
8. When the disc write is complete a CD icon will appear along the left hand side of iTunes (see image below).
9. Click that icon and then select all of the songs on the disk (click on the first song and then, holding down the shift key, click on the last song). Right click and select Convert To MP3.
10. When the import is complete, right click on the CD icon and select Eject and remove the disk (this step can be replaced with a step that erases the disk if you are using an CD-RW. I don't know how to do that). 
11. Remove all of the songs from the aaaburn playlist.
12. Repeat steps 5 to 11 with a different set of songs until all of the Purchased songs have been imported as MP3s.
13. Check the Recently Added playlist and you'll see all of your converted songs. Click in Music on one of the Artists whose songs you have purchased. You will see two copies of the song. 

Now that all of my audio content is in MP3 format I'm ready to move it over to Ubuntu. That will be my next post.

MontyHigh, www.worldofwallstreet.us

UbuntuMUEEE 002: Briefly Why I Want To Escape The Evil Empires

Why I want to escape Apple is easy: They want to lock me in:

• Their content comes with their Digital Rights Management and Apple formats so that it only plays on Apple devices. The universe of Apple devices is too small and changes from year to year. For example, I wanted an audio player that was nice and small, easy to use, could be customized to do what I wanted and had alot of flash space (e.g. 32 or 64 GB). Apple doesn't make that. So, I can't use iTunes.
• Their devices and programs are not open and cannot be improved and everyday tasks can't be automated.
• Their devices do not allow me to easily write my and use my own software.

Microsoft wishes they could lock people's content like Apple does, but they don't seem to have gotten away with it. I want to escape Microsoft because of its security issues. I don't like having to pay an annual fee for vius protection. I don't like owning PCs that become obsolete as soon as Microsoft discontinues support.

Finally, I think the whole Free Software idea (GNU, LINUX, open source, etc.) is cool and I'd like to be part of it.

I'm going with Ubuntu because when I picked my LINUX flavor, Ubuntu seemed to be the most usable LINUX desktop variety. I'm not a LINUX expert and I'd like to get LINUX things to work as easily as possible.

MontyHigh, www.worldofwallstreet.us

 

UbuntuMUEEE 001: Introduction, My Ubuntu Evil Empire Escape

This blog records the completion of my conversion to Ubuntu away from two of the three big computer / Internet Evil Empires: Microsoft and Apple. I'm not ready to attempt to flee Google, although that may be what's next. The blog focuses especially on my transitioning my audio music and investing news podcasts from of iTunes and iPod from MP3 music and investing news to Ubuntu and an Android Phone and a Cowon iAudio9. I focus on this because I failed to find a satisfactory Ubuntu alternative.

When I start this blog I have:

• A nice, dual-boot Dell desktop computer booting into Windows XP and Ubuntu.
• A nice enough set of basic Ubuntu desktop software with the following applications that seem quite acceptable:
  • Firefox and Chromium for web browsing.
  • LibreOffice as the Microsoft Office replacement.
  • Eclipse for C/C++ software development as a Visual C++ replacement.
  • GIMP for image processing.

What's really missing is an iTunes / iPod replacement. I bought a Cowon iAudio9, but have not gotten it to behave itself well with the LINUX iTunes replacements: RhythmBox, Amarok, Banshee and Clementine.

Here's what I'm looking for in an iTunes / iPod replacement (in priority order):

1. Easily download, tag and organize my mostly investing-related podcasts with easy access to the unlistened to ones and automatic removal and archiving of old ones.
2. Easily select and adjust the set of MP3 audio books that I have on my MP3 players. 
3. Avoid the Evil Empires of Apple and Microsoft. 
4. Support both my MP3 players: Motorola Triumph Android Phone and Cowon iAudio 9.
5. Load all my pop music into my MP3 players and provide a convenient way of Shuffling just the pop music. 

Here's the specialized skills I bring to the effort:

• Basic computer competence and every-day experience installing and using Ubuntu for software development.
• 30 years programming experience, but of a peculiar, specialized type: Low-level networking protocol development mainly in C and C++.
• A nice, general purpose reusable object-oriented framework for development in C.

Feel free to leave a comment if you find anything on this blog of value (to encourage me to continue) or if you have any questions or suggestions.

 

MontyHigh, www.worldofwallstreet.us