April 17, 2012

WordPress Widget for Ventrilo

by Rob Weeks — Categories: Gaming, Software Development — Tags: , , , — 1 Comment

I’ve recently developed a pretty flexible, basic, WordPress Widget for showing a Ventrilo server status. It’s written in PHP, is Object Orientated and uses XML style sheets (XSLT) to allow for easy customisation of the look and feel.

A while back I was setting up a Left4Dead Ventrilo server for a few friends, after I got it up and running, I went searching for a Widget that would let me add a server status to my home page. I just wanted to show is the server was up, who was currently connect and what channel they were in; I didn’t have much luck.

I decided to write one myself in PHP. I wanted it to be flexible enough to support multiple methods of retrieval but also allow to easy changing of the UI.

I went for a new Web Service utilising the PHP Ventrilo Status Script available from the Ventrilo site. It connects to the Ventrilo server and transforms the Object structure into XML. The widget itself connects to this service via Javascript, retrieves the XML, then applies an XSLT document and DOM injects the result asynchronously on page load. This gave a few advantages: -

  • It’s asynchronous and won’t affect the load time of the home page as the work is done after page load
  • Clear separate of logic between parsing the RCON command out of the server, the Widget code and the UI
  • Any information that a UI designer might want in styling the Widget is available in the XML, all they need to do is update the XSLT to hook into it

The widget currently supports: -

  • Direct interrogation of the Server – each page load the server is queried in real time to get the status – I’ve not thought about caching yet.
  • CRON interrogation – the server is interrogated from a CRON task, the result is stored in a cache XML file that’s served

I’m sure there’s better ways of implementing it so I’m open to suggestions but I’ve stopped playing at the moment, so lost interest in finishing off the plugin and making it pretty; if anyone wants the code I can hand it over.

April 14, 2012

How to backup Googlemail in Linux

by Rob Weeks — Categories: Geeky — Tags: , , — Leave a comment

I’m uncomfortable with several year’s worth of my email sitting only on Google’s servers, especially when they have lost data before. If anything every goes wrong, I want to be able to restore all my data or move providers.

I decided to implement a pretty simple backup routine that would download any newly received mail each day, then enter my normal rsync backups I’ve previously blogged about.

First up, I’m using getmail to do all the work, you can install it on Ubuntu as follows:

sudo apt-get install getmail4

Next, decide where you’re doing to be downloading the mail to and create the top level folder:

mkdir /path/to/backup/user@domain.com
cd /path/to/backup/user@domain.com

Getmail needs 3 folders created (it won’t create them for you), so let’s create them next

mkdir cur new tmp

We’re going to be using IMAP to connect over SSL; it gives a little more flexibility if you’re only interested in some Labelled mails. To do this you need to enable it in your Gmail account. Head on over to your Gmail Settings and go into the “Forwarding and POP/IMAP”. Select “Enable IMAP” and leave the other settings as default.

Now we need to configure getmail, so create a config file:

nano getmail.config

Now enter the following configuration:

[retriever]
username = user@gmail.com
password = mysecret

type = SimpleIMAPSSLRetriever
server = imap.gmail.com
mailboxes = ("[Google Mail]/All Mail",)

[destination]
type = Maildir
path = /path/to/backup/user@domain.com/

[options]
# print messages about each action (verbose = 2)
# Other options:
# 0 prints only warnings and errors
# 1 prints messages about retrieving and deleting messages only
verbose = 2
message_log = /path/to/backup/user@domain.com/message_log.log 
delivered_to = false
received = false
read_all = false

The username and password obviously need updating along with the path and message_log. The mailboxes setting tells getmail what to download; depending on your account “all mail” will either be [Google Mail]/All Mail or [Gmail]/All Mail. I’m not totally sure on the latter, but mine if the first.

You can, if you want, comma separate a list of labels here, no need for a prefix. Mails labelled as “Family” would be just that; “Family”.

An explanation of the last three settings are: -

  • delivered_to = if set, getmail adds a Delivered-To: header field to the message. If unset, it will not do so. Default: True. Note that this field will contain the envelope recipient of the message if the retriever in use is a multidrop retriever; otherwise it will contain the string “unknown”.
  • received = if set, getmail adds a Received: header field to the message. If unset, it will not do so. Default: True.
  • read_all =  if set, getmail retrieves all available messages. If unset, getmail only retrieves messages it has not seen before. Default: True.

This will perform an incremental download of newly arrived mail. To test it, run the following, you’ll see full debugging because of our verbose setting in getmail.config. Once it’s all working, you can drop this logging down to 0.

getmail -g "/path/to/backup/user@domain.com" -q -r getmail.config

Next just throw it into a CRON task to keep on top of it; I chose daily at 4am.

crontab -e

then add the following line

# Backup Gmail every day at 4am
0 4 * * * getmail -g "/path/to/backup/user@domain.com" -q -r getmail.config

That’s it, done. You can see all the mail downloaded in the “new” folder and can access it through multiple email clients that support Maildir.

April 13, 2012

Implementing rotational backups

by Rob Weeks — Categories: Geeky, Software Development — Tags: , — 1 Comment

Backups seem to be the main theme I’m posting about at the moment and the next on my agenda was to get proper rotational backups implemented.

“Proper” meaning incremental (only data being exchanged between client and server is that which has changed) and hard-linked (only data to have changed between backups will incur additional used storage). I.E. If a 1GB folder was backed up 10 times over a year and nothing was changed within it, only 1GB of disk space will be used storing all 10 backups (excluding link overhead) and only 1GB of bandwidth will be used exchanging files.

The main drive to do this was due to the WAN backup I have implemented from my Parent’s machine. I wanted the ability to recover data that had been deleted up to 3 months ago (easily).

To do this, I decided to write a shell script on my Linux HTPC server, it’s not particularly elegant, but get’s the job done quite nicely. The logic is as follows: -

  • First call the rysnc script over SSH from the client with the intended destination backup path and optionally, the number of rotations to keep
  • It will locate the newest sub-folder in the intended path and recursively copy it to a new folder with current date-time as it’s name. The copy uses hard-links so that minimal disk space is taken.
  • Delete the oldest folders that are over the specified number of rotations to keep (defaults to 10)
  • Re-link .latest to the new folder (so we can easily find the most recent backup)
  • Return the new target backup path for use by rsync over SSH
  • Use the returned string as the destination for the rsync command on the client.

It works great, and it pretty easy to navigate to a point in time to see the state of the backup, as you can see below: -

ls -al
drwxr-xr-x 12 owner owner 4096 2012-04-08 00:00 .
drwxr-xr-x 4 owner owner 4096 2012-01-06 18:18 ..
drwxr-xr-x 7 owner owner 20480 2012-01-25 21:12 2012-03-18_00.00.02
drwxr-xr-x 7 owner owner 20480 2012-01-25 21:12 2012-03-25_00.00.04
drwxr-xr-x 6 owner owner 20480 2012-03-26 20:41 2012-04-01_00.00.04
drwxr-xr-x 6 owner owner 20480 2012-03-26 20:41 2012-04-08_00.00.03
lrwxrwxrwx 1 root root 46 2012-04-08 00:00 .latest -> /path/to/backup/2012-04-08_00.00.03

Here’s the Linux script

#!/bin/bash

if [ -z "$1" ]; then
	echo "Missing parameter 1: usage rsyncRotate path [numRotations]"
	echo "	path: the path to a folder that should be rotated"
	echo "	numRotations: Optional parameter detailing how may rotations should be maintained - defaults to 10"
	exit -1
fi

if [ -z "$2" ]; then
        rotations=10
else
	rotations=$2
fi

base_folder=`ls "$1" -1rt | tail -1`

while [ ${rotations} -le `ls -l "$1" | grep ^d | wc -l` ]
do
	# Find the oldest file in the directory and remove it
	oldest_dir=`ls "$1" -1t | tail -1`
	if ! rm -Rf "$1/${oldest_dir}" >& /dev/null
	then
		echo ${base_folder}
		exit "1"
	fi
done

new_folder=`date +\%Y-\%m-\%d_\%H.\%M.\%S`
mkdir "$1/${new_folder}"

find "$1/${base_folder}" -mindepth 1 -maxdepth 2 -exec cp -al "{}" "`readlink -mn "$1/${new_folder}"`/" \;

# Relink the latest folder
latest_folder=$1/.latest
if [ -d "$latest_folder" ]; then
	rm "$latest_folder"
fi
ln -s `readlink -mn "$1/${new_folder}"` "$latest_folder"

echo `readlink -mn "$1/${new_folder}"`

Here’s the Windows backup script

It’s using cwrsync so I can used rsync on Windows, the areas you need to change are in bold.

@ECHO OFF
REM Make environment variable changes local to this batch file
SETLOCAL

REM ** CUSTOMIZE ** Specify where to find rsync and related files (C:\CWRSYNC)
SET CWRSYNCHOME=%PROGRAMFILES% (x86)\CWRSYNC

REM Set CYGWIN variable to 'nontsec'. That makes sure that permissions
REM on your windows machine are not updated as a side effect of cygwin
REM operations.
SET CYGWIN=nontsec

REM Set HOME variable to your windows home directory. That makes sure
REM that ssh command creates known_hosts in a directory you have access.
SET HOME=%HOMEDRIVE%%HOMEPATH%

REM Make cwRsync home as a part of system PATH to find required DLLs
SET CWOLDPATH=%PATH%
SET PATH=%CWRSYNCHOME%\BIN;%PATH%

SET HOST=USER@HOSTNAME
SET KEY=/cygdrive/c/PATH/TO/SSH/KEY/key.dsa

FOR /F "tokens=1 delims=" %%A in ('ssh -i "%KEY%" %HOST% "rsyncRotate \"/path/to/remote/backup\" 16"') do SET target=%%A
rsync -az --delete -e "ssh -i \"%KEY%\"" "/cygdrive/c/PATH/TO/BACKUP/SOURCE/" "%HOST%:\"%target%\""
echo Done: %target%

syncRotate is the name of the Linux script, the first parameter is the remote backup path, the second is the number of rotations to keep. Make sure the script is on the user’s path, or use it’s absolute reference.

April 12, 2012

Developing an online collaborate KJ tool with Google’s Web Toolkit and App Engine

by Rob Weeks — Categories: Software Development — Tags: , , , , , , , — Leave a comment

The Problem

Last year I was participating in a Six Sigma project that was focused on requirement elicitation and risk reduction. One of the tools we were advised to run was a KJ (also known as an affinity diagram); a method of organising ideas into logical groups based on their relationships.

The problem was that one of our team members was based in India, we wanted to include him in the process but typically, due to the large amount of data, KJs are performed in meeting rooms with many post-it notes representing the ideas (voices) and a large canvas (or wall) to move and categorise them on – not very feasible with a 4.5k mile difference.

The Idea

Around about the same time, I discovered the Google App Engine and Google Web Toolkit – two awesome development products from the bright sparks over at Google. I really wanted to get my teeth into these tools, experimenting with Collaborative web applications and some of design patterns I just didn’t get exposed to in my professional career (MVP mostly).

I decided to try out the App Engine and Web toolkit in writing an app that would allow all KJ team members to see the Canvas on a laptop screen and walk through the KJ together (using a conference call or VOIP for discussion), every action a team member took would be replicated on every member’s screen – it should feel like they’re all in the room together but all over a web browser.

I couldn’t do this in Work’s time, after all, I have a day job as a Program Manager, so I picked it up as a bit of fun at home.

High Level Use Cases

These are the very high level cases I proved out.

Authenticate with a Google Account

The tool should permit a user to authenticate with an already existing Google Account.

Import Data from an Online Survey

The tool should be capable of integration with Google Documents contained on the authenticated account. It should allow the importing of data collected from Google Document Forms.

Engage in text-based communication

Users participating in a session should be able to communicate in a “chat room”.

Reduce quantity of visible statements

Any user in the session should be able to mark a statement for including in later phases of the KJ; marking a statement should result in the displaying of a “red dot” (simulating a red marker pen dotted on a post-it note) on all user’s screens immediately. Statements with no dot should be hidden from future phases.

Rewrite content of statements

Text on a statement should be able to be modified by any user in the KJ, changes should be reflected throughout the participating users immediately.

Translate Statements

Text on a statement must be rewritten by the team (or just copied), changes should be reflected throughout the participating users immediately.

Group Statements

Statements must be dragged and dropped together, the statement being dragged should not be visible once dropped. The action of dropping relates the two statements together within the same context. There should be two phases of grouping, the first generating “red” groups, the second generating “blue” groups. The target of the dropped statement should change colour to reflect it’s level.

Export the Data

The results of the KJ should be exported to Excel and OpenOffice format, visualising the relationships between the various groups.

Patterns and Technology implemented

Model-View-Presenter (MVP)

All development was modelled on the Model-View-Presenter design pattern, this enforces the separation of logic between the data (model), the user interface (view) and presenter (domain logic). It’s a great pattern and I fully encourage any developer who hasn’t been exposed to it, or one of it’s related patterns (MVC), to check it out.

Activities and Places

This is an extension of the MVP pattern and handles the invocation of views and presenters based on tokens on the URL, it’s mostly about history management – it was a little clunky and not what I’m used to at all but once I got into it’s statelessness and lack of GET request handling, it was pretty handy.

Google App Engine

All application logic was implemented using Asynchronous GWT RPC calls to the Google App Engine, which was hosing my services and Data store.

I used the Channel API to achieve the collaborative aspect of the application – the main focus of the application. This is very cool, essentially holding an open socket between the browser and server, allowing the server to send commands to the client; it’s a much, much better implementation than Comet based implementations but totally threw me in that the Development mode for GWT actually uses a Comet implementation. If you’re wondering why you deployed application isn’t behaving the same as development; this might be why!

Scalable Data Model

I wanted to get to grips with non-relational database, the App Engine was ideal for this, it’s primary data model is Entities (an elaborate HashMap implementation). It’s simple, fast, distributed and pretty awesome. With little effort (and a lot of forethought) you can build highly scalable, cloud-savvy applications.

The Use of Translation

I’d never run a KJ session before, nor participated in one, so I based the application logic off some cheat-sheets I’d received in training. It occurred to me mid-development that my interpretation of the translation phase was wrong. I’d interpreted it as the subject matter experts and Business Analysts in the KJ session reading and translating the voices generated from the user base into verbage that can be used within the grouping phases.

I was wrong wasn’t quite right. This strategy would work for KJ sessions not intended to solve problems, like the project I was working on; gathering and categorising new requirements. It wouldn’t be totally suitable for KJs aimed at the identification of problems and potential solutions.

The translation within a KJ is intended to map “Images” onto “Voices” and is used to bind together issues and solutions. I think a better use would have been logic similar to below: -

  • Decide on the intention of the KJ, a problem statement
  • Send a survey out to the user base with questions covering what the issues are (Images) and how improvements could be made (Voices)
  • Extract the Images (problems), take them through reduction, red level and blue level grouping.
  • Extract the Voices (solutions), take them through reduction, red level and blue level grouping.
  • Run a translation by getting the Red level statements from the Images and finding red level statements from the Voices that map
  • The result is 2 KJs, one representing what the problems are, one representing the potential solutions and a series of mappings between them – the translations

Next Steps

I’m not planning on continuing the tool past Alpha and the GAE Application is currently disabled. I proved out all the cases in GWT with GAE’s channel API (see the demo below) and had great fun doing it, plus we managed to include our Indian colleague in our KJ session – that’s good enough for me!

Early Alpha Demonstration

The following is an early Alpha demo of the tool, it’s not very pretty but is fully functional and got the job done. I’m simulating two users participating in a session with two windows; we ran the Alpha for our Project with 4 participants and it worked great.

April 12, 2012

Syncing your Music Library to an Android Phone – Wireless

by Rob Weeks — Categories: Geeky — Tags: , , , , , , , , , , — 1 Comment

When I first posted about how to sync an Android music library, I couldn’t find any decent rsync clients on the Android store. That’s all changed now that I stumbled across the rsync backup for Android app. I’ve now got incremental Music syncing across Wireless working.

The reason I discarded this app the first time round was it’s reference to the dropbear SSH server; I though the app was dependant on having this SSH server implemented but it only needs the SSH keys in dropbear format – silly me.

You can just set up rysnc as normal, ensure you’ve got an SSH server up and running with suitable permissions to read the Library, generate the SSH key as described here, then dump the settings into the app, making sure you tick “rsync in reverse direction”.

I used the following rsync options:

-rltDhv –progress –delete –force –modify-window=1 –exclude=desktop.ini –chmod=Du+rwx,go-rwx,Fu+rw,go-rw –no-perms

This will ensure that the Library on the Droid is always kept the same as that on the rsync server (the master). If you want the libraries to be different, drop out the –delete tag. I also excluded the desktop.ini files as there’s no need to sync these and they’re in every folder (as the Library is originally coming from a Windows machine).

When first setting up, you might want to add the -n flag, this won’t actually make any changes but will perform a “dry-run”. When you’re happy with the output in the console window, drop off the the -n and it’ll start pulling files.

To top it off, there’s also Tasker integration, so you can schedule a sync to occur on lots of conditions; plugged in an charging, in range of my home wireless and it’s 3am, for example.

October 18, 2011

Syncing your Music Library to an Android Phone – Wired

by Rob Weeks — Categories: Geeky — Tags: , , , , , , , , , , — 2 Comments

My most recent geeky purchase has been to cast away my now 2 year old IPhone 3GS for a spanking new Android-based Samsung Galaxy S2. So far, I’m very impressed!

One particular pain point though in moving from an iTunes based system, which handled the syncing of content seamlessly (when it didn’t crash), to an Android-based system which isn’t as user-friendly. I know the s2 ships with the Kies software but, from what I’ve read, it isn’t compatible with 64-bit Windows 7 and is a little bit crappy/bloated.

I’ve already posted about using rSync to automate backups amongst home systems and applied the same tool to keeping my Music library in sync.

You’ll need to disable the Kies piece on the phone in order to access the SD card via the file system by enabling USB Storage in the option: “Settings –> Applications –> Development –> USB Debugging”.

Plug the USB cable in, go to the status page and click the button saying “mount the USB storage”.

The script is simple and a 1 liner:

On Linux: rsync -av –delete –force –modify-window=5 “/path/to/music/” “/path/to/sdcard/Music/”

On Windows: rsync -av –delete –force –modify-window=5 “/cygdrive/DRIVE_LETTER/path/to/music/” “/cygdrive/DRIVE_LETTER/path/to/sdcard/Music/”

The options are: -

  • -a: Archive (this is the same as -rlptgoD)
  • -v: Verbose
  • –delete: This will ensure that the source destination is the master, deleting anything from the phone that isn’t on your main machine
  • –force: Force deletion of directories even if not empty
  • –modify-window=5: This will allow a 5 second difference in timestamps from source to destination before a file is flagged as “changed”.

Execute the script and all your files will be pushed to the SD card, the next time you run it, only modified/deleted files on the source machine will be moved/deleted.

July 5, 2011

Virgin Media’s “Superhub” with your own router [Tomato]

by Rob Weeks — Categories: Geeky — Tags: , , , — 7 Comments

This week I upgraded my Virgin connection from 20 -> 30 Mb, the extra speed boost is great but unfortunately, this meant that I had to have the new (no so) Superhub; a modem + router combination. This, quite frankly, has to be the worse piece of network kit I’ve ever seen. It’s continuously dropping, terrible WI-FI speeds and top top it off, you can get bypass it as, at the moment, it cannot act as just a modem.

In order to get it to play ball with my super Tomato-based router, it took hours of fiddling and a few factory resets but, here are the instructions. This won’t turn it into a modem but will allow all the work on the network to be done by another router, the Superhub just handling the internet traffic.

  1. Plug the Superhub (LAN port) into the existing router’s WAN port – it has to be into the WAN one!
  2. Plug the cable into existing router [assuming 192.168.1.1] and jot down it’s WAN port MAC address (in Tomato this is Advanced -> MAC address -> WAN port)
  3. Plug the cable into Superhub [assuming 192.168.0.1], log in, and turn everything off you can find. Wireless, services, UPNP etc. but leave DHCP on.
  4. Assign the WAN MAC address you’ve jotted down to 192.168.0.20 via the static DHCP setting.
  5. Enable the DMZ and put in 192.168.0.20. This will forward everything your Superhub gets to your existing router, letting that handle all the security and port forwarding.
  6. Turn everything off, wait 15 seconds and turn it all back on again.
  7. Plug yourself into the existing router and, unless I’ve missed anything, port forwarding and internet will all work fine.

The gotcha’s for me are that the Superhub and the existing router need to be on different x subnets (192.168.x.y) and the WAN IP address of the existing router needs to be assigned via the Superhub’s DHCP – the DMZ won’t work if you assign it statically.

Also, if you’re like me and use an external domain name to resolve internal server’s IP addresses, using the Advanced -> Firewall -> won’t work by itself due to the way the external IPs are handled. You will need to enable this setting and create a custom DNS entry to route your custom domain back to your “external” IP address in order for it to be picked by by the routing tables.

Go to Advanced -> DCHP/DNS and enter the following in Dnsmasq box

local-ttl=1
address=/mysubdomain.mydomain.com/192.168.0.20

Lets hope Virgin pull their finger out and release the next firmware update for bridge mode (beta update).

November 2, 2010

Implement a remote backup solution at home with rsync

by Rob Weeks — Categories: Geeky — Tags: , , — 8 Comments

Ensuring that your important data is safe and secure is important in the modern age, yet most people that I know (even the tech savvy ones) don’t create backups.

I have a few backup solutions at home to ensure that key documents I don’t want to lose are safe; version control, on-site and secure off-site backups.

Version Control.

This is a must have tool for developers and with my strong Software Engineering background it’s hardly surprising that I’ve kept a version control solution at home for many years. I’ve got a Subversion server and a Mercurial server. The former is what I’ve grown up with and used for years. I feel at home with it, like those jeans your Mrs. keeps trying to throw away but you hang onto for several years past their life expediency.

The problem with Subversion is you must have an available connection to the server to use it… not always ideal. I still keep it around for nostalgia, peeking back at the revisions of my old software every now and they to see how much I’ve grown. Also, all my old University work is in here which, surprisingly enough, I still go back to reference every now and again, normally when a new Placement student starts at work.

Mercurial on the other hand is distributed. I can clone my master repository and hit-the-road, gaining all the befits of version control without a network connection and, when I’m back, push those changes back to the master repository as a backup.

Don’t get me wrong, version control isn’t just for developers. I have my girlfriend a repo when she was doing her teaching degree in order to keep track of evolving documents and have an off-site backup in-case of an emergency – it worked well. This leads me nicely onto the subject of this post; backups.

Rsync as a backup solution

I wanted my Home theatre PC to be the HUB for my backup solution; it’s low power, quiet and always on, plus it has over 5TB of storage on it today.

The HTPC machine runs an rsync daemon, I have seperate primary backup locations defined for various rsync tasks: -

  • Off-site backup for my parents documents and digital photos – This is accessed by a schedule task using secure rsync SSH tunnel (pass the -e “ssh -i /path/to/my/private.key” flag to rsync to go via ssh)
  • Location for my Desktop machine – rsync across LAN weekly
  • Location for my Version control repositories – rsync locally nightly

I also have primary backup locations for non-rsync tasks

  • Location for my PS3 data (although I really wish PS3 supported backups via Samba and not USB dongles)
  • robweeks.net – my domain does a full backup via SSH straight here when I request it (I’ve not got the web host admins to cron me it up yet)
  • Gmail archive – I got paranoid after the recent data loss experience by google users and implemented a getmail cron to download all my mail daily, just in case.

On a nightly basis I sync my primary backup to an external on-site hard-drive, in-case my disk blows up. On a weekly basis I backward sync my primary backup to an off-site location (in-case we get robbed!)

In short; rsync is brilliant. It does exactly what it says on the tin, is secure, supports compressed traffic transfer (if bandwidth is an issue) and was all set up in about an hour! If you’re interested there are hundreds of tutorials around to aid you in setting it up. If you’re looking at implementing it on Windows, check out cwrsync.

September 22, 2010

Setting up a Home Theatre PC on Ubuntu 10

by Rob Weeks — Categories: Geeky — Tags: , , , , , , , — 3 Comments

 

Background and Hardware

This has got to be my all-time favourite gadget I’ve ever had; a Linux based HTPC. It’s low power, silent, capable of comfortably playing full-HD content with 5.5TB of disk space.

Why bother? First up – TV. Well, I have a Sky HD+ subscription but it just doesn’t cut it. Let’s face it, the TV we have today is just terrible. We have a few months of the year where there’s something worth watching and, in the UK, we get those decent shows months after airing in the US. I want to be able to turn on the TV and have a selection of good stuff to watch, when I want to watch it.

Secondly – Movies. I’m sick and tired and all those Disks floating about, in the wrong box (because I’m lazy) or on the Side getting scratched where I’ve had the urge to jump onto the Playstation for a Call of Duty session. Having my Movie collection fully digitised is a no-fuss, “it just works” solution.

Cost wise; it wasn’t bad at all. I really wanted one of the NVidia ION boards so I went for the following set-up: -

This took the total to a respectable £247.84 for a fully working machine. I also added an MCE remote off Ebay for about £12 and I added a spare 1.5TB hard-drive I had kicking about and a 2TB External USB drive to backups etc.

One thing I can’t stress enough is this motherboard is the most fussy piece of kit I’ve ever seen. You must adhere to the Qualified Vendor List (QVL) or it just wont boot. I went through 3 sticks of RAM before I learnt that lesson. Despite that, it’s easy to build, even for beginners as the Motherboard, CPU and GPU are a fixed unit.

Installing the Operating System

You may have noticed I don’t have an optical drive, because of that I used the USB stick installation. Grab the latest distro from the Ubuntu home page and you’re off.

I’m no Linux expert by far but I know enough to get around. I went for 6GB for the root partition and the rest dumped to a secondary partition for data.

First thing you will want to grab the latest ION Graphics drivers from the NVidia website. Download the NVIDIA-Linux-x86-xx.xx.xx.run file and save to disk, then run: -

cd /path/to/file/
chmod 755 NVIDIA-Linux-x86-xx.xx.xx.run
sudo ./NVIDIA-Linux-x86-xx.xx.xx.run

Once complete, reboot the system. You’ll see a nice new crisp display in a decent resolution.

If the edges of the screen cant be seen you need to adjust the over scan settings. You can do this in the NNidia settings panel, run the following, find the setting and drag the sliding bar until it fits.

nvidia-settings

Post installation

After I got it up and running, it took me a good 10 days to sort out all the problems. Here are the big ones.

Tearing

This looks like the top half of the screen was being rendered a few milliseconds before the second half of the screen, creating a “horizontal line”. It was very, very notable, especially during fast-paced scenes. Also, without knowing what the issue is or what it’s actually called, you try finding how to fix “Weird line on screen” in Ubuntu amongst all the crap on forums.

To fix this, you need to uninstall “compiz”. From what I can tell, it’s like Aero for Linux; makes things pretty. You don’t need it as you’ll be spending all your time in XBMC which has a very sexy UI. Run this in a terminal: -

sudo apt-get purge compiz

You may also want to enable the “Sync to VBlank” option inside the NVidia settings. Run the following in a terminal: -

nvidia-settings

Find the option for both XVideo and OpenGL and tick it.

Audio Over HDMI

The version of Ubuntu I’m using (10.04 was the last update I did) didn’t a high enough version of ALSA to support the sound card out of the box for the NVidia board. If you’ve got the same problems as me, you’ll need to upgrade.

First check that the card isn’t recognised, run this in a terminal: -

rob@HTPC:~$ aplay -l
**** List of PLAYBACK Hardware Devices ****
card 0: Intel [HDA Intel], device 0: ALC887 Analog [ALC887 Analog]
 Subdevices: 1/1
 Subdevice #0: subdevice #0
card 0: Intel [HDA Intel], device 1: ALC887 Digital [ALC887 Digital]
 Subdevices: 1/1
 Subdevice #0: subdevice #0
card 1: NVidia [HDA NVidia], device 3: NVIDIA HDMI [NVIDIA HDMI]
 Subdevices: 1/1
 Subdevice #0: subdevice #0
card 1: NVidia [HDA NVidia], device 7: NVIDIA HDMI [NVIDIA HDMI]
 Subdevices: 1/1
 Subdevice #0: subdevice #0
card 1: NVidia [HDA NVidia], device 8: NVIDIA HDMI [NVIDIA HDMI]
 Subdevices: 1/1
 Subdevice #0: subdevice #0
card 1: NVidia [HDA NVidia], device 9: NVIDIA HDMI [NVIDIA HDMI]
 Subdevices: 1/1
 Subdevice #0: subdevice #0

If you see the NVidia card (in bold), but still don’t get sound, skip down to the mixer bit below, otherwise continue in upgrading ALSA.

Grab the script from here and run it with the following: -

sudo ./AlsaUpgrade-1.0.23-2.sh -d
sudo ./AlsaUpgrade-1.0.23-2.sh -c
sudo ./AlsaUpgrade-1.0.23-2.sh -i
sudo reboot

For some stupid reason, the line is muted by default. You need to un-mute it in the mixer. Run: -

alsamixer

Press F6 and select the NVidia card. The “MM” means muted, select each of the channels and press “m” on the keyboard so they show “00″.

Now find the card and device that maps to the HDMI port. You’ll need to grab the card and device numbers from running: -

aplay -l

For each entry listed, run the following: -

aplay -D hw:{{CARD_NUMBER}},{{DEVICE_NUMBER}} test.wav

When you get some sound coming out, you now tell ALSA to use that card by default: -

nano ~/.asoundrc

Now paste the following into the file, replacing the card and device numbers with what you found above.

pcm.!default {
 type hw
 card 1
 device 7
}

pcm.default {
 type hw
 card 1
 device 7
}

From now on the sound in the system should be routed to the HDMI port by default.

Fixing the MCE Remote

The remove I bought half worked out the box, that being, half the buttons didn’t do anything. The reason for this is that it’s modelled as two separate Human Interface Devices (HID) and LIRC (The infra-red application for Linux) was only interpreting one of them.

I can’t remember the exact steps I used to fix it but I know I had to merge to two channels for “Keyboard” and “Mouse” that the remote is sending to with a hack. This post got me most of the way through it but there was a lot of trial and error involved.

Installing XBMC

XBMC is available through Aptitude so just run: -

sudo apt-get install xbmc

I then just dumped it in the start up applications of Ubuntu so it loads when XServer is up and running. You can find that in the menu System -> Preferences -> Sessions

I would advice you keep your Movies and TV separate, using different scrapers for each. Check out here and here.

Conclusion

You should now have a very sexy Home Theatre PC capable of playing HD videos but that’s not the end of it, you can set it up to fully automate your TV downloads, work as a NAS device, web server, Remote Backup location for friends/family… and more.

Thanks

I just wanted to say thanks to the contributors of these pages. You helped me solve all my (HTPC) problems!

© 2012 Rob Weeks All rights reserved - Wallow theme v0.46.5 by ([][]) TwoBeers - Powered by WordPress - Have fun!