Craig Mayhew’s Blog

First, upgrade rails. This will take some time, go and make a tea/coffee.
#gem install rails

#cd /var/www
#mv railsapp railsapp-old
#hg clone –updaterev 2.0-stable https://bitbucket.org/redmine/redmine-all railsapp
#cp railsapp-old/config/email.yml railsapp/config/configuration.yml
#cp railsapp-old/config/database.yml railsapp/config/database.yml
#cp -r railsapp-old/files/ railsapp/files/
#chown -R root:www-data /var/www/railsapp
#cd railsapp
#gem install bundler
#gem install test-unit
#bundle install –without development test rmagick
#mkdir public/plugin_assets
#rake generate_session_store
#rake db:migrate RAILS_ENV=production

If at this point you get an error about the mysql-adapter:

Please install the mysql adapter: `gem install activerecord-mysql-adapter` (mysql is not part of the bundle. Add it to Gemfile)

Then edit this file:
#vi config/database.yml and change all instances of “adapter: mysql” to “adapter: mysql2″.

Then, install the addon:
#gem install activerecord-mysql2-adapter

edit you Gemfile
#vi Gemfile

replace the section:
platforms :mri_18, :mingw_18 do
group :mysql do
gem “mysql”
end
end

with:

platforms :mri_18, :mingw_18 do
group :mysql do
#gem “mysql”
gem “mysql2″, “~> 0.3.11″
end
end

then run this again and hopefully it should work:
#rake db:migrate RAILS_ENV=production

chown -R www-data:www-data files log tmp public/plugin_assets
rake db:migrate:upgrade_plugin_migrations RAILS_ENV=production

If you find this errors with: “don’t know how to build task ‘db:migrate:upgrade_plugin_migrations’”.
then use this instead:
#rake redmine:plugins:migrate RAILS_ENV=production

this next one may error, ignore it if it does
#rake db:migrate_plugins RAILS_ENV=production

#chmod -R 755 files log/ tmp/ public/plugin_assets
#rake tmp:cache:clear
#rake tmp:sessions:clear

References:
http://www.turnkeylinux.org/forum/general/20120722/guide-how-upgrade-redmine-latest-version-203-painlessly
http://www.redmine.org/boards/2/topics/33504

#sudo apt-get install libftdi1 libftdi-dev libconfuse0 libconfuse-dev cmake

Download the latest source code from Telldus at: http://download.telldus.se/TellStick/Software/telldus-core/ you can use the wget command for this.

In these next commands, Change {version} to the downloaded version.
#cd /usr/src
#gunzip telldus-core-{version}.tar.gz
#tar xvf telldus-core-{version}.tar
#mv telldus-core-{version} telldus-core

#cd telldus-core
#sudo cmake .
#sudo make
#sudo make install
#sudo ldconfig

#cd /etc/init.d
#sudo cp skeleton telldusd
Open the newly created file in your favorite editor (as root with sudo) and do these changes:

Line 3, change:

# Provides: skeleton
into:
# Provides: telldusd

Line 8-10, change:

# Short-Description: Example initscript
# Description: This file should be used to construct scripts to be
# placed in /etc/init.d.
into:
# Short-Description: Tellstick service daemon
# Description: Tellstick service daemon controlling remote switches.
#

Line 20-24, change:

PATH=/sbin:/usr/sbin:/bin:/usr/bin
DESC=”Description of the service”
NAME=daemonexecutablename
DAEMON=/usr/sbin/$NAME
DAEMON_ARGS=”–options args”
into:

PATH=/sbin:/usr/sbin:/bin:/usr/bin:/usr/local/sbin
DESC=”Tellsick service daemon”
NAME=telldusd
DAEMON=/usr/local/sbin/$NAME
DAEMON_ARGS=”"

Make the script executable.
#sudo chmod +x telldusd
Test the init-script by starting the service and checking its status.
#sudo service telldusd start
#sudo service telldusd status
Install the script to start at boot.
#sudo update-rc.d telldusd defaults

Configuring the TellStick for each remote power switch: http://developer.telldus.com/wiki/TellStick_conf

Finally use the tdtool command to turn switches on and off! So in this case I am turning on switch 1 (1 as specified in the config file)

#tdtool –on 1

To automate my home setup I have written the following python script, it turns things on using the tellstick when I walk in the room (including sending wake packets to computers etc). It then turns them all off again if I’m gone for more than 5 minutes.

https://github.com/craigmayhew/scratchpad/blob/master/python/pir-power.py

Place a copy of it /home/pi/ and edit the root crontab via:
#sudo su
#crontab -e

Then simply add the line to the bottom of the crontab:
* * * * * python /home/pi/pir-power.py

if you want to manually kick the script off simply type (in the command line):
#python /home/pi/pir-power.py

Sitting down for our regular Sunday breakfast at Pitcher & Piano on the marina. I’m perusing the FT feeling more and more negative about the economy, but hopeful that interest will stay low so my mortgage gets paid quicker. Meanwhile Beth is flicking through the Sunday Times feeling slightly frustrated that she can’t find a pen to do the puzzles section. Because we both end up so busy, Beth with medical school and a few side projects she’s working on, me with work and a healthy dose of more work, we make the time every Sunday to go for breakfast. That’s right, we are both so busy that we can only promise one morning a week to each other.

It seemed like just a few minutes ago I was back in my bedroom with old friends, trying to decide what film we should watch on a 21 inch CRT television somewhen around 2004. This was simultaniously the most difficult decision we had to make back then and about as far ahead as we ever needed to plan.

Somehow we got onto the subject of the “good old days” and started reminiscing about our old group of friends, the Rose Wilmot centre, trampolining, the fish and chip shop, never ending games of TA, a virtually unlimited supply of movies, a 512k cable connection (which was screaming fast in those days).

The Rose Wilmot centre was torn down, anything less than a 100Mb connection is now a hindrance and I haven’t seen a CRT screen in half a decade.

I suddenly feel I might be old…

Super Capacitors have been closing the gap with batteries over the last decade. Maxwell Institute currently dominates the Wikipedia article for super capacitors and has devices that store enough power to be a phone battery replacement. The problem is, the capacitors currently require 10x the volume.

With capacitors continuously improving, this got me thinking. What is the maximum power density that physics allows for super capacitors and what would this mean if we achieved it… Would power stealing suddenly become a problem? would the capacitors store enough power to be a weapon? Could electrical power be stored as an investment?

The answers to these 3 questions could all be yes, if power density is high enough.

Imagine going into work, happily sitting down at your desk, then police rush in, the manager points at a colleague and the police march him out of the office while taking a keen interest in his laptop. You later find out he was stealing power, to the tune of £8 per day for the last two years. He had retrofitted his laptop with a a futuristic supercapacitor which was hidden inside and charged quickly and silently, holding as much power as it could take from the plug socket at his desk. He only took it home some nights (when it’s fully charged) and would then plug it into his home power system and be paid by his energy provider to put the power back into the grid as “green energy”. His scheme worked well netting him about £5700 until either his employer or his power provider realized something was going on.

Perhaps don’t imagine this next one, but a capacitor will catch fire and explode if it is charged and shorted out. Hopefully future high energy ones will have some kind of built in safety to prevent them being shorted out and exploding. This could be particularly worrying if dogs can’t sniff out supercapacitors – but hopefully x-ray machines at airports would spot them.

Finally, if you can store energy in a concentrated and efficient way (and this doesn’t just apply to super capacitors) then you could store it when it is cheap to do so and sell power when/where it’s in demand. This could be to move power between giant solar arrays around the equator to cities that need it or simply storing power during the day for use at night (or vice versa in the case of nuclear power).

In the paper “Graphene-Based Supercapacitor with an Ultrahigh Energy Density“, a team have achieved power density’s of 85 watt hours per kg at room temperature. This will hold about 1.02 pence of power, assuming electrical cost of 12 pence per kwh.

(12 / 1000) * 85 = 1.02

This falls far short of being viable for any of my 3 scenerios. For instance, this power density would need to be improved by at least 3 orders of magnitude before power theft would be viable.

A counter argument to the economics of stealing power is that super capacitors may cause a drop in the cost of power per kwh. This might be realized in having less power stations but enough power storage to keep up with times of peak demand (e.g. store the power that is effectively “wasted” at night by nuclear power stations that prefer to be on 24×7 rather than constantly being turned up and down).

Supercapacitors could be a very disruptive technology, but they need to improve quickly enough to ensure other power storage methods don’t beat it to mass production.

For this test I will be running FreeBSD in a VirtualBox virtual machine. These instructions will work regardless of this.

Download the 64bit iso of FreeBSD. I am using version 9.1.

Install FreeBSD form the livecd. I decided to drop the games from the install.

I initially tried to get a bootable zfs partition going, but after several hours of failed attempts using this guide: https://wiki.freebsd.org/RootOnZFS/GPTZFSBoot/9.0-RELEASE, I decided I would leave plenty of drive space spare when partitioning the disk. I set 64GBs up for the operating system and left the rest spare for the ZFS.

Note: I have used da0 for my drive as it is a SAS drive. If you are using SATA then you will need ada0.

# gpart add -t freebsd-zfs -l disk1 da0

So, assuming you have booted freebsd, you now need to create the zfs volume.

# kldload opensolaris
# kldload zfs
# zpool create -m /zfs disk1 da0p1

Enable dedupe using fletcher4, verify hashes when writing to disk and enable compression:
# zfs set dedupe=on disk1
# zfs set dedup=fletcher4,verify disk1
# zfs set compression=gzip disk1

We will want it to mount the volume every time the system boots

# echo ‘zfs_enable=”YES”‘ >> /etc/rc.conf

Great, now we need something like network file sharing for windows so I can use it in a windows environment and just dump backups to it. (Prepare for a lot of compiling and a long wait)

# cd /usr/ports/
# portsnap fetch
# portsnap extract
# portsnap update
# cd net/samba36/
# make install

You will be asked for a whole bunch of optional packages, just stick with the defaults as the menus appear.

You will be wanting Samba to available after reboot:
# echo ’samba_enable=”YES”‘ >> /etc/rc.conf

I am sharing my /zfs directory and have setup my smb.conf like so:
# vi /usr/local/etc/smb.conf

In here goes your password:
# vi /usr/local/etc/samba/smbpasswd

Warning:
Do not ever use a ZFS partition for SWAP, do not place any swap files within a ZFS filesystem. The reason for this is that ZFS requires a fair amount of RAM (some argue over 1GB for every 1TB of unique data on disk). If the data ZFS keeps in RAM falls into swap then your system will very quickly ground to a halt and slowly consume all it’s RAM.

Further reading:
https://wiki.freebsd.org/RootOnZFS/GPTZFSBoot/9.0-RELEASE
http://www.datadisk.co.uk/html_docs/sun/sun_zfs_cs.htm
https://wiki.freebsd.org/ZFSQuickStartGuide
http://www.freebsd.org/doc/handbook/filesystems-zfs.html

So I accidentally let one of my SSDs completely run out of space. My VM running on this disk then promptly crashed. I knew I had to repair it – but while I’m at it – I’m going to convert it to Oracles virtual box.

Fixing corruption

First problem is that my virtual disk is corrupted – this happens when you run out of space because the disk suddenly can’t write the data it was expecting to write to disk. Ideally you will have a backup – but if you don’t then make one now and then try to repair it using the application vmware-vdiskmanager.exe. Obviously – it needs to be moved to a drive that has space or you need to create some space on the full drive.

Open a command prompt and navigate to the directory containing your virtual disk. In my case this is the command:

cd “E:\Windows 7\”

Then, in my case the repair command was:

E:\Windows 7>”C:\Program Files (x86)\VMware\VMware Server\vmware-vdiskmanager.exe” -R “Windows 7_1500GB.vmdk”

If all goes well you will see this message:
The virtual disk, ‘Windows 7_1500GB.vmdk’, was corrupted and has been successfully repaired.

In the case of my windows 7 guest OS – it still had corrupted files even though the disk itself was now fine.

To find out if you have corrupted drivers etc then run this tool from within your guest: http://support.microsoft.com/kb/929833

Fixing DLL corruption in Windows 7 Service Pack 1
If, like me you managed to run out of disk space and corrupt your virtual disk, you may well find some of the dll files are a little “crashy”. To repair this, you will need your Windows install DVD. If you are running Windows 7 Service Pack 1 – and you find your install DVD is without the Service pack, then the repair install will fail saying you can’t overwrite a newer version of windows. To “work around” the issue, simply run the setup.exe in compatibility mode for “Vista service pack 2″. This bodge worked for me. However, you should make a backup of your disk images before trying.

Converting to Virtual Box

To convert the disks run this command (modified for your paths etc):

E:\Windows 7>”C:\Program Files\Oracle\VirtualBox\VBoxManage.exe” clonehd –format VDI “E:\Windows 7\Windows 7_1500GB.vmdk” F:\dDrive.vdi
0%…10%…20%…30%…40%…50%…60%…70%…80%…90%…100%
Clone hard disk created in format ‘VDI’. UUID: 9c41acd3-1e97-4792-9b37-1b4f7329ba25

I decided to create a virtual box machine and import the two hard disks. This would have worked great except I picked the wrong disk controller when setting up the machine. Which results in windows 7 showing this message at boot time:

A problem has been detected and Windows has been shut down to prevent damage to your computer.
If this is the first time you’ve seen this stop error screen, restart your computer. If this screen appears again, follow these steps: If this screen appears again, follow these steps:

*** Stop: (0xF78E2640, 0xC0000034, 0×00000000, 0×00000000)

The fix for me was to realise that vmwares “SCSI: LSI Logic SAS” translates to “SAS Controller” of type “LSI Logic SAS”.

At this point my virtual box image boots but it is very unstable. It rebooted a few times to install the newly found hardware but then kept blue screening. This was due to disk corruption. Running chkdsk from within the VM (took about an hour) solved the issue. If still having issues – see the driver repair install further up this page.

Now – another issue I found is that copying VM disks can also cause issues due to the uuid that VirtualBox assigns to each disk. If you ever need to change the uuid, this is how you do it:

cd c:\Program Files\Oracle\VirtualBox
VBoxManage internalcommands sethduuid “E:\Windows 7\cDrive.vdi”

Do not forget about VMWARE Tools

This is a pain to remove. This is the best guide I have found, also even the automated removal fails to get rid of all the registry keys, so you need to do manually (see the guide). http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1308

This is a simple guide to setting up a PIR sensor with a raspberryPI to send a wake-on-lan packet to one or more computers. The idea being that as I walk into my office, everything boots up or comes out of hibernation automatically.

SSH into the pi and get the latest python dev code:

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install python-dev
sudo apt-get install python-rpi.gpio

Install the Wake On Lan command:

sudo apt-get install wakeonlan

Plug in your PIR sensor using the 5 volt power line (pin 2), GPIO 7 for data (pin 26), the ground pin (pin 6). A word of caution, these pins should absolutely not be shorted out, they are also static sensitive.

RaspberryPI with a working PIR sensor:

Create a file called PIR-wake-on-lan.py on the raspberryPI and put this source code into it:
https://github.com/craigmayhew/scratchpad/blob/master/python/PIR-wake-on-lan.py

You will need to edit the file and put in your own mac address in place of “wakeonlan 00:00:00:00:00:00″.

Try running the script.
sudo python PIR-wake-on-lan.py

I’m now running:

Virtualized Windows 7 64bit
I now work from a virtual machine. It is one of many workstations and servers now running on a single chunk of hardware. All main drives are 256GB ssds (VMs are selectively spread across these), 3TB spinning drive for backups, 32GBs of RAM, quad core i7-3820 CPU and a cheap / efficient GeForce GTX550 Ti. Not currently using any RAID, just regular backups (Really should RAID my backup drive).

7-Zip
This creates very very compressed 7z archives and now supports a multitude of other archive types too. The more RAM you have, the better it will compress larger volumes of data. If you have loads of RAM then for best compression, use in 7z mode with ultra LMZA2, max dictionary and word size.
http://www.7-zip.org/download.html

AVG
This has been updated to the latest paid version. Still impressed and still virus free.

E Text Editor
Not really using this as much as I should. Regex support is very nice in this.

Grepwin
Where would I be without this awesome search tool? Lost, that’s where.

Mozilla Thunderbird
Still the best free and open source email client in my opinion.
http://www.mozilla.com/en-US/thunderbird

Netbeans IDE 7.2
This is still catching up with some of the other IDEs (e.g. NuSphere) but it’s open source and supports XDebug so I’m happy.

Skype
Aside from it’s recent major security issue involving account takeover with their dodgy password reset page, it is still arguably the best free instant messenger, audio/video conferencing software. http://www.skype.com/download

Titanium Studio 2.1.2
This was insanely complex to install. So many bugs in the install process or failed updates that I’m surprised anyone uses this. However, once installed I think very good for developing cross platform mobile apps.

UltraVNC
lThis can display the screen of another computer (via internet or network) on your own screen. This allows you to use your mouse and keyboard to control the other PC remotely. It means that you can work on a remote computer, as if you were sitting in front of it.
http://www.uvnc.com/download

VideoLAN
VLC media player is a highly portable multimedia player for various audio and video formats.
http://www.videolan.org/vlc

WinMerge
Useful for comparing two or more text based files and highlighting the differences.
http://winmerge.org/downloads

So what’s gone from my previous list in 2008?

Windows Messenger
No one is on here anymore and Microsoft are about to completely drop support for this.

CutePDF Writer
This has been replaced by bullzip for me.

Firefox
We have moved on from version 3 back in 2008 (to version 16!). Chrome however has slowly won me over due to it’s speed and multi threading.

Opera
I never quite “got” opera.

WinRAR
It cost money, isn’t open source and is worse than 7zip on so many levels. It had to go.

MWsnap
Windows 7 has a snipping tool that seems far quicker to use. I’ve pinned it to the start bar and it’s quick.

Open Office
I ended up buying Microsoft Office 2010. Open office really doesn’t come close to how good office 2010 is.

Folding@home
Woah, when did I drop this from my machine? Probably the same time I started using world community grid. To be perfectly honest I’m now not too fussed about either. I would still like to heat my house from computers that are running charitable simulations, maybe next year?

Synergy
Gone and haven’t looked back.

I wondered how long one needs to wait before owning the current worlds computer processing power for under $1000.

A few studies have been done to measure the world’s processing power. In 2007 the entire world’s computer capacity measured at 6.4 x 10¹⁸instructions per second^[1]. as estimated by Martin Hilbert, et al. Unfortunately that’s not really a useful measure of processing power. FLOP/s would have been more useful as it relates to the useful work you can do.

So I’m going to have to do some estimating myself. The research firm Canalys have come up with a figure of 112 million PCS sold in 2011. I’m going to estimate (complete guess) the processing power of the average computer sold at 30GFLOP/s. I’m basing that on an Intel Core i7 980 XE six core processor available in 2010 measuring at 109 GFLOP/s. Most people are going to go for mid range processors though so that’s where I get my 30GFLOP/s from. I’m aware this does not include the graphics processing power. So we have a total of 112000000 x 30GFLOP/s = 3360000000 GFLOP/s or 3.4 ExaFLOP/s.

This same processing power will be available in a laptop or smaller device by the year 2033^[2] as measure in MIPS. So that’s 26 years for the processing power of the entire planet to be condensed into a device that sits on your lap. If you want to argue that my estimate of the worlds processing is off by a factor of 10 then make it 2037. This does however ignore any game changes such as quantum computing, optical computing, spintronics… the list goes on.

Also, that is just one data point and relies on moore’s law, perhaps 26 years has been a constant since the dawn of computing, or perhaps it’s a number that get’s steadily smaller over time. If I find more data, I’ll update this blog post.

Sources:
[1] http://www.uvm.edu/~pdodds/files/papers/others/2011/hilbert2011a.pdf
[2] http://www.transhumanist.com/volume1/moravec.htm

So I wanted to backup and archive my tweets. I threw together this quick and dirty script to retrieve the last 200 tweets of a twitter user (In this case, me).

Latest source code available here:  https://github.com/craigmayhew/scratchpad/blob/master/php/retrieveTweets.php

My archived tweets: www.craigmayhew.com/tweets/

<?php

define(’DB_NAME’, ‘dbname’);    // The name of the database
define(’DB_USER’, ‘dbuser’);     // Your MySQL username
define(’DB_PASSWORD’, ‘agoodpassword’); // …and password
define(’DB_HOST’, ‘localhost’);    // 99% chance you won’t need to change this value

//connect to the database
$mysqli = new mysqli(DB_HOST,DB_USER,DB_PASSWORD,DB_NAME);

//retrieve a twitter feed
$jsonFeed = file_get_contents(’https://api.twitter.com/1/statuses/user_timeline.json?include_entities=true&include_rts=true&screen_name=craigmayhew&count=200′);
$json = json_decode($jsonFeed);

//input each of the tweets into a database table
foreach($json as $js){
$mysqli->query(’INSERT IGNORE INTO tweets (time,text) VALUES (\”.$mysqli->real_escape_string(date(’Y-m-d H:i:s’,strtotime($js->created_at))).’\',\”.$mysqli->real_escape_string($js->text).’\')’);
}
?>