Raspberry Pi with 16-bit I2C IO-expander MCP23017

I want to add some physical buttons to my Raspberry Pi. The most easy (and cheap) way is by just adding some push-buttons.

Since the GPIO ports are quite limited I purchased a MCP23017. This is a so-called 16-bits I/O expander which can be used over I2C. This gives me 16 extra inputs/outputs by using only two wires (four if you count VCC and GND), they are adressable so you can chain a lot of them and have even more ports.

I don't have any prior experience wiring and using those. Luckily there are a lot of people already using it, also with their Pi:

http://nathan.chantrell.net/20120519/raspberry-pi-and-the-mcp23017-i2c-io-expander/

Having the datasheet open also helps a lot, some articles had the wiring a little wrong (SCL/SDA mixed up). There are a lot of other I/O expanders, but most of them are 8-bit. Also, that I saw other people using this on 3.3V was one of the reasons I bought this one.

My goal is to get this working with the linux gpio-keys module, so the pushbuttons act like real keys and work directly with all programs (e.g. XBMC). Unfortunately I can't seem to find a lot of information to make this work. Since I also want to make sure I understand what's happing I'm first trying to get the I/O-expander to do what I want using Python.

Really helpful Python-libraries I found are:
http://code.google.com/p/raspberry-gpio-python/ (For the RPi GPIO's in general)
https://github.com/nathanchantrell/Python-MCP230XX

Also, this website which explains using the MCP23017 with a microcontroller helped me understand how to use this:
http://www.gammon.com.au/forum/?id=10945

The expander also has built in pull-up resistors, so you don't need "real" ones between your buttons and the ports, saving some space.

I hope to get this working soon with the python-uinput mode to realize what I want.

Other related and helpful resources:
https://projects.drogon.net/raspberry-pi/wiringpi/
https://github.com/WiringPi/WiringPi-Python

Linux SSD optimalization

Since the SSD prices are dropping and it's still summer vacation so I have a more spare time than usual, I decided to upgrade one of my servers with an SSD. Since the server I upgraded is very multifunctional it could benefit a lot from optimization, it functions mainly as NAS (CrashPlan for Windows, Time Machine for OS X) but is also used as Home Theatre PC (XBMC).

Luckily there are a lot of resources about optimizing your SSD in Linux.

Very helpful resources were:
https://wiki.archlinux.org/index.php/Solid_State_Drives
http://www.void.gr/kargig/blog/2012/01/11/linux-ssd-partition-alignment-tips/

I ended up doing the following:

• Partitioning the hard disk with gdisk, which helps me align the partitions
• Use Logical Volume Management (lvm) 
• Using the --dataalignment option (see also void.gr link) for the physical volume
• Creating an ext4 partition on the logical volume with adjusted block size and stripe/stride-parameters
• Mounting the ext4 partition with noatime and discard options
• Using noop scheduler for the ssd's only (see script below)
• Mounting /var/tmp and /var/log in ramdisk
• Enable writeback (see script below)
• Make the "swappiness" as low as possible
• Use ionice to give XBMC priority over netatalk/crashplan when watching movies
  This is not because of the SSD, but I moved one of my RAID-0 harddisks from S-ATA600 to S-ATA300 because I expect the SSD will use the higher throughput better but apparently this caused problems (stuttering in movies) when a computer was backing up and watching a movie at the same time..

To set the scheduler and writeback to the right disks I modified the rc.local-script I found on the ArchLinux wiki:

declare -ar SSDS=(

        'ata-OCZ-AGILITY4_OCZ-xxxxx',

        'ata-SAMSUNG_SSD_830_Series_xxxxx'

)

for SSD in "${SSDS[@]}" ; do

  BY_ID=/dev/disk/by-id/$SSD

  if [[ -e $BY_ID ]] ; then

    DEV_NAME=`ls -l $BY_ID | awk '{ print $NF }' | sed -e 's/[/\.]//g'`

    SCHED=/sys/block/$DEV_NAME/queue/scheduler

    #scheduler

    if [[ -w $SCHED ]] ; then

      echo noop > $SCHED

    fi

   #writeback

   hdparm -W1 /dev/$DEV_NAME

  fi

done

I also used this opportunity to switch to UEFI and LVM. Especially the UEFI took a whole lot more than I expected. I will write an article about that later.

LCDproc on Mac OS X

I had the opportunity to order some small LCD's (HD44780) at a very fair price, since I already got one in my backup-server i tried to get it working on my docked MacBook running Mac OS X (10.8). Took some time, but in the end I got it working with not too much effort.

Steps I took were:

1. Download LCDproc source from the official website
2. Install libusb from homebrew (brew install libusb)
3. Install libftdi from homebrew (brew install libftdi) <-- this was needed because I used a FT245RL brekout board to connect it.
4. Configure with "./configure --prefix=/usr/local/ --enable-drivers=all --includedir=/usr/local/include/ --enable-libftdi=yes"
5. make -j6
6. sudo make install

Then changed /usr/local/etc/LCDd.conf like this: (only changed/added values shown).

[server]

DriverPath=/usr/local/lib/lcdproc/

Driver=hd44780

[hd44780]

ConnectionType=ftdi

ftdi_mode=4

ftdi_line_EN=0x10

ftdi_line_RS=0x20

ftdi_line_RW=0x40

Device=/dev/tty.usbserial-A80048fE

Size=16x2

Of course, you need to have the XCode Command Line tools installed. Also I had the FTDI D2XX driver already installed, the VCP drivers will most likely conflict with libftdi.

You can get them on http://www.ftdichip.com/Drivers/D2XX.htm 

I also got it showing my Growl notifications thanks to http://maclcd.blogspot.nl/

Getting DVB-C to work well

Anysee E7TC and Windows, the horror...

I bought the Anysee E7TC Plus some time ago because I wanted to record some programs from DVB-C. Since the Anysee was one of the few cards which works for a lot of people having Ziggo (Netherlands) as A cable company I bought one as well. The nice thing about this card is that it has a softcam so I could insert the smartcard (which is needed for all Dutch cable companies if you want digital television) directly and should work on Windows quite well. 


Well my experience was different, I tried three different drivers to get all channels working on Windows 7 and when I finally got all the channels working I found out that the recording software was really unstable. I tried several alternatives like MediaPortal and ForTheRecord. ForTheRecord was actually quite nice but I couldn't get the smartcard reader to work. 

Using the Anysee on Linux, with a separate cardreader

I gave up on Windows and read if the card was supported in Linux, luckily it was. Only the SoftCam was not supported (yet?) so I bought a generic cardreader by the name of Smargo SmartReader Pro. It costs around EUR 30,-. I set it up with OSCam, there are enough tutorials around to help you out but I found out that installation is quite a trial-and-error process. If someone is interested in my configuration just ask me and I will post it here. 

MythTV?

Then I had to decide which program to use, I already used TVHeadend in a IPTV environment. The only alternative I heard/read about for Linux was MythTV. It didn't have to be working really quick so I decided to make a temporary switch to MythTV in the IPTV environment to try it out. 


The setup was quite a pain. The server I used it on was headless and the configuration program needed X to configure. The only solution for me was to use X11 forwarding over SSH. It was a disaster. Slow as hell and not very intuitive but after three hours I finally managed to get it working.


I used MythWeb as frontend, this was quite nice. It offers a lot of options and does not look bad. The problem is it ate a lot of resources and after a while it just hangs after a recording. I didn't optimize the MySQL database behind it but I think that should not be needed for something like this.

Tvheadend

So back to tvheadend it was then. The webinterface is not perfect but good enough for simple recording  and TV watching. The light footprint of tvheadend makes up for the sometimes in-intuitive way of setting up things. For example in the IPTV environment I had to add each channel one-by-one by hand.

I think tvheadend is more focussed on using it in DVB-C/T/S environments since this part is more user-friendly. They already provide some muxes for a lot of DVB providers in different countries. The Dutch companies like Ziggo and UPC add a lot of channels lately so it didn't surprise me the muxlist was incomplete.
Tvheadend does have an option for automatically finding frequencies (muxes) but that didn't work very well for me. I found a frequency list on the Gathering of Tweakers-forum which I used to add the multiplexes by hand. This worked quite well, sometimes I had to re-add muxes to get all the channels but after a while I got all channels. The EPG worked automatically (I was used to needing a separate XMLTV source). Connecting tvheadend and OSCam was really easy and worked directly. 

Problem frequency

Problems arose when I wanted to record something from channels on the frequency 626,750 MHz, this was a frequency I also had problems with on Windows. After checking the DVB-C signal which showed excellent signal quality on other devices on the same outlet I came to the conclusion I had to get another card.

Getting another card, the Terratec Cinergy T Dual PCIe

Since I already got stuff working without using the built-in cardreader I decided to go for a card without cardreader. I also wanted to be able to record two programs at the same time so I looked on the LinuxTV wiki which dual-tuner was supported and not too expensive. I decided to go with the Terratec Cinergy T Dual PCIe. A card almost at 1/3rd of the price of the Anysee E7. The wiki stated that it was supported in the LinuxTV drivers somewhere in 2011 or that I needed to use a recent kernel (3.3 or 3.4), since I have gained quite some *nix experience both would not be a problem for me.

Installing the kernel was the hardest part, initramfs gave me some problems but after those where solved and booted into 3.4.7 it worked directly in tvheadend. Also the 626,750 MHz channels now work without any problems, so it turned out the Anysee card was indeed the problem.

I also solved the problem of the not-so-intuitive interface of tvheadend by using XBMC PVR and it's TV add-on. It's like a match made in heaven ;) I will post some screenshots later

ffmpeg filters, PTS timestamps and mencoder

Well, today I learned a lot more from ffmpeg and video formats and it's encoding in general.

First of all, i'm really happy with the filters it provides.

The drawtext allows you to burn timestamps in your video. I used it to burn it into my scene detection mosaic (see post below) so I can see at which time the scene changes.

The showinfo filter shows you some detailed frame information, first I was wondering why the (PTS) timestamps didn't match with the timestamps in my mosaic, but then (after a few hours) I discovered that the PTS timestamps are relative and can be changed to zero-baed with the "setpts" video filter.

The command I use to get a nice timestamped mosaic with scene changes and a log file that can be parsed to create a chapters file is:


ffmpeg -i $INPUT -vf drawtext="fontfile='/usr/share/fonts/truetype/ttf-dejavu/DejaVuSans-Bold.ttf':timecode='00\\:00\\:00\\:00':rate=25:fontsize=72:fontcolor=white@0.8: x=0: y=0",select='gt(scene\,0.5)',scale=160:120,showinfo,tile=10x16 -s 1920x1200 -frames:v 1 $OUTPUT 1> output.txt 2>&1




The problem is that there is no audio-filter which modifies the audio "frame" pts, so the audio-track is out-of-sync. There was a patch proposed in the mailinglist some time go but was declined because it share some code with the video-pts-set filter.

Then I found out that mencoder also does things like ffmpeg, so I simply tried to copy the audio and video stream and apparently the PTS timestamps are reset. There are a lot of issues though.

Apparently the mencoder libavformat muxing is broken for a looong time (I see mailinglist messages dated from 2008 with the same message), so now I try repacking it with ffmpeg or just get separate tracks and merge them together with ffmpeg.

ffmpeg scene detection and how to use it

I needed scene detection and since I am using ffmpeg to convert my home movies it would be handy to also use ffmpeg for that.

I found out that it was recently added here:

http://avcodec.org/trac/ffmpeg/ticket/442#comment:8

The related commit I found here: http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=72868144e5862168a4f829557b7a41f686b8b12d

which also included some instructions on how to use it:

2644 Complete example to create a mosaic of the first scenes:

2645

2646 @example

2647 ffmpeg -i video.avi -vf select='gt(scene\,0.4)',scale=160:120,tile -frames:v 1 preview.png

2648 @end example

Luckily I built directly from source yesterday and should been able to use this new functionality right away. Unfortunately I get this error:

Assertion ref->buf->refcount > 0 failed at libavfilter/buffer.c:147

I don't know how to fix that yet.
UPDATE: Apparently, this is because the amount of tiles are to much for the mosaic which apparently has a standard size based on the tile-size, also my recordings were very "dirty" (some of them DVB-recordings). First converting them to MKV fixed the latter issue.

Please note that this is very new functionality and I do not recommend to use this in production.


Related links:
http://ffmpeg.org/trac/ffmpeg/wiki/UbuntuCompilationGuide