Jenkins, Robot Framework, Selenium2Library and Linux

I’ve been learning Robot Framework recently and I wanted to see how it ties in with Jenkins so I can get a handle on Continuous Integration.

A while back I installed Robot Framework and the Selenium2Library on a Linux VM and found the Robot Framework reasonably easy to setup and use. After that I though it would be useful to see how much work is involved in setting up a Jenkins system. So I installed Jenkins on the same Linux machine using yum install jenkins, created a ‘Freestyle’ project and added a ‘Build’ step that called a Robot Framework test.

So far, so easy! The Robot Framework Selenium tests should start running (along with the browser window executing the tests).

At least according to the demos I had seen, that should have been enough to get it working but It didn’t work for me.

I was seeing

'Keyword 'Capture Page Screenshot' could not be run on failure: No browser is open'

and

| FAIL |
WebDriverException: Message: The browser appears to have exited before we could connect. If you specified a log_file in the FirefoxBinary constructor, check it for details.

Lots of googling suggested it was something to do with running headless. Except the Linux VM in question isn’t running headless, but as far as Jenkins and/or Selenium were concerned, it was headless.

The options were;

  • Run Jenkins from the console  so that any browser sessions launched by Jenkins would run in the fore ground.
  • Run Jenkins as a service. The theory is that the init scripts should be configured correctly to allow any launched browser sessions to open in the foreground. But I was already running this was because the yum install setup the init scripts for me.
  • Run some virtual desktop software like vnc or Xvfb.

As I was already running as a service and the first option doesn’t seem robust, I figured I would try the last option.

Yum install Xvfb got me the software. I ran it from the console with

Xvfb :1

which starts an Xwindows virtual frame buffer which listens on ‘server number’ 1.

Next, I had to tell Jenkins that it should use ‘server number’ 1 as the display. I guess there are other places I could do this, but for the sake of expediency I added it to the console ‘Build’ step.

My ‘Build’ step then looked like this:

export DISPLAY=:1
robot /tmp/robot_nvbu/valid_login.robot

Where the export line tells Jenkins to send the display to ‘server number’ 1 and then executes the robot tests successfully.

If that works you can install the Jenkins Xvfb plugin to automatically manage the virtual frame buffer which means you don’t need to run it and you don’t need the export DISPLAY=:1 line in the build step. Apart from supplying Jenkins with the location of the xvfb executable and a name, the defaults (leave the fields blank) work fine and the documentation for the plugin is pretty good.

In reality the Jenkins server is not likely to be executing the browser tests. These would be farmed out to other servers and I read a few posts about running Jenkins slave servers in the foreground because of this issue, but the above seems reasonable.

Arduino Nano and Raspberry Pi NRF24L01+ adaptor boards

While working on another MakeBournemouth project, I made some NRF24L01+ adaptor boards for the Arduino Nano and the Raspberry Pi and thought others might find them useful for their own projects so I am offering a few spare boards, in kit form, for sale.

IMG_20160828_131353356

The board makes it easy to connect an NRF24L01 to an Arduino Nano or Raspberry Pi to test it works or to talk to other NRF24L01 wireless modules in your own projects.

IMG_20160828_131832269

There is also a PCB design for the Uno/Leonardo but the PCBs would be a bit more pricey so haven’t ordered any of those. If there is any interest, get in touch and I could be persuaded to get some made.

The nano version is £4.50 + £1.50 shipping in the UK. (Nano and NRF not included. For shipping outside the UK, please get in touch.)

IMG_20160828_131426785

The kit consists of:

IMG_20160828_131438070

  • 1 x Nano NRF adaptor PCB
  • 1 x 2×4 pin socket
  • 2 x 1×15 pin or 2 x 1×16 pin sockets
  • 1 x 100uF Capacitor

(The 1×15 pin sockets are hard to find and often expensive while 1×16 pin sockets are much cheaper and easier to find so kits may be supplied with 1×16 pin sockets depending on what I have available. The extra pin can be cut flush or left hanging over the edge of the board but I would advise you to leave the plastic housing alone. When I’ve tried cutting the plastic housing the pins tend to fall out and the housing gets chewed).

The NRF is powered by the Nanos 3.3v output (the Arduino runs off USB power) and is connected to the SPI bus with CE & CSN pins connected to Arduino digital pins 9 & 10 respectively (the NRF data pins are 5v tolerant). So when using the RF24 library you call the library like this:

RF24(9, 10)

The Raspberry Pi version is £5.50 + £1.50 shipping in the UK. (Pi and NRF not included. For shipping outside the UK, please get in touch.)

The kit consists of:

IMG_20160828_131930395

  • 1 x Raspberry Pi 2/3 Adaptor PCB
  • 1 x 2x4pin socket
  • 1 x 2x20pin socket
  • 1 x 100uF Capacitor

The NRF is powered by the Pi 3.3v output and is connected to the SPI bus (GPIO9,10 & 11 = MISO, MOSI & SCK) with CE & CSN pins connected to GPIO25 and GPIO8 (As per https://blog.the-jedi.co.uk/2013/10/07/nrf24l01-and-raspberry-pi/)

See also http://tmrh20.github.io/RF24/Linux.html (Don’t forget to edit the GPIO pins in the python example (something like: radio(25,8)) and then run it as root or you will get a segmentation fault!)

If you are interested, email mark at sub-design.co.uk (or use the contact form on this site) and i’ll get in touch with availability and we can arrange shipping and payment via paypal.

Quantities are very limited as these are spares from a small batch PCB production run.

These are also available from ebay but are a little more expensive to cover ebay and paypal fees.

Nano NRF Adaptor here

Raspberry Pi NRF Adaptor here

Collectd could not find plugin rrdtool

This is an annoying gotcha that I keep meaning to document. When installing collectd and rrdtool via yum on Centos 6/7, I always miss out the rrdtool lib. This manifests as

Starting collectd: Could not find plugin rrdtool.

when restarting the services.
Thankfully it is easy to fix. Run

yum install collectd-rrdtool

and restart the collectd service and it should start without complaint.

Collectd-web and hostnames?

In my previous post I asserted that collectd-web would display hostnames based on the directory created by collectd to store the rrd files, but that doesn’t appear to be true.

After staging a server and installing collectd & collectd-web and getting that and everything else I needed working, I changed its hostname as it was replacing an existing server.

Collectd-web now displays host entries for the old temporary hostname and the new one but I don’t know where collectd-web is getting or even storing those hostnames.

If I figure out how to remove the redundant hostnames i’ll update this post.

Update: Looks like I was right, but I didn’t bother to check the directory. Collectd is creating the directories based on the hostname. For each change of hostname (and after restarting collectd) collectd created a new set of RRD files in a new directory that reflects the change of hostname.

In the default collectd dir I see something like:

/var/lib/collectd/hostname1

/var/lib/collectd/hostname1a

/var/lib/collectd/hostname1a.lab

So that is where collectd-web is getting its hostnames from. Remove the ones you don’t want (Note: You will lose any rrd data collected under those hostnames) and the hostnames will vanish from collectd-web. Easy!

Configuring Collectd-web for collectd on CentOS 6

If you have tried to get collectd-web running and were presented with the nice graphs without having to mess around for hours, you are lucky. It never ‘just works’ for me…

collectd-webAfter installing collectd and making sure the rrd files were being updated, I ‘installed’ collectd-web. This just involves getting the files from git.

I copied the files to a directory under apache (/var/www/collectd-web), then in the apache config file, made that the root, modified the cgi-bin directory to point to the one in the collectd-web folder. Restarted apache, started the collectd python server (You don’t need this if you are running under apache, but that isn’t obvious from the instructions) and loaded the collectd page which gave me a few menus and not much content.

First off it took me a while to discover that you need to create a config file called ‘collection.conf’ in ‘/etc/collectd’ and in that file, you tell collectd-web where your collectd rrd files are located with the ‘DataDir:’ directive. In a default collectd install on Centos 6 the rrd files are located in ‘/var/lib/collectd/rrd/” under a directory named the same as the host. So within the collection.conf file you have this:

DataDir: "/var/lib/collectd/rrd/"

Collectd-web then lists ‘hosts’ (in its WeUI) for every directory under this one. So if you have a directory called ‘host1’ then you will see a host listed in the WebUI called ‘host1’.

Clicking on the host shows the available ‘plugins’ but clicking on them, I still wasn’t seeing graphs…

Eventually I tracked down an error in the apache error logs.

Can't locate RRDs.pm in @INC (@INC contains: /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at /var/www/collectd-web/cgi-bin/collection.modified.cgi

A quick google for “Can’t locate RRDs.pm” suggest that ‘librrds-perl’ is missing, however on Centos,

yum install perl-rrdtool

fixed the problem.

After refreshing the page I still don’t see graphs – clicking the host and then a plugin finally showed some graphs!

 

Programming one USBasp with another USBasp

The lazy way to update the firmware on a USBasp without faffing with an Arduino Uno as a programmer and lots of jumper leads? Get another USBasp and use it to program the other one.

Instructions based on this blog post.

1) Plug both USBasp programmers together using the 10 way cable.

2) Short JP2 on the target USBasp (the one you want to update).

3) Plug the source USBasp into your computers USB port. Do NOT plug the target USBasp into a USB port!

4) In a terminal type

avrdude -c usbasp -p atmega8 -u -U hfuse:w:0xc9:m -U lfuse:w:0xef:m

to set the fuses. “-c usbasp” tells avrdude that you are using a USBasp as your programmer and “-p atmega8” tells avrdude that the target device (the other USBasp) is based on an Atmega8 microcontroller.

5) Then program the firmware (you can download the precompiled hex file from the blog post I mentioned earlier).

avrdude -c usbasp -p atmega8 -U flash:w:usbasp.atmega8.2011-05-28.hex

If everything worked you should be done. Disconnect everything, remove the jumper on JP2 and give the new firmware a spin burning bootloaders.

MSP430 POV kits

With the firmware nearly complete, I started bagging up the components to make some kits for one of the Make Bournemouth workshops.

This is what is in the through hole kit:

IMG_20140930_203815

And this is the surface mount kit:IMG_20140930_204122

That gives me 8 through hole and 8 surface mount kits. (I Still need to source some more batteries for the surface mount kits).

IMG_20140930_204606

IMG_20140930_204434

Each kit comes with a CP2102 USB to serial adaptor to talk to the MSP430 microprocessor and upload text messages. Firmware programming will require a TI Launchpad board or TI MSP programmer.

MSP430 based POV

I’ve been working on the MSP430 based Persistence of Vision (POV) board for a while now. The original version was meant to be etched with DIY PCB kit, but the results were less than spectacular and I kinda gave up and never put a board together.

The current version (rev2) is a much nicer board but is meant to be manufactured professionally. This isn’t a big deal anymore as it is possible to get small runs of small boards made at a really good price. I used the iTead Studios PCB service and have been really impressed with the results.

IMG_20140728_214829Surface mount board (left) vs thru-hole board (right).

It has taken me a while to sort out the firmware. For a long while it has just been displaying a pacman style ghost and didn’t do anything else. It could only be programmed via a TI Launchpad, and I couldn’t upload a text message via serial.

IMG_20140723_005024Basic firmware – displaying Pacman style ghost

Right now the firmware is pretty much working how I had hoped. I can now upload an ASCII text message via serial and display that on the leds, store it in flash memory so it will be persistent when the device is powered off and powered back on. It still needs a bit of work to tidy things up, but it is about 80% of the way there.

Firmware is on github.com/markjb

Check out the flash_memory branch – that is where all the interesting changes are taking place, but be warned, the code may not be in a working state.