Data-Driven Testing in Go aka Table Testing or Parameterized Testing

When writing tests, we want to focus as much as possible on the actual test cases and test data, and not on implementing the individual cases. The less time you spend in writing code to implement a test-case, the more time you can spend on actual test data.

This is where data-driven testing comes in handy. Data-driven testing splits the test data from the test logic.

What is Data-Drive Testing?

So what is data-driven testing exactly? In data-driven testing you reuse the same test script/invoker with multiple inputs.

To do so you need:

  • Have test-data in files. For each test you should have:
    • Description of the test
    • Input for the test
    • Expected output
  • Run the same test script on each of the input data.
  • Check whether the actual output of the test script matches the expected output you defined in the input file.
Overview of Data-Driven Testing

You probably know data-driven testing already as “Table Testing “or “Parameterized” testing

How to Do Data-Driven Testing in Go

But how do you implement data-driven testing in Go?

The examples I use originate from tests I wrote to test Sanity’s patching logic on documents. This means we need an input document, a patching function to apply on this document, and an expected output after the patching is applied.

Test Input File

I opted to put the test input in Yaml files. Each file contains a list of (related) test cases.

  • description of the test is string.
  • input, patch, expected_output, are multi-line strings, which contain JSON. This can be of course anything, but in my tests I needed JSON.

An example of such an input data file:

- description: inc
  input: |
    {
      "x": 0
    }

  patch: |
    {
      "patch": {
        "id": "123",
        "ifRevisionID": "666",
        "inc": {
          "x": 1
        }
      }
    }

  expected_output: |
    {
      "x": 1
    }

Parse File

Creating a datafile isn’t enough, it must also be parsed. To do so I created a custom UnmarshalYAML function to implement the Yaml Unmarshaller interface. So that it gets automatically picked up by the go-yaml/yaml package when trying to unmarshall it. I left this implementation out because it is very specific to what we do in our tests at Sanity.

The datafile is represented in Go with a type alias and a struct as follows:

// A TestFile contains a list of test cases
type TestFile []TestCase

// TestCase represents a single patch test case.
type TestCase struct {
    Description    string                `yaml:"description"`
    Input          attributes.Attributes `yaml:"input"`
    Patch          mutation.Patch        `yaml:"patch"`
    ExpectedOutput attributes.Attributes `yaml:"expected_output"`
}

Execute File

To test the patching mechanism we have a testing function which takes the input, patch and expected_output as parameters:

func testPatchPerform(
    t *testing.T,
    patch mutation.Patch,
    input attributes.Attributes,
    expectedOutput attributes.Attributes
) {

    // ...

}

So know we need to call it for each test case from each test data file.
To do so I created a test helper which parses a test file and runs all the test cases in it (with the above helper). For each test-case I added a t.Run() which discribes the test being executed. This simplifies debugging a lot.

func testPatchPerformFromFile(t *testing.T, file string) {

    yamlInput, err := ioutil.ReadFile(file)
    require.NoError(t, err)

    testFile := TestFile{}

    err = yaml.Unmarshal(yamlInput, &testFile)
    require.NoError(t, err)

    for _, testCase := range testFile {
        t.Run(file+"/"+testCase.Description, func(t *testing.T) {
            testPatchPerform(t, testCase.Patch, testCase.Input, testCase.ExpectedOutput)
        })
    }

}

Now we just need to go over all the test files in our data directory and execute the testPatchPerformFromFile for each file. So the actual top-level test function that will be executed by go test looks like this:

func TestPatchPerformFromTestDataDirectory(t *testing.T) {

    err := filepath.Walk("./testdata/", func(path string, info os.FileInfo, err error) error {

        if err != nil {
            return err
        }
        if info.IsDir() {
            return nil
        }

        if strings.Contains(info.Name(), "patch_") {
            testPatchPerformFromFile(t, path)
        }

        return nil
    })
    require.NoError(t, err)
}

Test Output

Test about in verbose mode looks like this:

--- PASS: TestPatchPerformFromTestDataDirectory (0.00s)
    patch_increment.yml/inc (0.00s)
    --- PASS: TestPatchPerformFromTestDataDirectory/testdata/patch_increment.yml/inc_variable_number (0.00s)
    --- PASS: TestPatchPerformFromTestDataDirectory/testdata/patch_increment.yml/dec (0.00s)
    --- PASS: TestPatchPerformFromTestDataDirectory/testdata/patch_increment.yml/dec_variable_number (0.00s)

Conclusion

With this data-driven testing approach we can easily write tests. We implement the test script only once, and after that we can add as many data files as possible. Need a new test case? Just create a new case in a Yaml file and run the tests again with go test.

Data-driven testing also makes it possible to reuse test-cases in other places/languages in your stack since the Yaml test input is language-independent.

Running `go fmt`, `goimports` and `golangci-lint` on save with GoLand

Recently I started using GoLand for Go development. This means that I’m constantly adapting to this new editor and looking up how to do certain things.

One of the things I really liked in Visual Studio Code was the formatting/linting/goimports on save. It appears that it is super simple to enable this in GoLand.

Go to Preferences, Tools, File Watchers. Click on the plus sign and add the wanted Go tools there:

How to convert all your old AppleWorks and ClarisWorks documents to PDF?

Recently someone asked me if I could help him open old documents on his Mac. Those documents were made in 1997 with ClarisWorks (ClarisWorks is de predecessor of AppleWorks) and can’t be opened with any version of Pages (not even the oldest iWork version that runs on Intel Macs).

Luckily, there is still a way to open these documents.

How to open old AppleWorks and ClarisWorks documents?

How do you actually open old .cwk files on your new Mac? AppleWorks will surely not work, because it requires a PPC Mac or Rosetta Code, which doesn’t ship anymore for ages. Luckily you can open any old AppleWorks and ClarisWorks file with LibreOffice.

Converting all the old .cwk documents on your Mac to PDF

But it is very inconvenient to do this by hand for all the old documents on your Mac. That is why I programmed a small Python script which converts all the .cwk suffixed documents in a folder (or any of its subfolders) to PDF using LibreOffice.

So how to use it?

Install LibreOffice first in the /Applications folder.

Download my cwk_to_pdf.py script from Github.

Then open the Terminal application on your Mac and execute the script while passing a the folder with the .cwk files to it.

$ python cwk_to_pdf.py /some/folder/with/cwk/files

If you don’t know how to navigate in the Terminal or work with relative directories in the Terminal, you can simplify the process by:

  1. Open the Terminal application
  2. Type python
  3. Type a space
  4. Drag and drop the cwk_to_pdf.py file in the terminal
  5. Type another space
  6. Drag and drop the folder you want to run the script on in the terminal
  7. Hit enter

While executing, the cwk_to_pdf.py script will go through all files and subfolders in the specified directory, and will convert all files ending with .cwk to PDF. It will save those files in the the same directory.

Don’t forget to backup your files before running scripts like this! (And in fact you should always backup, not only when you run stuff!!!)

What I learned from working as an expat in Paris

Enjoying a last glass of wine in Paris at one of my favorite spots.

Starting a new professional adventure is the ideal moment to look back on a previous experience. In my case, I worked during the last year for Scaleway in Paris.

“Scaleway is an Iliad Group brand supplying a range of pioneering cloud infrastructure covering a full range of services for professionals. Scaleway is growing its reputation around the world and currently serving business clients in four datacenters located in France and one in the Netherlands.”

Working in Paris taught me some things about the French culture…

Taking things easy

In France, they take things easy. Every meetings starts with at least ten minutes delay. This can be frustrating if you are used to starting on time. But the French way of living also has its advantages: sweet lunch break where you can take your time to enjoy freshly made meals. During the heatwave my manager even told me “Take another glass of rosé wine and take it easy in this hot weather”.

Sadly though, the cassiers in the supermarket also take it easy when you don’t necessarily have a lot of patience 🙂

Working together with people from different backgrounds is nice

The fact that in France they take things more at ease is not the only difference. Working in France, I quickly noticed that most French need a lot more words to express the same information. This makes meetings sometimes cumbersome or tiring due to long discussions. But this also makes that my colleagues by default could express themselves with more nuances.

French ❤️ Paperwork

French companies and governmental agencies really do like paperwork. They want a ‘justificatif’ prove for everything. Obtaining a bank account and social security quickly took some months due to the severity on the necessary paperwork.

Language will always be a limiting factor

Even tough my French is very good (people in France sometimes ask me where in France/Belgium I come from with my accent), I eventually stumbled upon a language barrier. Calling with bad quality to the French social security or bank when they speak very quickly can be a challenge. Or understanding all the jokes of my colleagues when they speak with the typical Parisian ‘verlan’ slang words. But luckily, my French has even gotten better, I learned a lot of new words and expressions!

Working at Scaleway was both challenging and fun

I really enjoyed working at Scaleway. I had the possibility to learn a lot of new technologies on interesting projects. This all while working with passionate and smart people.

Paris is an awesome city

Hell yeah, Paris is an awesome city. I had visited Paris plenty of times before moving there. But even after a year, I was still discovering cool places, nice dishes and fancy restaurants. Paris is one of those cities where you have always plenty of stuff to do and visit. Paris never sleeps.

HTTP Test Recording with Go and go-vcr

When testing code that interacts with HTTP APIs, it is often cumbersome to test them, and especially to automatically test them with real data as part of a continuous deployment process.

To tackle this, it is useful to record your API requests and responses. This can be done in Go by using go-vcr, which gives you a http.Transport you can use in a custom http.Client.

go-vcr uses the concept of a recorder and a cassette. The recorder will save a request-response mapping. This is stored in a cassette.

Creating http.Transport with go-vcr

Creating a custom http.Transport is very straightforward. In fact, the recorder struct implements the http.Transport interface. So it suffices to create a recorder.
In my example I change the mode of the recorder (recording vs replaying) based on whether the UPDATE environment flag is set.
The cassette is stored in testdata/go-vcr, because testdata directories are ignored by Go.

// UpdateCassette ENV variable so we know when to update the cassette.
_, UpdateCassette := os.LookupEnv("UPDATE")

recorderMode := recorder.ModeReplaying
if UpdateCassette {
    recorderMode = recorder.ModeRecording
}

// Setup recorder
r, err := recorder.NewAsMode("testdata/go-vcr", recorderMode, nil)
if err != nil {
    return nil, nil, err
}

Deleting Sensitive Information

Since your tests are version controlled, it is very important to delete all sensitive data from the cassette. This can be done by adding filters to the recorder. Below, I delete the x-auth-token and authorization headers. But depending on the HTTP API you are testing you might need to delete more sensitive data.

// Add a filter which removes Authorization and x-auth-token headers from all requests
r.AddFilter(func(i *cassette.Interaction) error {
    delete(i.Request.Headers, "x-auth-token")
    delete(i.Request.Headers, "authorization")
    return nil
})

Doing HTTP Requests with the Custom Transport

Now you have a recording transport, you can start writing tests. First create a HTTP client with the custom transport. Once you have done this, performing HTTP requests and performing tests on them is business as usual…

// Create new http.Client where transport is the recorder
httpClient := &http.Client{Transport: r}

req, err := http.NewRequest("GET", "http://api.example.com/some/object", nil)
if err != nil {
    // handle error
}
resp, err := httpClient.Do(req)
if err != nil {
    // handle error
}

// perform tests on the response...

Go mobile example: running Caddy on iOS

I while ago I did a small experiment to run Caddy on my iPhone. I currently have no time to do something with it, and actually build a useful Caddy iOS app, but I wanted to do a quick writeup about how you can achieve this. So others can maybe do something with Caddy on iOS, because compiling a Go project for iOS is really easy!

Installing Go Mobile

You first need to install Xcode and the Xcode command line tools.
Then you can simply install Go Mobile with the following commands:

$ go get golang.org/x/mobile/cmd/gomobile
$ gomobile init # it might take a few minutes

More information can be found in the official docs.

Building an iOS framework bundle from Caddy

Now you have to create a framework bundle of the Caddy project, which you can then use in your iOS application:

$ gomobile bind -target=ios github.com/mholt/caddy/caddy/caddymain

This will generate a framework called Caddymain.framework.

Building an iOS app with the framework

Now you can just drag and drop the compiled framework bundle to the Xcode project of the iOS app.
Check “Copy items if needed” if you want to include a copy of the framework bundle within the Xcode repo. Otherwise, modifying the Go package source code and rerunning gomobile bind ... will update the Caddymain.framework in the app.

If everything went well, your Xcode project overview will look like this:

Once this is done, you can import the Caddymain framework in your Swift files and call the functions from the Caddy package and use them to start Caddy.

I put everything in my ViewController:

import UIKit
import Caddymain

class ViewController: UIViewController {

    override func viewDidLoad() {
        super.viewDidLoad()
        // Do any additional setup after loading the view, typically from a nib.

        // Don't forget to compile with ENABLE_BITCODE = 'No'
        // Startup Caddy...
        CaddymainRun()

    }

    override func didReceiveMemoryWarning() {
        super.didReceiveMemoryWarning()
        // Dispose of any resources that can be recreated.
    }


}

Now you can hit run, and you’ll be running Caddy on your iPhone!

Note: since Go mobile doesn’t compile frameworks with bitcode enabled, you have to turn bitcode off in the project settings in Xcode: Go to the Build Settings tab. Type bitcode in the search field and change the Enable Bitcode setting to No.

Conclusion

Running Go code on an iOS device is easy!
So don’t hesitate to do some useful stuff with it yourself!

Serving self-hosted Invoice Ninja with Caddy

Invoice Ninja logoInvoice Ninja is an open-source platform which helps you take care of clients, invoices, payments, expenses, time-tracking, and more…
In this guide I explain how you can run Invoice Ninja on your own server with Caddy and PHP-FPM.

Prerequisites

  • MySQL Server up and running
  • PHP-FPM up and running
  • I assume that you have already setup Caddy and know it’s basics. I have other guides if you need help running Caddy with systemd or upstart.

Download the latest version of Invoice Ninja

Download the latest version here: Download Invoice Ninja.

Setup the database

Open the MySQL console:

$ mysql -u root -p

Execute the following SQL statements to create the database for Invoice Ninja, and grant a new user access to it:

CREATE DATABASE ninja;
CREATE USER 'ninja'@'localhost' IDENTIFIED BY 'ninja';
GRANT ALL PRIVILEGES ON ninja.* TO 'ninja'@'localhost';

Caddyfile

I use the following Caddyfile for Invoice Ninja:

invoiceninja.domain.com {
    root sites/invoiceninja/public
    fastcgi / unix:/var/run/php/php7.0-fpm.sock php
    log sites/invoiceninja/access.log
    errors sites/invoiceninja/error.log
    
    // Rewrite rules for Invoice Ninja (Laravel)
    rewrite {
        r .*
        ext /
        to /index.php?{query}
    }
}

invoiceninja.domain.com will automatically be served over HTTPS.

Installing Invoice Ninja

Installing Invoice Ninja is then straightforward: point your browser to invoiceninja.domain.com, and follow the on-screen steps.
I used Sendgrid as email provider, but you can also use your local mail server or any email service.

Invoice Ninja setup (database and general)
Invoice Ninja installation wizard.

Benchmarking a baremetal Scaleway server

This week I needed database (RDBMS) and PHP (frameworks) benchmarks for the Scaleway C2M server. Whilst doing so, I thought it would be useful to fully benchmark the server and share it in a blogpost.

CPU & Memory

I tested the dedicated server (C2M) with 8 cores, 16GB ram and 50GB storage (running Ubuntu 16). At the time of writing, this server costs 17,99 euro/month.
This baremetal box runs an Intel Atom C2750 processor clocked at 2.4GHz

Geekbench

Geekbench 4 scores:

Single-Core Score Multi-Core Score
1187 6031

De detailed Geekbench result can be found here: Scaleway C2M

SysBench

Multi-Core SysBench CPU benchmark:

Test execution summary:
    total time:                          5.7415s
    total number of events:              10000
    total time taken by event execution: 45.8982
    per-request statistics:
        min:                                  4.59ms
        avg:                                  4.59ms
        max:                                 10.45ms
        approx.  95 percentile:               4.59ms

Threads fairness:
    events (avg/stddev):           1250.0000/1.12
    execution time (avg/stddev):   5.7373/0.00

MySQL database performance

MySQL performance benchmarks using SysBench on a test database with respectively 1M and 10M records:

MySQL Benchmark Transactions Transactions/s Records Time (s)
1 36528 608,73 1000000 60,0072
2 41212 686,29 1000000 60,0500
3 27503 458,34 10000000 60,0054
4 34156 569,19 10000000 60,0078

Scaleway MySQL database benchmark

PHP frameworks performance

PHP requests per seconds using Apache 2 and a simple controller in the given framework (using PHP Framework Benchmark):

Benchmark
(requests/s)
bear-1.0 fuel-1.8 symfony-2.7 symfony-3.0 laravel-5.2 lumen-5.1 zf-2.5
1 350,27 889,72 39,57 356,89 294,87 1239,55 143,75
2 512,64 926,63 125,72 373,66 321,85 1246,79 154,66
3 472,75 932,44 124,33 381,75 326,33 1233,77 155,71
4 569,57 927,51 124,47 380,31 321,88 1254,69 153,92
avg 476,3075 919,075 103,5225 373,1525 316,2325 1243,7 152,01

Scaleway PHP frameworks benchmark

Network speed

Network speed tested with Speedtest.net to different locations around the world:

Speedtest Ping (ms) Download (Mbps) Upload (Mbps) Distance (Km)
France Paris 2,126 933,77 450,86 1,88
Netherlands Haarlem 12,643 803,83 313,28 423,92
Russia Moscow 61,75 316,53 126,94 2487,01
United States New York 131,036 146,30 41,29 5836,26
United Arab Emirates Dubai 160,098 98,37 29,97 5246,32
United States San Francisco 191,342 99,96 49,63 8952,54
Singapore Singapore 233,212 35,85 28,26 10732,95
Australia Sydney 365,136 52,51 19,59 16960,81


Scaleway latency ping global benchmark

Scaleway global network connectivity speed

Disk

FreeVPS.us benchmark

Disk I/O benchmark with freevps.us:

# I/O speed
1 106 MB/s
2 136 MB/s
3 88 MB/s
avg 110 MB/s

vpsbench

Disk I/O benchmark with vpsbench:

# I/O speed
1 153 MB/s

SysBench

Disk I/O benchmark with Sysbench:

Extra file open flags: 0
128 files, 320Mb each
40Gb total file size
Block size 16Kb
Number of random requests for random IO: 0
Read/Write ratio for combined random IO test: 1.50
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random r/w test
Threads started!
Time limit exceeded, exiting...
Done.

Operations performed:  73500 Read, 49000 Write, 156744 Other = 279244 Total
Read 1.1215Gb  Written 765.62Mb  Total transferred 1.8692Gb  (6.3795Mb/sec)
  408.29 Requests/sec executed

Test execution summary:
    total time:                          300.0315s
    total number of events:              122500
    total time taken by event execution: 42.8032
    per-request statistics:
         min:                                  0.00ms
         avg:                                  0.35ms
         max:                                 11.18ms
         approx.  95 percentile:               1.07ms

Threads fairness:
    events (avg/stddev):           122500.0000/0.00
    execution time (avg/stddev):   42.8032/0.00

Building a macOS Server hackintosh with an Intel NUC

Hackintosh macOS serverLast week I built a hackintosh server for macOS server. This machine replaces an old Intel Atom server running Ubuntu.

Hardware

For this project I bought an Intel NUC 6i3SYH with 8GB DDR4 ram and a Western Digital 256GB m2 SSD. The box version of the NUC allowed me to add another 2,5″ SATA disk for storage.

For the Time Machine backup service I used external harddisks.

macos-server-hackintosh-nuc-6i3syh-specs

Software

I downloaded the latest macOS Sierra from the Mac App Store and installed it using Unibeast and Clover.

I followed this Hackintosh guide. The tutorial is very complete, so it useless for me to try and reproduce it in this blogpost. (Make sure to use the correct tutorial for your hardware. The tutorial I linked is for the Skylake version.)

Once macOS was successfully installed, I could download the Server app from the App Store and configure everything on the machine.

Don’t forget to disable auto sleep in the System preferences or your Mac will go in standby after a short period of inactivity.

Configuring macOS server

Remote macOS Server management

You can manage your headless Mac server with the Server app from another Mac. Just download de Server app on the other Mac and select the server. This will give you a nice interface from which you can manage your server.

File sharing

Creating a shared volume is very easy. Go to File sharing in the navigation and do it yourself! ? It’s very intuitive, like all Apple products, so I won’t explain it here…

macOS server file sharing

Time Machine server

You can easily add a Time Machine volume in the server interface: Go to the Time Machine service in the left column. Then add a volume (set the correct permissions and limitations if you want) and enable Time Machine.

macOS server Time Machine backup

On your client Mac, go to Time Machine in the System Preferences, select Add backup disk and choose your new network volume. I had to restart my MacBook before the network drive showed up in the Time Machine settings.

Time Machine network backup

VPN server

Configuring the VPN server can sometimes be a bit harder because you don’t always get relevant error messages when the client can’t connect to the server.

I generated a new shared secret (make sure that it has the correct length, otherwise you might end up with useless error message). Then I choose IP addresses which are not in the range of the DHCP IP’s of my router. The DNS server are by default the same as your server, but you can add any reachable DNS server in there. You could e.g. use Google DNS (8.8.8.8 and 8.8.4.4).

macOS Server VPN configuration

If you’ve configured everything, you can start the VPN server and connect to it with your Mac or iPhone. The VPN type you need to choose is L2TP.

Caching server

I had unfortunately no luck yet configuring the Caching server. Everywhere on the internet I read it’s a piece of cake: just enable the service and it should work. Well… my clients aren’t downloading through the caching server ?.

Configuring advanced command-line and development

The second part contains all the ‘expert’ ? configuration I did, i.e. anything which isn’t provided through the Server app config panel.

SSH authentication

First you must enable SSH access in the OS X server panel. You can do this in the main settings. You can connect to SSH using your username and password, but if you’re server is accessible outside your homenetwork, it’s a lot safer to disable passwords and use SSH keys. If you don’t need SSH keys, you can skip the rest of this section. Else you keep reading…

If you haven’t yet generated a key pair, you can do it using this command:

$ ssh-keygen -t rsa -b 4096 -C "your_email@example.com"

Now copy your public key to the server:

$ cat ~/.ssh/id_rsa.pub | ssh user@192.168.0.184 'umask 0077; mkdir -p .ssh; cat >> .ssh/authorized_keys && echo "Key copied"'

Then disablee password login for SSH: Edit the file /etc/ssh/sshd_config and add the following directives at the end of the file:

PermitRootLogin no
PasswordAuthentication no
PermitEmptyPasswords no
ChallengeResponseAuthentication no

Then restart the SSH (or Remote login as Apple calls it) service.

Install Xcode and command line tools

First you need to install Xcode from the Mac App Store. Once installed, you also need the Command Line Tools, you can initiate the download process by executing $ xcode-select --install in the Terminal.

MacPorts

Download MacPorts from the official website. And install it.

Install Fish shell

$ sudo port install fish

Manually set your PATH correctly for Fish so it finds MacPorts binaries. Add the following line to ~/.config/fish/config.fish:

set -xg PATH /opt/local/bin /opt/local/sbin $PATH

Download YouTube videos to iTunes server

I created an shared iTunes library on my server in which I save YouTube movies I can then play from other Macs in the network. To download a youtube movie and automatically add it to that library, you need the following command:

$ cd /Volumes/Arry/iTunes\ Youtube/iTunes\ Media/Automatically\ Add\ to\ iTunes.localized/
$ youtube-dl -f "[ext=mp4]" "https://www.youtube.com/watch?v=BzqjhC2OYnM"

Don’t forget to run iTunes as a login item.

Disable webserver on port 80

If you want to run your own web server, you must stop Apache on port 80 and 443:

$ sudo launchctl unload -w /Applications/Server.app/Contents/ServerRoot/System/Library/LaunchDaemons/com.apple.serviceproxy.plist

I run Caddy server on my Mac. Read this blogpost about running it as a service: Running Caddy as a service on macOS X server

Dynamic DNS

If you are hosting the server at home, your IP might sometimes change. You can solve this by buying a domain name and using dynamic DNS for it. To do so you need to execute a script which your DNS provider gives you. For ClouDNS the command looks like this:

$ wget -q --read-timeout=0.0 --waitretry=5 --tries=400 --background https://ipv4.cloudns.net/api/dynamicURL/?q=...

With cron you can create a job which executes the command for your dynamic DNS on your given time interval. This opens the crontab editor:

$ crontab -e

Running a Git server

To host my git repositories I run Gogs on my Mac. More info in this blogpost: Running Gogs (Go Git Service) as a service on macOS X server.

Gogs go git service