Interactive visualisation of Pi and friends with D3.js

Inspiration

I recently stumbled onto the magnificent posters created by Martin Krzywinski over at http://mkweb.bcgsc.ca/pi/. He’s come up with some truly original ways to illustrate the appearance and complexity of various irrational numbers.

Specifically, I really liked the minimalism and simplicity of the 2013 edition, shown here:

pi-dots-01

The rules used for generating the coloured dots is fascinatingly simple. Each digit 0-9 is assigned a unique colour. The i’th circle is then coloured according to the value of the i’th digit of Pi. The smaller circle inside is coloured based on the value of the following digit.

Simple rules, complex outcome.

Martin created other versions as well, e.g. one where adjacent equals are connected by same coloured lines. They’re all fascinating and you can even buy them as posters!

Interactive

Looking at these posters I couldn’t help wonder, what lay beyond the chosen boundaries for each poster. What if some great pattern or sequence was lurking just outside of view? If only there was a way to go explore further digits and other constellations using the same visualisation format.

Always the tinkerer, this got me thinking about how to create something like that. One thing led to another and before long I had a rough prototype cobbled together with D3.js.

I spent a bit more time adding a few input controls and polishing it into a neat little demo. I’ve put it up at:

winski.rhardih.io

Go try it out! Instructions are at the bottom.

As an example, this is how the Feynman Point looks, at a column width of 31:

Screen Shot 2016-06-10 at 12.30.29

For the more curious out there, I’ve put the code on GitHub at github.com/rhardih/winski.

Happy π hunting!

Retro gaming part 1 – The Elder Scrolls: Arena

Why

These days, The Elder Scrolls V: Skyrim is all the rave, and it does look like an awesome game. I’ve briefly tried out Morrowind back in the day, but not enough to really remember anything from it.

I’ve decided that I want to try Skyrim at some point. I’m perfectly sure it is a complete standalone installment in the series, but then again… Why not get familiar with the game-world by playing it’s predecessors first?

  • A proper waste of time? ✓
  • An endeavor that will take months and months, if only playing a little now and then? ✓
  • Great fun? ✓

With this in mind, I better get crackin’.

The games

Bethesda Softworks has released the first two Elder Scrolls games as free downloads from their website. They are both targeted for MS-DOS so to play them on my mac, they have to run in DOSBox. So far I’ve only gone ahead with Arena, but it seems to run without error on DOSBox.

Cover art from Wikipedia.

Playing

Getting over the grueling 17 year old graphics and interface takes a bit of patience and playing the game without a manual or other instructions does provide some initial frustrations. Luckily some guy was awesome enough to put this online: The Elder Scrolls Arena Player’s Guide, so it wasn’t entirely impossible to figure out, how to get started.

From what I’ve gathered so far, the main quest revolves around finding eight pieces of The Staff of Chaos. When found the staff can be resembled and used to defeat the evil wizard who has taken the kings place, and bring back the true kind from another dimension. Good stuff.

So far I’ve only found the first two pieces, but I’m firmly determined to play the game through. No matter if it will take close to – forever.

Let’s see if I ever get to Skyrim.

Latest tinkerings – simply-json

Intro

Working with JSON data it is sometimes necessary to visualise it in a human readable way. Since we care about the number of bytes we send to the browser, JSON is usually stripped of any kind of unnecessary whitespace.

Good for size, bad for readability.

Doing a quick search reveals that there are already plenty of options for online formatters. Here’s just a few examples:

For their intended purpose, they all work – Trying them out though, I started to think of how it could be done in a simpler way and that it might be a fun little project, to try out on my own.

Application

For the lack of a better name, as the post title suggests, I decided to call it simply-json, feel free to suggest a more fitting name.

I wanted a minimal feature set, mostly what all the other formatters also provide:

  • Input a URL which points to some JSON data. Have it fetched and then formatted.
  • Input raw JSON and have it formatted.

I also included some design criteria, to formalise my idea of what “simpler” is:

  • No page noise. Content that isn’t relevant to solving the task at hand, should be kept to an absolute minimum.
  • No buttons. Why should I have to click a button, when the browser is perfectly capable of detecting when I’ve input some text in a textfield?

Supplemental features:

  • Highlighting of matching brackets. A feature I like, that some editors have – When moving the cursor over a bracket, the matching opening or closing bracket is indicated.
  • Collapsible regions. Clicking a bracket should collapse the content. Also an editor feature.
  • Loading indication when fetching data via URL. (http://www.ajaxload.info/)
  • Unobtrusive error indication. Borders around text boxes goes red on error.

Considerations

Due the to the same origin policy enforced by most browsers, it’s not immediately possible to request JSON from a different domain than the current one. To do it, some form of proxy is needed.

An easy, fire-and-forget solution would be to use Yahoo! Query Language. The problem with YQL though, is that it transforms the JSON into XML and, if requested as JSON, transforms it back again to JSON. This transformation is lossy, which means e.g. numbers are sent back as strings. {"number":42} => {"number":"42"}. According to the docs:

To prevent this “lossy” transformation, you append the query string parameter jsonCompat=new to the YQL Web Service URL that you are using.

At the time of writing, testing this does in fact reveal, that the loss in number transformation is fixed. What hasn’t been fixed yet though is null values, which is returned as "null" strings instead.

So much for YQL.

Custom proxy

Keeping things in the spirit of simplicity, a service that can proxy a GET request, shouldn’t be more than a few lines of code.

To that end, I chose to use Sinatra, which I’ve had good experiences with in the past. It really is an awesome lightweight web framework. Using Sinatra, this is all it takes:

require 'rubygems'
require 'sinatra'
require 'net/https'
 
get '/' do
  uri = URI(URI.encode(params[:uri]))
  https_session = Net::HTTP.new(uri.host, uri.port)
  https_session.use_ssl = true if uri.port == 443
  https_session.start
  response = https_session.get(uri.path + "?" + uri.query)
  response.body
end

N.B. Updated Feb 10, 2012. New version includes uri query part.

I specifically chose 'net/https' so https sources were also supported, it only adds one extra line and can handle plain http just as well.

The proxy should be very straightforward, assuming the endpoint is at thelabs/simply-json/proxy, making a request to "~thelabs/simply-json/proxy?uri=http://google.com" should be the equivalent of a request directly to "http://google.com".

The immediate problem though, is that the current web server, Apache, is already running on port 80 and the same origin policy even blocks requests to different ports on the same domain.

Fortunately, this can be solved with some config magic server-side.

Apache

Firing up the Sinatra service on localhost:6789, we need Apache to redirect traffic to the /proxy path, back to the service instead of trying to serve content from that directory. This is what is needed in the config, in my case in the virtual host for éncoder.dk:

<Proxy *>
Order deny,allow
Allow from all
</Proxy>
 
ProxyPass /thelabs/simply-json/proxy http://localhost:6789
ProxyPassReverse /thelabs/simply-json/proxy http://localhost:6789

In case Apache hasn’t loaded mod_proxy, enabling it can be done with this command:

$ a2enmod proxy proxy_http

That’s all

Working demo can be found here:

http://éncoder.dk/thelabs/simply-json/

Go check it out.

Using ncurses in C for Text User Interfaces, featuring Xcode

Premise

Being the *nix fanboy that I am, I love having terminal access to my system. Most *nix based OS’s have the same base set of awesome command-line tools. The majority of these are simply “set and run” programs, but some have Text User Interfaces, (TUI), as well. A few of my favourites include screen, bmon, htop and lynx. Be sure to check those out if you haven’t already.

If you’ve ever written a small command-line program that relied on any kind of user input, you’ve probably already coded your own rudimentary menu system at some point. It’s really not that difficult, but.. things can quickly get messy and you’d soon wish you had found an easier way of handling terminal control. Enter ncurses (new curses), a library for writing terminal-independent TUIs.

The project

There’s quite a few run-of-the-mill tutorials for curses out there, but doing a traditional “Hello World!” program just feels so uninspired. Instead we are going to do a simplified version of the classic game Snake, let’s call it PieceOfCakeSnake. In PieceOfCakeSnake you win simply by playing, there is no opponents, no consumables and no way of dying. Just a single, fixed-size, snake moving around in it’s little box world. The game starts right away upon launch and Snakey, our main character, is moving happily along from the get-go. The game ends when ‘x‘ is pressed.

Hammer and chisel

For no particular reason, I’m going to use Xcode for PieceOfCakeSnake and write it in C. If you want to use another editor be my guest, it’s much the same since it’s going to be run in a regular terminal anyway.

Now fire up Xcode and create a new project. Chose “Command Line Utility” > “Standard Tool”. Name the project and save it where you want.

Xcode have already created a main.c file for us that simply outputs “Hello World!” and exits. Click “Run” > “Console” and click “Build and Go”. If all is well, the program builds without error and you should see something like this:

Now, ncurses comes native with Mac OS X, but for other systems you might need to install it beforehand. E.g. for Ubuntu there is the libncurses5 package.

There is still a little more to be done before we start coding. To get access to all the ncurses functions we have to tell the linker, to include the library at compile time. This is done by adding the line #include <ncurses.h> at the top of main.c and by supplying the linker flag -lncurses to the compiler. If you are compiling this from the terminal with GCC, the command would be:

gcc main.c -lncurses -o pocs main.c

Where pocs is the resultant executable. In Xcode however, we rely on the provided build system and so, the linker flag is set in the project properties. Click “Project” > “Edit Project Settings”, chose the “Build” tab and find the field named “Other Linker Flags” and insert “-lncurses”.

That should do it, we are set to go.

Let’s see some code

In this first step, we’ll create a world for Snakey, a square box positioned in the middle of the terminal screen. Here’s the code:

#include 
 
#define WORLD_WIDTH 50
#define WORLD_HEIGHT 20
 
int main(int argc, char *argv[]) {
 
    WINDOW *snakeys_world;
    int offsetx, offsety;
 
    initscr();
    refresh();
 
    offsetx = (COLS - WORLD_WIDTH) / 2;
    offsety = (LINES - WORLD_HEIGHT) / 2;
 
    snakeys_world = newwin(WORLD_HEIGHT,
                           WORLD_WIDTH,
                           offsety,
                           offsetx);
 
    box(snakeys_world, 0 , 0);
 
    wrefresh(snakeys_world);
 
    getch();
 
    delwin(snakeys_world);
 
    endwin();
 
    return 0;
 
}

Running this example, you should see something like this:

Notice the WINDOW type. With ncurses everything is drawn on windows. By default, ncurses sets up a root window, stdscr, which backdrops the current terminal display.

To use it we call initscr(), which prepares the terminal for curses mode, allocates memory for stdscr and so forth.

The windows in ncurses are buffered, in the sense that you can do multiple drawing operations on a window, before making them show up on screen. To display the contents of a window in the actual terminal, the window needs to be refreshed.

For stdscr, this is done by calling refresh(), for child windows we use wrefresh(). This also shows the easy-to-remember naming convention used in the ncurses library – most functions that can be applied to stdscr, also has a counterpart, which applies to child windows, simply named by prepending a ‘w’ to the function name. E.g. refresh() and wrefresh(). We’ll se more of this in the finished version.

Instead of drawing the box manually, we take a shortcut by creating a new window and using the function box() to draw a border around the window. box() can use any displayable character to draw the borders. Using 0 defaults to a system specific line character.

Note: COLS and LINES are environment variables, that holds the current width and height of your terminal. That is the number of horizontal and vertical character positions available in the window.

The getch() function is simply there to pause program execution until some keyboard input is received. Thus a key press exits the program.

Functions delwin() and endwin() handles memory deallocation and returns the terminal to it’s former state. If these are omitted, the terminal will not behave as expected upon program termination and will probably need to be reset.

Time for some action

Now for the fun part – putting Snakey in his box and getting him to move about. Since this entry is about ncurses, I’m not going to go into the mechanics of the game itself. It’s a very simple implementation and the code should be rather self explanatory. You can download the source file here or just read from the following:

#include <ncurses.h>
 
#define TICKRATE 100
 
#define WORLD_WIDTH 50
#define WORLD_HEIGHT 20
 
#define SNAKEY_LENGTH 40
 
enum direction { UP, DOWN, RIGHT, LEFT };
 
typedef struct spart {
    int x;
    int y;
} snakeypart;
 
int move_snakey(WINDOW *win, int direction,
                snakeypart snakey[]);
 
int main(int argc, char *argv[]) {	
 
    WINDOW *snakeys_world;
    int offsetx, offsety, i, ch;
 
    initscr();
    noecho();
    cbreak();
    timeout(TICKRATE);
    keypad(stdscr, TRUE);
 
    printw("PieceOfCakeSnake v. 1.0  -  Press x to quit...");
 
    refresh();
 
    offsetx = (COLS - WORLD_WIDTH) / 2;
    offsety = (LINES - WORLD_HEIGHT) / 2;
 
    snakeys_world = newwin(WORLD_HEIGHT, 
                           WORLD_WIDTH, 
                           offsety, 
                           offsetx);
 
    snakeypart snakey[SNAKEY_LENGTH];
 
    int sbegx = (WORLD_WIDTH - SNAKEY_LENGTH) / 2;
    int sbegy = (WORLD_HEIGHT - 1) / 2;
 
    for (i = 0; i < SNAKEY_LENGTH; i++) {
        snakey[i].x = sbegx + i;
        snakey[i].y = sbegy;
    }
 
    int cur_dir = RIGHT;
 
    while ((ch = getch()) != 'x') {
        move_snakey(snakeys_world, cur_dir, snakey);
        if(ch != ERR) {
            switch(ch) {
                case KEY_UP:
                    cur_dir = UP;
                    break;
                case KEY_DOWN:
                    cur_dir = DOWN;
                    break;
                case KEY_RIGHT:
                    cur_dir = RIGHT;
                    break;
                case KEY_LEFT:
                    cur_dir = LEFT;
                    break;
                default:
                    break;
            }
 
        }
    }
 
    delwin(snakeys_world);
 
    endwin();
 
    return 0;
 
}
 
int move_snakey(WINDOW *win, int direction,
                snakeypart snakey[]) {
 
    wclear(win);
 
    for (int i = 0; i < SNAKEY_LENGTH - 1; i++) {
        snakey[i] = snakey[i + 1];
        mvwaddch(win, snakey[i].y, snakey[i].x, '#');
    }
 
    int x = snakey[SNAKEY_LENGTH - 1].x;
    int y = snakey[SNAKEY_LENGTH - 1].y;
    switch (direction) {
        case UP:
            y - 1 == 0 ? y = WORLD_HEIGHT - 2 : y--;
            break;
        case DOWN:
            y + 1 == WORLD_HEIGHT - 1 ? y = 1 : y++;
            break;
        case RIGHT:
            x + 1 == WORLD_WIDTH - 1 ? x = 1 : x++;
            break;
        case LEFT:
            x - 1 == 0 ? x = WORLD_WIDTH - 2 : x--;
            break;
        default:
            break;
    }
 
    snakey[SNAKEY_LENGTH - 1].x = x;
    snakey[SNAKEY_LENGTH - 1].y = y;
 
    mvwaddch(win, y, x, '#');
 
    box(win, 0 , 0);
 
    wrefresh(win);
 
    return 0;
}

And here is what the game should look like in the terminal:


There is a few new ncurses functions being used here. Let’s start at the top. In main I’ve added:

noecho();
cbreak();
timeout(TICKRATE);
keypad(stdscr, TRUE);
printw("PieceOfCakeSnake v. 1.0  -  Press x to quit...");

From top to bottom. noecho() subverts the terminal from printing back the users key presses. This is useful, since otherwise we would quickly have a lot of garbage on-screen from using the arrow keys to guide Snakey.

cbreak() disables line buffering and feeds input directly to the program. If this wasn’t called, character input would be delayed until a newline was entered. Since we would like immediate response from Snakey, this is needed.

timeout() sets an input delay, in milliseconds, for stdscr, which is applied during input with getch() and sibling functions. If the user doesn’t input anything within this given time period, getch() returns with value ERR. Useful in this part of the code, where we would like Snakey to move, even when we are not pressing any keys.:

while ((ch = getch()) != 'x') {
    move_snakey(snakeys_world, cur_dir, snakey);
    if(ch != ERR) {
        ...
    }
}

The keypad() function enables or disables special input characters for a given window. F keys and arrow keys for example.

printw() works like the standard library function printf. That is print a given string at the current cursor location.

To separate things a little, we have an auxiliary function move_snakey(), which handles movement and redrawing of Snakey within the box. There is a few ncurses specific functions in there as well:

You could chose to add and remove individual characters if you want to be explicit, but I’m lazy, so I clear the whole window and redraw it again every time Snakey has moved. Clearing is done with clear() for stdscr and wclear() for child windows.

The last function to mention is mvwaddch(), which moves, notice mv, to coordinate x, y, in a child window, notice w, and adds a character at that position.

The mv prepend, like w, is also a part of the naming convention of ncurses. Thus most drawing operations have an extended version, that besides the item to be drawn, takes a set of coordinates of where to move the cursor before drawing it. E.g. printw() and mvprintw.

Goodbye Snakey

PieceOfCakeSnake is a very simple demonstration and only shows a very small subset of the features available with ncurses. For the inspired reader however, it should be no problem extending the game with a menu system, a scoreboard and more, using only the small part demonstrated here.

References and source