Computers and Questions

"Computers are useless. They can only give you answers." -- Pablo Picasso

Picasso nailed it philosophically. But computers are even more useless when they try to ask questions. From the vaguely annoying "Do you want to restart your computer now?" (No), through "Do you want to automatically recover your [document/tabs/settings]?" (No), through "Do you want to install updates?" (No), through "Do you want to subscribe to the pro version?" (No) all the way to "Do you want to sign up for Google Plus" (No). I can't think of a single instance when any piece of software has asked me a useful question.

Even worse is stacking several of these questions in a row. Signing into Gmail sometimes gives "Do you want to give us your phone number, for your security and convenience?" (No, I want to read my emails) "Are you sure, a few minutes now could save you hours of frustration in the future?" (Show me my damn email now), "Do you want to sign up for Google Plus?" (No, No, and for the fourth minus one time, No).

And yet you can go further to ensure maximum user frustration. Three or more stacked questions, all blocking, before you show the user what he originally asked for might sound bad enough; removing the "Never ask again" option is one step further. But to reach the levels of many modern apps, remove the "No" button entirely. The "Yes/No" dialogue has become the "Yes/Later" or "Yes/Not now" dialogue. Maybe even these are temporary luxury. How soon before this becomes the norm?

JQuery Accordion

Today I wanted to implement an accordion effect on a website. Unfortunately I hadn't heard the term "accordion" used in this way before, so it took a couple of badly thought-out Google queries before I chanced upon what I was looking for.

In case I'm not the last human on the planet who is not familiar with JQuery's accordion effect, it refers to text which is displayed and/or hidden by headings - so one can click on a header to expand the body. Probably the best way to work out what I mean is to see an example, so take a look at jqueryui.com/accordion.

It's quite neat, but not exactly what I was looking for. I found a much simpler solution which was exactly what I wanted: see jacklmoore.com/demo/accordion.html. Jack's full tutorial/explanation is at jacklmoore.com/notes/jquery-accordion-tutorial/

Adobe hack

Everyone is writing about the Adobe hack, where 150 million email addresses, encrypted passwords, and password hints were stolen, and some may consider it old news, as it happened in October.

However, over the last couple of days it is showing new life - Mainly because while the 10 GB text file was a little bit difficult to find before, it is now easily accessible to anyone who knows how to ask Google nicely. Also, Facebook, Evernote, and a couple of other large companies recently contacted users who may have used the same password in more than one place.

I don't remember signing up for an Adobe account, but my email address is in the list. I strongly advise that you go to https://lastpass.com/adobe/ and type in your email address(es) to check if yours was included, too. Do this whether or not you think you have an Adobe account. If you think you may have used the password on any other site, change it ASAP. Change it to something unique.

Some people are finding entertainment value hidden the hack: there was a particularly good xkcd cartoon about it, which one Ben Falconer decided to implement, allowing you to play at solving real crossword puzzles, using people's passwords as answers and their password hints as clues.

150 million is a big number. Really big. In fact, it's hard to comprehend exactly how big it is, and I was far more shocked by the number of students and lecturers from Rhodes University (several hundred) who appeared in the list than by the total number. Looking at the number of people who used "bank password", "same as bank password", or simply "bank" as a password hint (many thousands) also helps one realise how big 150 million is. Everyone with any sense is constantly advising everyone else to not use reuse passwords, ever. Everyone else is constantly ignoring them. But hopefully at least some people were have learnt their lesson.

And I really hope that Adobe learnt theirs.

Python quirks

My old favourite Python quirk

Reversing a list

  l = [1,2,3,4,5]
  print l[::-1]

Output:

[5,4,3,2,1]

My new favourite Python quirk (discovered today)

for j in range(3):
    print j
else:
    print "the last item was", j

Output:

0
1
2
the last item was 2

Yes, after coding in Python for all these years, I had no idea that one could write a "for... else..." loop. Not quite as useful as the list reversal one, but strangely appealing nonetheless.

Now try:

from __future__ import braces

And there are still some people who won't admit that Python is hands-down the best language there is.

How and When to Use Sqlite

Sqlite is a very simple and fast open source SQL engine. This tutorial will explain when it is optimal to use Sqlite, as opposed to a full-blown RDBMS such as Mysql or Postgres, how to install it, and will also provide some basic usage examples, covering CRUD - Create, Read, Update, and Delete.

Clearing up a few misconceptions

Don't be deceived into thinking that Sqlite is only for testing and development. For example, it works fine for websites receiving up to 100 000 hits a day. And this is a conservative limit. The maximum size for a Sqlite database is 140 Terabytes (should be enough, right?), and it can be substantially faster than a full-blown RDBMS, as the full database and all other necessary data is stored in a normal file in the host's file system, so no separate server process is needed, cutting out all need for slow inter-process communication.

Optimal Usage of Sqlite

Sqlite is focused on simplicity. Because it is completely internal, it is often significantly faster than alternatives. If you are looking for portability (with regards to both languages and platforms), simplicity, speed, and a small memory footprint, then Sqlite is ideal. Its shortcoming are only apparent if you need high reading or writing concurrency: Sqlite can only support one writer at a time, and the normally high file system latency may be inconvenient if there is a need for many clients to access a Sqlite database simultaneously. A final possible disadvantage is that its syntax, though similar to other SQL systems, is unique, so while it's fairly trivial to move to another system, if you do 'outgrow' Sqlite, there will be some overhead involved in the transition. There are some very good outlines on the pros and cons of Sqlite here.

Installation

The sqlite3 module is part of the standard Python library, so on a standard Ubuntu installation, or any system with Python installed, no further installation is strictly necessary. To install the Sqlite command line interface on Ubuntu, use the commands:

sudo apt-get update
sudo apt-get install sqlite3 libsqlite3-dev

If you need to rather compile it from source, then grab the latest autoconf version from sqlite.org/download.html. At the time of writing:

wget http://sqlite.org/2013/sqlite-autoconf-3080100.tar.gz
tar xvfz sqlite-autoconf-3080100.tar.gz
cd sqlite-autoconf-3080100
./configure
make
make install

(Notes for building from source: 1) Don't do this on a standard Ubuntu installation, as you'll probably get a "header and source version mismatch" error, due to conflict between an already installed version and the newly installed one. 2) If the make command seems to expect further input, just be patient, as the source can take a while to compile.)

Basic Command Line Interface Usage

To create a database, run the command:

sqlite3 database.db

Where 'database' is the name of your database. If the file database.db already exists, Sqlite will open a connection to it; if it does not exist, it will be created. You should see output similar to:

SQLite version 3.8.1 2013-10-17 12:57:35
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite>

Now let's create a table, and insert some data. This table, named "wines" has four columns - for an ID, the wine's producer, the wine's kind, and country of the wine's origin. As it's not Friday yet, we'll insert only three wines into our database:

CREATE TABLE wines (id integer, producer varchar(30), kind varchar(20), country varchar(20)); 
INSERT INTO WINES VALUES (1, "Rooiberg", "Pinotage", "South Africa");
INSERT INTO WINES VALUES (2, "KWV", "Shiraz", "South Africa");
INSERT INTO WINES VALUES (3, "Marks & Spencer", "Pinot Noir", "France");

We've created the database, a table, and some entries. Now press Ctrl + D to exit Sqlite, and type the following (again substituting your database's name for 'database'), which will reconnect to the database we just created:

sqlite3 database.db

Now type:

SELECT * FROM wines;

And you should see the entries we've just made:

1|Rooiberg|Pinotage|South Africa
2|KWV|Shiraz|South Africa
3|Marks & Spencer|Pinot Noir|France

Great. That's it for creating and reading. Let's do an update and delete:

UPDATE wines SET country="South Africa" WHERE country="France";

Which will update the database so all wines which are listed as coming from France will instead be listed as coming from South Africa. Check the result with:

SELECT * FROM wines;

And you should see:

1|Rooiberg|Pinotage|South Africa
2|KWV|Shiraz|South Africa
3|Marks & Spencer|Pinot Noir|South Africa

Now all our wines come from South Africa. Let's drink the KWV in celebration, and delete it from our database:

DELETE FROM wines WHERE id=2;
SELECT * FROM wines;

And we should see one fewer wine listed in our cellar:

1|Rooiberg|Pinotage|South Africa
3|Marks & Spencer|Pinot Noir|South Africa

And that covers all of the basic database operations. Sqlite has wrappers and drivers in all the major languages, and can run on most systems. A list of many of them can be found here. Good luck, and have fun.

A basic web app using Flask and MongoDB

This tutorial will show you how to set up a very simple database web application using the micro-framework Flask. We'll be using the Python library pymongo, which is available on PyPI and installable through PIP, and the free sandbox database at Mongolab. We'll use git with bitbucket to deploy the code to the server.

I assume that you already have root access to an Ubuntu server. This example was tried on a newly created Ubuntu 12.10 64-bit "Droplet" on Digital Ocean.

Included are very basic instructions to install Apache2 and mod-wsgi, too, but the emphasis of this tutorial is simplicity instead of security, so if you want instructions on setting up Apache2, read a tutorial specifically about that as well.

Let's begin with installing a few things we'll need. I've listed them all on separate lines, as you may have some of them already, and some of them are not absolutely necessary.

sudo apt-get install apache2 libapache2-mod-wsgi
sudo apt-get install libpcre3-dev
sudo apt-get install python-pip
sudo apt-get install build-essential python-dev
sudo apt-get install git-core

Apache2 is the web server we'll be using to serve our app, and libapache2-mod-wsgi is the wsgi mod for Apache which will let us use Python as a backend for a webpage. We'll use pip, a python package manager, to install Flask, which will allow us to easily interface with wsgi, and pymongo, which we'll use to connect to a Mongo database.

Note that the libpcre3-dev install is not strictly necessary, but pip will complain that it was unable to install the Flask C extensions necessary for speedups if you do not have it. Similarly, the build-essential and python-dev installs are not necessary, but are required for the C extension speedups for pymongo.

Now the python libraries we need

sudo pip install flask
sudo pip install pymongo

Make sure that you have pip, Flask, Pymongo, and git installed on your local machine as well. If you're using Ubuntu, follow the same steps above. I created this demo using Windows 7 as a local machine, and the tools are all available for Windows as well.

Next, we want to create a Mongo database. This is most easily done through Mongolab, which offers a free option. Head to mongolab.com, and create an account.

Create a new database, (for host options, the default AWS is probably the best option. Just select the location closest to your digitalocean server. As mine is in Amsterdam, I chose Ireland). Give it a name. Add a new user for the database and remember the username and password. Take note of the five pieces of information you need, given to you in the form of a url. See the screenshots below for a step-by-step guide on doing this.

Create a new database

Choose the free "development" server

Name your new database and press Create

Click on the database in the list to configure it

Click add a new user

Choose a database username and password.
(Note that this is different from your MongoLab username and password)

Make a note of the five bits of information we'll need in the python script.

Now create a new directory tree on your local machine with the following structure

demo
    \templates
    index.py
    index.wsgi

In index.py, type or copy the following, changing the database info in the connect() method as relevant.

from flask import Flask, render_template, request, redirect
import os
from pymongo import MongoClient

def connect():
# Substitute the 5 pieces of information you got when creating
# the Mongo DB Database (underlined in red in the screenshots)
# Obviously, do not store your password as plaintext in practice
    connection = MongoClient("ds053128.mongolab.com",53128)
    handle = connection["demo"]
    handle.authenticate("demo-user","12345678")
    return handle

app = Flask(__name__)
handle = connect()

# Bind our index page to both www.domain.com/ 
and www.domain.com/index
@app.route("/index" ,methods=['GET'])
@app.route("/", methods=['GET'])
def index():
    userinputs = [x for x in handle.mycollection.find()]
    return render_template('index.html', userinputs=userinputs)

@app.route("/write", methods=['POST'])
def write():
    userinput = request.form.get("userinput")
    oid = handle.mycollection.insert({"message":userinput})
    return redirect ("/")

@app.route("/deleteall", methods=['GET'])
def deleteall():
    handle.mycollection.remove()
    return redirect ("/")

# Remove the "debug=True" for production
if __name__ == '__main__':
    # Bind to PORT if defined, otherwise default to 5000.
    port = int(os.environ.get('PORT', 5000))

    app.run(host='0.0.0.0', port=port, debug=True)

Open index.wsgi and add:

import sys
sys.path.insert(0, '/var/www/demo')
from index import app as application

In the templates directory, create a new file called index.html. Add the following code: 

<html>
<head>
<title>Demo</title>
</head>
<body>
<h1>Hello</h1>
<form action="/write" method="POST">
<input type="text" name="userinput" id="userinput">
<input type="submit" value="Submit">
</form>
<a href="./deleteall">Delete all</a>

{% for userinput in userinputs %}
<p>'{{userinput.message}}' has database id {{userinput._id}}</p>
{% endfor %}
</body>

</html>

Try this out by running python index.py from the demo directory (or just index.py if you're in Windows.) Then open a web browser and navigate to http://locahost:5000

You should see a page like this:

The demo web app

You can submit text through the html form, and the results will appear below, displaying the Mongo Database _id associated with that entry.

Great - now head over to bitbucket.org and sign in or create an account. Press "Create" and create a new Git repo called demo. Now go to your demo directory in Terminal or Command Prompt and type (substituting your bitbucket username):

git init

git remote add origin https://sixhobbits@bitbucket.com/sixhobbits/demo.git

git add .

git commit -m "initial commit"

git push origin master

[enter your bitbucket password when prompted]

On your server, navigate to /var/www and type (again substituting your bitbucket name)

sudo git clone https://sixhobbits@bitbucket.com/sixhobbits/demo.git

[enter your bitbucket password when prompted]

And your code is ready. Now we just need to tell Apache where to find it. type

cd /etc/apache2/sites-available

ls

If you've just installed Apache you'll see a default file (and possibly a default-ssl one too). Type:

sudo a2dissite default

To disable the default site. Then type (substituting your domainname in place of flaskmongodemo.tk. You can get a .tk domain for free - just head over to dot.tk, find one that hasn't been taken yet, and point it to your digital ocean server's IP address).

sudo nano flaskmongodemo.tk

And paste the following code (I use vim for this step, but nano is installed by default on Ubuntu). Substitute your domain name in place of flaskmongodemo.tk.

<VirtualHost *:80>
    ServerName www.flaskmongodemo.tk
    ServerAlias flaskmongodemo.tk

    WSGIDaemonProcess www.flaskmongodemo.tk
    WSGIScriptAlias / /var/www/demo/index.wsgi

    <Directory /var/www/demo/index>
        WSGIProcessGroup index
        WSGIApplicationGroup %{GLOBAL}
        Order deny,allow
        Allow from all
    </Directory>

</VirtualHost>

Save your file (in Nano Ctrl + X and then Y when prompted) and type (again substituting your domain name)

sudo a2ensite flaskmongodemo.tk

sudo service apache2 reload

And it's done. You should be able to navigate to your url and see your app in action! Now that you can write to and read from a database, the possibilities are endless.

A useful command if things don't go as expected and you see an error instead of your app:

tail -f /var/log/apache2/error.log

This will show the error log of Apache, which is often helpful in tracing down cause of a problem.

The full app is currently at bitbucket.org/sixhobbits/demo.git, so you can simply clone it directly from there if you want.

Use all the keys on your keyboard - AHK

How often do you actually use the CAPSLOCK key? What about the "other" Alt and Control keys on the right-hand side of your keyboard? Or the AppsKey? Or the the much-hated "Windows" key? What about the left-right-up-down functions of your number pad when numlock is turned off?

Imagine if they could all easily be modified to do something useful.

Enter AutoHotkey (AHK), a free and very simple scripting language, which is designed to help you use your keyboard more efficiently. It can do anything from a simple remapping (make the 'a' key send a 'b' command, the 'b' key send a 'c' command, and annoy your friends and enemies), to creating full-blown GUI Applications.

Today, I'll be describing how you can use AHK for some simple window manipulation. Overusing a mouse or a touchpad leads to RSI (Repetitive Stress Injury), and also each time your fingers leave the keyboard you're wasting valuable time. Time you could be using in order to stop wasting time...

I'm assuming that you already know about the Windows shortcuts such as:
Alt + Tab switches between open applications
Win + Tab switches between open applications with a wheel-like graphic
Alt + F4 closes active window
Win + d shows the desktop

AutoHotkey is fairly straightforward to install. Head to the home page, press the download button, and run the .exe file. Once you've installed it, you should be able to right click on your desktop, select "New" and choose AutoHotkey Script from the menu. Name the file something like "hot keys", right click on it, and press "edit" to open it in notepad (Or press edit in Notepad++ if you have that installed).

So, let's start out by making use of one of the easiest to reach keys on a standard keyboard: The CapsLock key. We're not going to rob it of its primary function, so if you enjoy SHOUTING AT PEOPLE on forums, or you're a post-modernist writer who uses lots of block capitals to 'express yourself', fear not.

I often switch through a bunch of open applications, closing ones I don't need anymore. Normally this requires a bunch of Alt + Tabs interspersed by Alt + F4s, which is awkward as your middle finger keeps switching between the distant Tab and F4 keys, and your cramped thumb starts complaining that this wasn't what it went through all that evolution for.

Adding the lines below to your newly created AHK script will save you time and pain.

CapsLock & r::AltTab
CapsLock & e::ShiftAltTab
CapsLock & Space::WinClose,A

This means that holding down the CapsLock key and pressing 'r' will switch between your open applications. CapsLock and 'e' will do the same, but in reverse order. And most useful of all, pressing CapsLock and Space will close the active window. Now you never have to reach for that F4 key again. Just make sure that you release and repress the CapsLock key in between switching between applications and pressing the space bar, otherwise you'll end up quitting the Alt-Tab menu itself, and you won't be able to use Alt-Tab or Capslock + 'r' anymore (if you do do this by accident, the easiest way to restart the functionality is by logging out of windows and then back in again).

That was fun. But let's get a bit more inventive, and add some basic arithmetic. Windows 7 does a pretty neat job of snapping an application to exactly half the screen when you drag a window the the left or right, but it's a bit restrictive and it requires using the mouse, which we're trying to avoid. The few lines below will do a bit of arithmetic based on the size of your screen and snap the active window to the right or the left when you press Capslock and 'f' (right) or Capslock and 'd' (left). Note the "-40", which is to account for the 40 pixels that the taskbar needs. This works on a 15.6" laptop monitor with a standard sized Windows 7 taskbar. If you have your taskbar set to use small icons, or have a different sized monitor you'll need to play around with this number a bit to get it right. If your taskbar is set to auto-hide, you can remove it completely.

CapsLock & f::
{
WinRestore,A
WinMove,A,,A_ScreenWidth/2,0,A_ScreenWidth/2,A_ScreenHeight-40
return
}

CapsLock & d::
{
WinRestore,A
WinMove,A,,0,0,A_ScreenWidth/2,A_ScreenHeight-40
return
}

Now let's take that idea a bit further: the following lines will let you use CapsLock + 1, CapsLock + 2, CapsLock + 3, and CapsLock + 4 to make the active window take up exactly one quarter of the screen.

CapsLock & 1::
{
WinRestore,A
WinMove,A,,0,0,A_ScreenWidth/2,A_ScreenHeight/2
return
}
CapsLock & 2::
{
WinRestore,A
WinMove,A,,A_ScreenWidth/2,0,A_ScreenWidth/2,A_ScreenHeight/2
return
}
CapsLock & 3::
{
WinRestore,A
WinMove,A,,0,A_ScreenHeight/2,A_ScreenWidth/2,A_ScreenHeight/2-29
return
}
CapsLock & 4::
{
WinRestore,A
WinMove,A,,A_ScreenWidth/2,A_ScreenHeight/2,A_ScreenWidth/2,A_ScreenHeight/2-29
return
}

Now you can use use GMail, Facebook and VLC Media player, while doing some work in Microsoft Word. Isn't that productivity. And better still, try this. Put VLC in full-screen mode (open a movie and press F11), then press CapsLock + '2'. VLC will snap to the top right corner of screen, but remain in full-screen mode, so you won't have to sacrifice screen space to menu bars and borders. If you don't know what VLC is, it's the best media player out there, and it doesn't constantly complain about not being able to play your music and video files because they're in the wrong format: it plays anything*.

I can watch Top Gear and write at the same time...

Let's just add two more lines to our script:

CapsLock & m::WinMinimize,A
CapsLock & g::WinMaximize,A

So now we can maximize and minimize windows without the mouse as well. Just press CapsLock + 'm' to minimize the active window, and Capslock + 'g' to make the current window full size. And that's the end of window manipulation for now, but the possibilities of AHK are endless. My current main AHK script is over a thousand lines long. Check back soon for some hints on how to make typing more efficient, also using AHK.

If installing AHK is too much mission, here is a link to an .exe containing all the shortcuts mentioned in this post, which should run on any Windows PC without any installation. If you do get around to installing AHK, here is the script in an .ahk file, so you can download it and edit it to suit your needs.

---

* Well, maybe not everything. But why would we want to get technical in a blog with this one's title?