Thursday, 25 August 2016

WhatsApp's new Privacy Policy [fixed]

WhatsApp recently updated their privacy policy. To prevent users from getting skittish, they also wrote a blog post explaining how wonderful everything was. I found some mistakes in their blog post, though, so I thought I'd fix it up for them. The original post can be found here: https://blog.whatsapp.com/10000627/Looking-ahead-for-WhatsApp


About those 17 billion dollars we paid for a chat app? Um, we kind of need to make that back again

Today, we’re updating WhatsApp’s terms and privacy policy for the first time in four years, as part of our plans to test ways for people to communicate with businesses making WhatsApp profitable by allowing businesses to contact you in the months ahead. The updated documents also reflect that we’ve joined Facebook and that we've recently rolled out many new features (we’d like you to focus on the new features, instead of the changes to our privacy policy), like end-to-end encryption, WhatsApp Calling, and messaging tools like WhatsApp for web and desktop. You can read the full documents here.

People use our app every day to keep in touch with the friends and loved ones who matter to them, and this isn't changing (Please go ahead and think about just how useful WhatsApp is to you for a moment. You don’t really have a choice but to agree to our new terms). But as we announced earlier this year, we want to explore ways for you to communicate with businesses that matter to you too may be able to finally turn a profit for us, while still giving you an experience without third-party banner ads and spam (depending on your definition of Spam). Whether it's hearing from your bank about a potentially fraudulent transaction, or getting notified by an airline about a delayed flight, or maybe seeing a text message or two that’s actually an advertisement to help us become profitable, many of us get this information elsewhere, including in text messages and phone calls. We want to test these features in the next several months, but need to update our terms and privacy policy to do so (well, maybe “need” is a strong word, but the current ones are a bit inconvenient for us).

We're also updating these documents to make clear that we've rolled out end-to-end encryption (remember to focus on our new features please). When you and the people you message are using the latest version of WhatsApp, your messages are encrypted by default, which means you're the only people who can read them. Even as we coordinate more with Facebook in the months ahead, your encrypted messages stay private and no one else can read them. Not WhatsApp, not Facebook, nor anyone else (History and common sense say that we’ve probably opened up a back door for NSA, but that’s for like terrorism and stuff, so don’t worry about it). We won’t post or share your WhatsApp number with others, including on Facebook, and we still won't sell, share, or give your phone number to advertisers (but we might let them contact you through WhatsApp. Even though they can use your number in the only way that matters, please focus on the fact that they don’t actually possess those 10 digits that you value so much).
But (remember, anything we say before the word “but” doesn’t really count) by coordinating more with Facebook, we'll be able to do things like track basic metrics about how often people use our services and better fight spam on WhatsApp (Please focus on the ‘fight spam’ part, and skip over the ‘tracking’ part. Also please don’t read this piece on how much can be inferred by looking only at metadata from the EFF: https://www.eff.org/deeplinks/2013/06/why-metadata-matters). And by connecting your phone number with Facebook's systems, Facebook can offer better friend suggestions and show you more relevant ads (which will help us make money) if you have an account with them. For example, you might see an ad from a company you already work with, rather than one from someone you've never heard of (not in a creepy way though. Don’t worry. This is all about profit). You can learn more, including how to control the use of your data, here.
Our belief in the value of profiting from private communications is unshakeable, and we remain committed to giving you the fastest, simplest, and most reliable experience on WhatsApp. As always, we look forward to your feedback and thank you for using WhatsApp.


Friday, 19 August 2016

Do what other people are doing, but more meta

A common pattern among the computer science crowd is the desire to find a gap in the market. We've seen people like Mark Zuckerberg receive the same knowledge that we have, and turn that knowledge into money. Many people I know of have gone through approximately the same progression that I did in terms of becoming dissatisfied with academia for being too impractical (is anyone actually going to read that thesis?), followed by becoming dissatisfied with industry for being too uninspiring (yay, I fixed that unit test. Again). These people then start looking for gaps in the market -- waiting for that One Great Idea (tm) to come down from above and strike them between the eyes.

The first thing to realise is that ideas are worthless. As many people have noted, there is no market for ideas, and this is for good reason. They're not worth anything. You can patent an invention, but not a startup idea. Your idea might be good, but it's not going to make money on its own. Your product might be OK, but it's not going to make money unless it's polished and marketed. And as a single developer working on your weekends, you're unlikely to be able to build anything reliable that's also easy to use and which solves an actual problem. And then tell people about it.
Now that we have that out of the way, ideas are still important. And ideas are fun. I have notebooks full of ideas -- some of them I've shared with others for feedback. A select few are in the process of being transformed into code in private git repositories. I enjoy playing around with ideas, even if it's good to keep a healthy scepticism on how successful they'll become.
A good shortcut for finding more interesting ideas than those of other people is through the concept of 'meta'. A meta-thought is a thought about thoughts -- i.e. one of the things that we believe makes us better than the apes. Metadata is data that we keep about other data -- think of that "last modified" column in your file explorer. That's data. Your files are also data. So it's data which is describing data. Wow. Inception. Metaception. Mind == Blown.
But more seriously, as you listen to other people's ideas, try to see a layer behind their idea. Or if you are thinking of an idea, look for the idea behind that. Three quick examples will hopefully clarify this:
  • People are creating startups. Most of them fail. Some smart people avoid failure by creating startup incubators instead of startups. They buy some cheap warehouse space and offer internet, coffee, and 'mentorship' to other people who want to run a startup. Most of the startups themselves fail, but they still pay their fees to the incubator. And the few that are successful also give a percentage of their shares to the incubator. The incubator isn't hurt by the failures and makes a fortune out of the successes -- all through taking other people's ideas one layer of meta deeper.
  • People are playing on the stock market and buying crypto-currencies. Some of them make a lot of money and write about their successes to encourage others to try the same. Many others are losing all their money -- they tend to be a bit quieter and keep their heads down. No-one likes talking about them. The people in the game who are reliably making money are either the stock markets themselves (Wall Street is worth a bit), or the ones who are selling data, books, code, and tutorials to the people who want to gamble their money directly. Again, these people are making money on others' successes and not losing it on their failures.
  • In non-tech circles, people still make money by proofreading, though not very much. If you are part of the minority that has a good understanding of the grammar of your native language, it's easy enough to find clients who are a bit bewildered by exactly how commas and apostrophes work, and who have read the distinction between effect and affect several times and have given up trying to work out when to use which. However the hourly rate for proofreading tends to be pretty miserable. I once attended a three day proofreading course though, and paid the single instructor several thousand ZAR for the privilege. I was one of dozens of people to do so, and the instructor made more money in three days using his proofreading knowledge than many of the attendees would make in their lifetimes with the same knowledge.
Of course, once you start doing this, you might never stop. What about a startup incubator that trains other people to create startup incubators? Or someone who teaches people who to teach? Or someone who writes blog posts like this one? Be careful of the rabbit hole, Alice. People who go down do not always re-emerge.

Saturday, 26 March 2016

Data Science and Higher Education South Africa data

TL;DR 
* I'm exploring "data science" and related technologies, including R
* This is a fun "puzzle": http://priceonomics.com/the-priceonomics-data-puzzle-treefortbnb/ 
* There exist some nice open data sets relating to Higher Education in South Africa 

Data Science
"Data Science" is as much of a buzzword as "The Cloud", "Big Data", and "Artificial Intelligence", and many intelligent people will make unidentifiable sounds of contempt when they hear or read it. But like like the other buzzwords mentioned, "Data Science" started out as an interesting idea, which the media, recruiters, and marketing departments ran away with in order to impress various stakeholders and make lots of money.

With an increasing amount of open data sets being made available (see https://en.wikipedia.org/wiki/Open_data), being able to get information from raw data is an an ever-more useful skill to learn. I came across an fun and simple puzzle recently here http://priceonomics.com/the-priceonomics-data-puzzle-treefortbnb/ and decided to use it as as a starting point for learning more about technologies that are useful for data analysis. While Python is normally the first tool I'd turn towards to solve a problem like this, I recently saw some quite impressive work done with R. I was surprised by how easy it was to carry out common data manipulations and visualisations and I wanted to try it for myself. 

I won't go into detail in how I solved the puzzle linked above, as Priceonomics use it as part of their recruitment process. But I downloaded R, and messed around with it and the Treefort dataset for an evening and had a lot of fun. Below is a brief write-up on my first experiences with R, and the most interesting graphs from the South Africa education data set I was using. There's also a link to the Excel spreadsheet I used instead of R.

Why not Python?
One of the main reasons I enjoy Python is the intuitiveness of its syntax. If I don't know how to do something using a Python library, I can usually fire up a shell and with a combination of dir() and guesswork work out how to do what I want faster than looking it up on stackoverflow. However, with matplotlib, numpy, and pandas, I always find the opposite. Even when faced with a very basic problem, I often find myself trawling through documentation and examples to work out how to solve it. 

While manipulating and plotting data from a .csv file in R, I very quickly got into my Python habits of using trial-and-error and the R help() command. It was very satisfying to read, manipulate, and plot data in a few lines of code. My current impression of R (which will almost certainly change drastically as I use it more), is that it will fit somewhere between M$Excel and Python for me. If I just want to do some really basic calculations, I'll use Excel. If I want to build and maintain a 100+ line programme, that I'll need to use and change for the foreseeable future, I'll use Python. And if I need to mess around programatically with rows and columns, but I don't need to build anything maintainable, R looks like it could be a good compromise between the two. 

South African Education Data
There's no shortage of data sets to play with. Cape Town open data (https://web1.capetown.gov.za/web1/OpenDataPortal/) was the first place I looked, but it seems that that initiative was a bit of a let down. While there's some interesting data available, most of it is hugely inconsistent in format, and looks as if it was intended for human consumption instead of for programmatic analysis. I thought education data might be interesting, and I found that the datasets available here http://chet.org.za/data/sahe-open-data were comprehensive and fairly consistent. Unfortunately they're also presented in xlsx format instead of .csv and are not as ideal as the Treefort data set to load directly into R. 

I converted them to .csv files and loaded them into R, but I need to spend some more time with R's syntax and libraries to efficiently work with data in non-ideal formats. The pain point was the double headers in most of the data sets. For example, the dataset of enrollments by race looks like this: 



I wanted to graph the data by institution, as in the picture below. Getting the specific row that represented each institution in R was straightforward enough, but I couldn't easily find a way to transform the data to use the year as the x-axis, the categories as separate series, and the the numbers as the y-axis. I'm sure it'll seem trivial once I've worked out how to do it, but I decided to play around with the data in M$Excel first so I could have a clear goal in mind before diving deeply into R.  



UCT enrollment by race
Interestingly, of all the institutions listed, UCT is the only one to have any crossing lines
.
I've used Excel pretty extensively in the past, and even taught an introductory course on it, so it was much easier to clean and manipulate the data and create pretty graphs than working out how to do everything in R. I loaded the simplest datasets from the CHET collection (Race, Gender, and Success) into separate worksheets, created some hacky VLOOKUPs to separate the time and category data by institution, and added some graphs in a separate sheet. A screenshot of the result is below - the big cell at the top is a dropdown that contains all the institutions, and the graphs update dynamically when a new institution is selected.
All data for Rhodes University
I'm not sure what happened to the success rate in 2013 - none of the other institutions showed a similar decline. Hopefully it's a mistake. (My brief lecturing attempt at Rhodes was last year, so it can't be caused by that).
Soon, I'll attempt to replicate the graphs using R, and write a follow up post about how I do it. I'll also extend the data sets I looked at, and if there's anything interesting I'll write a post which focuses on the data instead of the technology used to analyse it. 

If you want to play around with the education data and see the graphs for the other institutions, you can download the Excel spreadsheet I built here: https://docs.google.com/uc?authuser=0&id=0ByEENivQuwUBSmNJUXdPbDI2cU0&export=download. The messy VLOOKUPs would probably be enough to have me expelled from any respectable computer science institution, but luckily I don't belong to any. Feel free to write me snarky comments below on how I could have done it in a cleaner way.


Saturday, 6 February 2016

Favours, Apologies, and Thanks

No Python today. Today we talk about my other favourite language – English.

Some people write English beautifully, causing their readers to pause and focus on what they are reading, momentarily forgetting about life's distractions – and other people don't English so well. But nearly everyone is bad at using English for asking favours, saying sorry, and giving thanks. Which is odd really, because nearly every user of English will use the language for expressing all three of these ideas many, many times over their lifetime. 

I'm tired of reading empty apologies, of people pretending to offer a favour when they're actually asking for one, and of people getting oh-so-very close to saying thank you, but bailing at the last moment. And it's really not that hard to do any of these things – we're just all so used to seeing them done badly that we take these as the norm, and emulate them in our own attempts. Here are some pointers on how to ask people to do things, how to tell them that you messed up, and thank them for their actions. I'm going to assume written language here, but most of the points are transferable to spoken equivalents as well (with the caveat that when speaking you can't backspace a paragraph and reword it to make sure that it says what you want it to say).

Saying sorry


How not to start an apology – Adobe spends the whole first paragraph on excuses before even getting to what has happened and what they are doing about it.

Bad apologies are probably the worst and most-common offender on our list of three, so I'll start off with it in case you don't read to the end. 

The most important part of apologizing seems simple, and you might be surprised that so many people mess it up. The first thing I look out for in apologies is to see if the apologizer actually apologised. In a majority of apologies I see, the apology itself is missing, which makes the whole thing as unsatisfying as eating lamb with mint sauce but forgetting the lamb. There are many ways to communicate an almost apology. Three of the main offenders are:

  • Using the subjunctive (would like to), and using too many words
  • Using the word 'but'
  • Focusing on the explanation instead of the apology
...would just like to take this opportunity to apologise...

We've all seen this phrase in countless variations, and most of us probably don't think much of it. But there are a few reasons why this is a bad, bad apology, that will leave the recipients feeling like they just ate a large spoon of mint sauce without any lamb. 

The first is the phrase "would like to". I would like to procrastinate less, know more, and not make language mistakes in blog posts about language. That doesn't mean that any of those things are going to happen. And it's the same for the apology – "I would like to say sorry" might sound like you've apologised, but actually you haven't. The reader feels like you're just about to, and then forgets that you haven't. The reader might even believe that you've apologised, but they'll feel unsatisfied even without really knowing why.

The word "just" thrown in there adds to the problem. You're suggesting that the apology isn't really a big deal for you. You'll find this nasty word "just" thrown in to the mix in all sorts of similar phrases with similar results – if you find it in your own writing, just delete it. 

"Take this opportunity" is playing for time – you're still uncomfortable, even though "would like to" gets you out of a real apology, the idea of the apology still makes the writer uncomfortable, and this useless filler simply puts off the word "apologise" for a little bit longer.

...sorry, but there wasn't much more I could do... 
...sorry, but I was feeling pretty down that day...
...sorry, but you did kind of deserve it...

The famous but. "I'm not racist, but". Nothing that someone says before the word but really counts. You can look out for this innocent looking word "but" in many areas of communication, and perhaps especially in apologies. The three examples above are hardly subtle – in the wild you'll find many apologies that try a little harder to veil the following excuse, but they'll still follow the same pattern. In the worst cases, the apology is nothing more than a tool to drive home the fact that the speaker really really is not sorry. In the subtler cases, even the speaker may even believe that an apology has been given, but ... actually it hasn't.

...I can explain. What happened was...
Most explanations for why the thing-to-be-sorry-for happened are merely excuses. Sometimes, however, an explanation is desirable (excuses never are). If the person you're apologising too needs an explanation in order to understand what happened and to prevent any negative consequences from spreading, adding an explanation to an apology can be acceptable. 

But read over your apology-plus-explanation and ask what the explanation is actually doing. Is it drawing focus away from the apology? Is it detracting from the legitimacy of the apology in the same way an excuse would?  What is the focus of the explanation? To explain the situation, or to explain why you mishandled it? 

The three above points can be, and after are, combined into a single nauseating non-apology that is unhelpful to all parties. But we've looked at enough ways of how you shouldn't apologise. Let's take a look at how you should.

Apologising and responsibility
The first step to apologising is working out what you were responsible for, and what you went wrong because of your actions. A real apology will always include the idea of "I should have done x, and instead I did y. I'm sorry." and sometimes will include "because of z" and/or "I realise why y and x are different and in future I will always do x" and/or "I have now done w to attempt to make amends for doing y and not doing x". If you already have a draft apology of this form, consider deleting the "because of z" – it'll almost always make your apology more genuine. You might think that "because I am a shitty human being" or "because I am an idiot" will give the reader some kind of compensation for your wrongs, but actually it won't. Using yourself as an excuse is still an excused-apology, and it will likely come across as inauthentic. 

Apologising should be a really unpleasant experience for the apologiser. No-one wants to feel guilt, shame, and inadequacy. If you feel great while writing an apology, it's probably not genuine. Next time you write an apology, start with the words "I am sorry that I...". Fill in the blank with the shortest possible amount of text that makes it clear what you are apologising for, and hit the full stop key. You'll probably almost subconsciously hit backspace, replace it with a comma, and add some excuse. Don't. Don't add the excuse.

And again, the most important thing is to make sure that your apology communication does at some point include a literal apology. Here's Kickstarter's apology – even though this paragraph is towards the end of the post, what comes before is brief and to the point. They start off with a short, bolded, literal apology, and say that they're not feeling great about it either (a great way to gain genuine sympathy from your reader if you don't whine too much). They end off with an brief explanation about what they have already done to mitigate their mistake, and what they will do in the immediate and longer-term future. They don't even come close to suggesting that the subsequent actions make the mistake OK, or claim any factors that direct the fault away from them. I'll give them 10/10 for this apology.



Saying thank you
This section will be short, because nearly everything from the apology section above is applicable to thanks as well – and because thank yous should be short, and I believe in teaching by example. "I would just like to take this opportunity to say how grateful I am to [so and so] for being here tonight". Again the "just". Again the "would like to". Again the too many words. 

"Thank you [so and so] for being here tonight. I appreciate it" is much more genuine, and is not going to bore anyone. Adding "really really incredibly oh-so-very grateful" is not going to impress anyone, and is not going to make your gratitude seem greater. Drop the adjectives, drop the adverbs, drop the repetition.*

Saying thank you is difficult because a majority of expressions of gratitude aren't genuine. Acknowledgements, speeches, fund-raisers, and other charity announcements have used nearly every possible combination of words that they can to say empty thank yous. To make yours genuine:
  • Keep it short. Your sentences should be short, and there shouldn't be many of them. When saying thank-you, less really is more.
  • Make it personal. Would you be able to copy your exact message and pass it on to someone else you want to thank? If yes, it'll probably not sound genuine. Talk about specifics; make a reference to something that will only be understood by the person you're thanking. Get creative.
  • Focus on the person you're thanking. Again this might sound obvious, but some have a tendency to start out with good intentions, and then slip into talking about themselves rather than the person that they're thanking. If you're doing this, you've probably gone against both the first two points already. Stop now, sign off. Say goodbye. 

Asking a Favour
Most decent people find it slightly uncomfortable to ask others for benefits while offering nothing in return. But sometimes its necessary. The most common mistake is for the asker to try to hide the discomfort by masking the favour as actually being a benefit to the askee, or to try pass it off as fairly insignificant and Not A Big Deal (tm) for either party. 

When asking someone for a favour, the first step is to think about it for a moment. Make sure that you know you're asking for a favour. Is the other party gaining anything from this transaction? If your initial response to this is "no", then don't think about it too much - don't look for a contrived benefit for the other party. Are you offering something in return? Is the thing you're offering a token, or is it actually something valuable enough that the other party would consider the transaction a fair exchange rather than a favour? This doesn't mean that you need to offer something of equal value, or that asking favours is bad. This step is only to make sure that you – the asker – know what you're asking for.

Here's an example – while there is nothing really bad about this email, there's some room for improvement. The asker is clearly asking for a favour – 30 minutes of the askee's time towards academic research. And yet some phrases creep in that show the asker is not really comfortable with the idea of requesting favours. "This is your chance" would normally indicate something highly desirable, and "you will receive nice sweets" indicates that the favour is actually a trade. 



No reader of this is going to be weak-minded enough to actually be influenced by these attempts to frame the favour as not-really-a-favour. And the author isn't actually trying to con readers into believing that it's anything other than a favour. The highlighted phrases merely indicate the author's discomfort with favour asking – and will probably have a negative impact on responses.

Here is an improvement to the favour request:

For my Master's thesis, I am looking for native English speakers or fluent non-native speakers of English. Participants would be evaluating text-to-speech (TTS) synthesis systems (participation takes a maximum of 30 minutes). Unfortunately there is no funding available to compensate participants for their time, but you will receive nice sweets as a token of my appreciation. If you are willing to help me out, please get in touch at: _____ I would be really grateful!

Making it obvious that you are asking for a favour is likely to attract more positive responses. Most people enjoy granting favours – after doing something for someone else, knowing that you've helped them out and got nothing in return can be a positive experience and leave you with a spring in your step for the next several hours. In the rewritten request we:
  • Make it obvious that we're asking for a favour,
  • Explicitly state that we will be grateful,
  • Show that we are aware that we are asking for something of value and would like to compensate accordingly, but are unable to due to circumstances.
These three points will not only give us more participants, but the participants are also less likely to be grumpy if the experiment ends up taking longer, or if the sweets turn out not to be to their taste. It's no longer a "deal", in which longer hours or reduced payment mean that the participant loses out. And the gratitude will make the participants happy.

Now go out into the complicated social world and be genuine about what you ask for, what you are grateful for, and what you are sorry for.

Disclaimer: This post is intended for educational purposes, and the author takes no responsibility for people who use the information contained herein to make inauthentic communication seem genuine.

* unless it's as elegant as the repeated 'drop' in this sentence

Thursday, 17 December 2015

Creating floating point arrays in Python

Note: all script output is included at the bottom of each code block and indicated with '>>>' (or sometimes the output is summarised in a code comment). Yes, it's confusing, but you're smart ;)
Let's imagine you want a simple array of consecutive floating point numbers in Python: [0.1, 0.2, 0.3 ... 0.9]. You start by trying to use Python's built-in range() function:
x = range(0.1,1,0.1) # Don't do this

Expecting to get an array of numbers from 0.1 (first argument, inclusive) to 1 (second argument, exclusive) with a step-size of 0.1 (third argument).
But the Python range function can only deal with integer step sizes, and complains:
>>> TypeError: range() integer step argument expected, got float.

OK, so you import numpy and use arange(), right?
import numpy as np
x = np.arange(0.1, 1, 0.1)
# x is [0.1, 0.2, ..., 0.9]

Exactly what you wanted. But what if you don't have numpy? What if you care about code footprint, portability, and all those things? What if you want someone else to be able to generate your super interesting array, and they don't have numpy? Do you ask them to install numpy, knowing their lives will be better in the long run? After internal debate, you delete the numpy dependency, and try a list comprehension instead:
x = [x * 0.1 for x in range(1,10)]

Hah, now you must surely have the best of all worlds. One line, no dependencies, and a list comprehension. This is great. This is amazing. You print it to double-check your handiwork:
print (x)
>>> [0.1, 0.2, 0.30000000000000004, 0.4, 0.5, 0.6000000000000001, 0.7000000000000001, 0.8, 0.9]

Wat?
Oh yes, computers suck at floating point numbers. Now what? Back to numpy? Round the numbers? Write a library to handle all of this? Re-write numpy? You try once more for a simple solution:
x = [x/10.0 for x in range(1,10)]
print(x)
>>> [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]

Interesting. Division works where multiplication doesn't. You try out a few variations of each to make sure the distinction is consistent, and find out it is. You're kind of happy with the division solution, but floating point division is slower than floating pointmultiplication, right? What if someone wants to do this a million times? You decide to see how much time you're losing for floating point precision:
import time

N = 1000000
t1 = time.time()

for j in range(N):
    foo = [x * 0.1 for x in range(1, 10)]
print("multiplication: {}".format(time.time() - t1))

t1 = time.time()
for j in range(N):
    foo = [x / 10.0 for x in range(1, 10)]
print("division: {}".format(time.time() - t1))

>>> multiplication: 4.11618614197
>>> division: 4.24211502075

That .1 second for every run of a million hurts a bit. Division is slow. Why not just multiply the numbers as in the first attempt, and then round them to 2 places? Last try:
# ...
t1 = time.time()
for j in range(N):
    foo = [round(x * 0.1, 2) for x in range(1, 10)]
print("round: {}".format(time.time() - t1))

No more slow division! And how long can it take to shave off some decimal places?
Quite a while, it turns out. Is that nearly 30 seconds? Yes, it is.
>>> multiplication: 4.11618614197
>>> division: 4.24211502075
>>> round: 29.5979890823

OK, that's slow. Slower than sub-string manipulation as it turns out. You convert the broken float to a string, take the first three characters off it, and then turn it back to a float, just to prove a point:
# ...
t1 = time.time()
for j in range(N):
    foo = [float(str(x * 0.1)[:3]) for x in range(1, 10)]
print("string: {}".format(time.time() - t1))

>>> multiplication: 4.10650587082
>>> division: 4.21635198593
>>> round: 30.1995661259
>>> string: 23.5980100632

Now that you've put that 0.1 second paid for division into context, you feel OK using it. For comparison (even though you said 'last attempt' a while back), you add timing for numpy as well:
# ...
t1 = time.time()
for j in range(N):
    foo = np.arange(0.1, 1, 0.1)
print("numpy: {}".format(time.time() - t1))

>>> multiplication: 4.17134094238
>>> division: 4.23171901703
>>> round: 36.1641287804
>>> string: 23.5332589149
>>> numpy: 3.10889482498

A whole second faster! Maybe you should include that massive dependency after all. But you remind yourself what you've read a million times in start-up blogs. The most important thing is code readability. The compiler will do optimization better than you can ever hope to, right? Right?? Or - maybe not.


Saturday, 12 December 2015

Spritz and applications that use it



When Spritz announced their new speed-reading technology with a flashy website, an impressive demo, and some complimentary media articles, people noticed. They claimed that they would change the future of reading, and people believed them. I believed them. The premise is simple - words flash one-by-one in front of you on a fixed point, saving you the time you usually spend moving your eyes backwards and forwards while reading text in lines.


It's difficult to re-imagine ideas as popular as reading. Books have chapters, pages, paragraphs, and lines. They have many of these things not because they are inherently important to reading, but because they were necessary to print words out on paper. Some web pages still try to incorporate the idea of pagination - you get half way through a news article and then have to press "Go to page 2". Most people agree that pagination is generally A Bad Idea and no longer use it.

Two years since the Spritz announcement, and the technology built on Spritz is disappointing. There are some half-baked attempts to create speed-reader applications for most popular platforms, but nothing revolutionary. Most of these applications are just a wrapper of the Spritz demo that allow users to upload the content they want to read or find it online.

The Spritz applications that I find useful, but far from complete, are:
  • The Spritz 'bookmarklet' (web) allows you to Spritz most text that you come across online, simply by selecting it and pressing the bookmark in your (desktop) browser
    • Pros: Free, easy to install and use, fairly versatile
    • Cons: Doesn't work on mobile devices, Isn't designed to read books or other files.
  • SpeedRead (web) - a website that allows you to upload your own files (including PDFs), and read them with a very attractive modification of the standard Spritz interface.
    • Pros: Looks good, works well, allows user files
    • Cons: Not free, although the 'try it out' functionality has no time or number of use restrictions.
  • ReadMe! (Android) - an Android e-reader that includes Spritz technology
    • Pros: Free, shows whole page behind Spritz window, allows for 'normal' reading too;
    • Cons: Android-only, difficult to add files (you have to transfer from PC or download directly, but there are no options to link to Dropbox or equivalent).
There are a number of other applications that I have tried over the last few weeks, and they have all been very disappointing. Many have been very unstable or are incomplete/ no-longer developed. Perhaps the iOS ones are better, but not owning any Apple devices (and generally finding myself unwilling to pay for digital 'things'), I haven't been able to test these.

I find Spritz very useful for reading longish articles (looking at you Medium), and to skim through books that I'm not sure I want to read. But when reading through Spritz, one tends to mentally 'hear' everything in a monotone, and the applications of the technology so far still make it very difficult to navigate through a book in anything but a beginning-to-end pattern. When people read, they often tend to go back to re-read a complicated sentence, or to compensate for their mind wandering off for a bit. This is still something that is very difficult to do in all the Spritz applications I have seen so far.