May 30, 2012
Reminder: secret_token.rb is named so for a reason.

Earlier today I realised that our secret_token.rb of openSNP is publicly available on GitHub - which is a bad thing, because our server signs all cookies with that thing!

Any old attacker can take the secret token and re-generate valid cookies for your applications. The user-id is digested with the secret token and then stored in the cookie, and because the user doesn’t know the secret token, he/she cannot change his/her user-id. If the attacker knows the secret token, it’s relatively easy to regenerate a valid cookie to log in as the admin (or check out what other users have inside their account…)

It seems this is not public knowledge - if you google for “secret_token.rb” you can find quite a few projects on GitHub and elsewhere which store their secret key in the open, I just hope that’s not the actual secret on the server.

How to fix it: Create a new token using “rake secret” (I had to install the vegas-gem for that - alternatively you can make up your own long random string), put it in it’s own file, have that file in the .gitignore and link to it inside your secret_token.rb: like this for example, then restart your server (this will invalidate your old sessions, users will have to log in again). I don’t think this works when you deploy to Heroku, here’s a solution for that.

Takes 5 minutes and makes your application much more secure!

3:39pm  |   URL: http://tmblr.co/ZfqvFxMPTnHj
(View comments  
Filed under: rails github 
April 22, 2012
"Reality is that which, when you stop believing in it, doesn’t go away."

— Philip K. Dick, How to Build a Universe That Doesn’t Fall Apart Two Days Later

April 19, 2012
"Not using downloaded repomd.xml because it is older than what we have:
Current  : Sat Mar 17 01:08:34 2012
Downloaded: Thu Jan 1 10:00:00 1970"

— Thanks to yum for this amusing bug.

April 17, 2012
On the speed of dictionaries in Python

This page opened my eyes to one of my gravest programming mistakes I did so far: handling dictionaries in an extremely slow way!


This is how I used to do it:

And this is how it’s actually done:

The difference in coding is minimal (don’t forget the specialised except and not just a general one!) but the difference in speed is HUGE. I had a script like this running over a 100mb tab-delimited file and it took roughly 5 minutes to finish, with this change it went down to 3 seconds.

Edit: Here’s an implementation using defaultdict on suggestion of Bastian Greshake:

Marginal increase in speed for files < 200mb (roughly 0.2s compared to the try/except-solution?) but probably very useful in files in the range of several gigabytes.

April 10, 2012
How fast are modern languages when it comes to parsing files?

The bread-and-butter of most bioinformatics-programmers (at least the ones I know) is writing parsers for different kinds of output. Packages like BioPython/BioPerl/Bio* provide parsers for common output, like for example BLAST provides. But we still have to write parsers for tab-delimited output, and I recently had a short discussion which led me thinking - which language is actually the fastest for that?

My candidates were: C++ (gcc 4.6.3, with Boost-library), Perl 5.14.2, Python 2.7 and 3.2, D, Ruby 1.9.2, Rubinius 2.0.0dev, JRuby 1.6.5 and because it’s new and I wanted to have a look: Julia, 0.0.0+1333823704.r2d0b/2d0bb43e7e.

Pseudo-codes for all languages except D goes roughly like this:

for each line in file_handle:

    split the line by tab

Simple, no?

You’ll find the code for each language at the end of this post, feel free to criticize! Especially my D-implementation is weird as it apparently doesn’t support iterating over the lines directly.

Now to have a look at how fast each language is! I used a 90mb tab-delimited file with 8 columns for this, I played around with bigger files but the time needed in each language just grew linear anyway.

Here’s how fast each language was: I ran each implementation on the same file on my machine 5 times and took the fastest time because I’m nice. I measured the time using the language’s own time-library or, if not available, Bash’s time-command.

Here’s the list ordered by time, all in seconds:

  1. C++: 0.49
  2. D: 0.76
  3. Python2.7: 0.82
  4. Python3.2: 1.30
  5. Perl: 1.31
  6. jRuby: 1.32
  7. Ruby: 2.19
  8. Julia: 4.34
  9. Rubinius: 8.15

I feel it’s unfair to include Julia as it’s a very new language, but I just had to check out these claims of “being very fast”. Since I started comparing the languages I have been in contact with the creators and speed has already greatly increased to what it’s now.

Also surprising to me is the speed-difference Python/Perl: A lot of people told me I should suffer more Perl because it’s so much faster than Python, which it (at least in this case) isn’t! Another surprise is jRuby, being up there with Python3.2. I would take the listing of D with a grain of salt as the code I’ve written for that differs quite a lot from the other implementations, anyone got a better one?

There’s a couple of ways I could improve this, my methods of time-measurements are highly unscientific and system-dependent, I could add more languages (Java, JavaScript, Bash, C#, Rust etc.), and I could improve the different implementations with respect to their language’s quirks.

tl;dr: Python is faster than Perl and what happened in Python3.2? Also, Ruby is becoming quite the fast language thanks to jRuby.

tl;ak (too long; already knew): C++ is fast.

Here’s the gist from GitHub, if you can’t see it, please activate JavaScript or just go here.

April 3, 2012
"If, however, the aim of the War on Drugs was to create a dynamic and vigorous black market, and provide an ever-expanding variety of drugs of increasing purity at lower and lower prices while enriching organised crime, bikie gangs and corrupt police, then drug prohibition has been an overwhelming success."

http://theconversation.edu.au/the-war-on-drugs-has-failed-now-what-6222

March 27, 2012
"'I had one son; why did you take him from me?' I shouted and shouted, but who could expect him to hear! Only once did I see the heavens open. It was at midnight, on the top of the prophet Elijah's mountain. I heard a thunderous voice: 'Shout yourself hoarse, for all I care.' Then the heavens closed again; and that was the last I ever called to God."

— The Last Temptation of Christ, p. 80, Nikos Kazantzakis, from the i-know-this-is-supposed-to-be-serious-but-i-had-to-laugh-so-much-dept.

March 25, 2012
"I looked over at the kangaroo at the window table, and the kangaroo looked quickly back down at his drink again."

— Gun, with occasional music - Jonatham Lethem, from the wtf-am-i-reading-department

August 4, 2011
Disconnecting by local choice - reason = 3

If you use Ubuntu and the standard network-manager, but keep on getting disconnected in busy WLANs (my university’s network in my case) have a look at your /var/log/syslog (use tail -f or something like that). If there is something in there like the title describes, that’s apparently a weird bug by network-manager. People on the Internet suggest the following, and it worked for me too:


De-install network-manager (<code>sudo apt-get remove network-manager</code>) and install wicd, an alternative network manager, which does not have this feature. After I set up the correct initialization (careful - wicd, unlike network-manager, lists all available channels for the WLAN separately instead of grouping all channels in one entry - so if you move around you have to re-enter the credentials) everything worked fine for me.

If you, by some weird coincidence, go to Bond University: Use “PEAP with GTC” for Encryption, enter Student ID and your normal password to get online.

August 4, 2011
How to get the Citrix client to work under Debian/Ubuntu

I recently wrote a small post for Bond University’s Information Society’s Facebook group, in which I wrote how to get the university’s idea of Off-Campus connect to work under Debian/Ubuntu. The university offers some oldish version of the Citrix MetaFrame Presentation Server & Client, so if you’re here from another university this manual might not work for you. I tried this under Ubuntu 11.04, but I think it should work fine under older versions too.

First, download and install the x86 version from the link Off-Campus Connect gives you after you log in the first time (MetaFrame Presentation Server Client for Linux x86).

Then, you probably have some libraries missing, but Citrix is too cool to tell you so. Run this:
CLIENT_EXEC=”/usr/lib/ICAClien​t/wfcmgr”
ldd ${CLIENT_EXEC}

In my case, I got a couple of output showing that libraries work and one error-line:
libXm.so.3 => not found
So install libmotif4:
sudo apt-get install libmotif4

Then symbolically link that bitch up:
sudo ln -s /usr/lib/libXm.so.4.0.3 /usr/lib/libXm.so.3
(We can see here that our outdated Citrix-client wants som bum-old version)

Neeeaaarly done, if we know try and open the connection in Firefox, it will (probably) try to open the launch.ica-script in Gedit, please redirect it instead to open it with /usr/lib/ICAClient/wfica.sh .
Now it will complain about “You have chosen not to trust the “Thawte Premium Server CA” the issuer of the server security certificate.” which is complete bullshit as we have not chosen anything anywhere. Go and download this:
http://www2.slac.stanford.edu/​computing/windows/services/cit​rix/downloads/ThawteRoot.crt

Now place that into /usr/lib/ICAClient/keystore/ca​certs .
Now re-try opening launch.ica with wfica.sh again - worked in my case from here on!

I think you should be able to place the launch.ica-file somewhere and modify it a little bit so that you don’t have to log in using the page - if you just use the launch.ica-file without logging into Off Campus connect you get an error because the Server “forgot” that you existed and doesn’t let you in. No idea how yet.

Source for all this: Random half-sleepy half-drunk googling

Liked posts on Tumblr: More liked posts »