I just found this gallery of images of the super-kamiokande neutrino detector in Japan.

The pictures are pretty breathtaking, especially when you realise that each of those little domes is a hand blown piece of glass...

Just had to leave Starbucks due to a fire alarm :(. Still they have promised free re-fills for those of us forced to up and leave our seats!

Also I got my brand new gransfors bruks Small Forest axe, my lapplander folding saw and my fallkniven (spelling?) dry whetstone this morning (my Japanese sandstone is good but messy and not so portable...)

And just to top it all off I made some delicious double chocolate muffins this morning, although most of them have gone to Holly's meeting so I only got to try one (but I think I will make some more this evening!)

When using numpy efficiently you often end up using broadcasting operations on large matrices to e.g. calculate a difference over all possible combinations of feature vectors and then take the minimum.

This kind of thing is very CPU efficient compared to looping and calculating in Python as all the looping etc. is handled by numpy's internal libraries but it can be memory innefficient (I keep on running out of RAM on our server which has 32Gb of the stuff...) and also as a monolithic operation doesn't make use of multiple threads (it also has 16 cores so I want to use them all).

Anyway I discovered the following usage pattern for easily splitting up a big matrix operation in to smaller chunks so it can fit in to memory and parallelising the operation (which is one of the big benefits of having all the heavy lifting done in numpy's C code rather than in GIL encumbered python)

First off you decide how you are going to split things up. So something like this:

cs = 10

Means I am going to split things up in to 10 column chunks for processing.

Then define a functor or lambda expression that operates on the data you want to process taking a start index as the parameter:

f = lambda x : log(A[x:x+cs] + B[newaxis])**2

Note that this is baking A, B and cs in to the expression - you could wrap things up in a functor and then specify A, B and cs in your final expression, but I can't be bothered.

finally you run parallel_map on your function and stitch together the results:

nt = 10
res = vstack(parallel_map(f,range(0,len(A),cs), nt))

This would split it over 10 threads. If you just want the parallelism speed up you set cs = len(A)/nt. If you want memory and speed optimisation you set nt and cs independently. If (for memory reasons) you want to further subdivide your problem you can use map within your lambda function to split up the indexing of B

Finally, vstack will do something wierd to your results if you are iterating over a single item each time so you may need to do something like this:

f = lambda x : [log(A[x] + B[newaxis])**2]

so you get a bunch of single entry lists which vstack will stack correctly.

So there you go - happy and efficient processing!

I've organised for me and my dad to go on a Hog's back Tour on Saturday as a joint Father's day / Birth day present

You get generous sampling and a souvenir glass to keep!

And I will get to use my new BBQ show off my new guitar and drum machine and my new Bass Strings (as used by Steve Harris - although they are really heavy and have put the relief and action on my Bass all out of whack) when the family come over for a birthday BBQ.

It will be great!

I was discussing ways to get python to automatically reload modules (useful when using matplotlib to try out algorithms and stuff) and came up with the following:

class Autoreloader:
    def __init__(self,moduleName,path = None):
        self._moduleName = moduleName
        self._path = path
        
    def __getattr__(self,attr):
        import inspect
        import imp
        import sys
        sys.path.append(self._path)
        _module = imp.load_module(self._moduleName,*imp.find_module(self._moduleName))
        sys.path.remove(self._path)
        if attr == "__members__":
            return [a[0] for a in inspect.getmembers(_module,inspect.isfunction)]
        else:
            return _module.__dict__[attr]

Then instead of doing

    import my_module

you do

    my_module = Autoreloader("my_module")

After that you can add a function to my_module.py, go back to ipython, and type my_module.<TAB> and your function will magically appear! To change the function just hit ctrl-S in your text editor and the next time you run the function it will pick up the changes.

This does incur a performance hit and this is version 0.1 I'm sure it can be improved...

If you have ever had the problem where you have a bunch of stuff in subversion and you want to check it all out (so you can update and submit with one command) but for space or other reasons you want to exclude a certain directory, then I have found the solution thanks to this post

It is pretty simple, just create an empty directory somewhere on the repository and use "svn switch" to switch the directory you do not want to update to an empty directory. Then you can update and submit to your heart's content without having to have all the extra baggage you don't need in your source tree.

Wonderful.

jk

What a ridiculous story. This hotel works out a way to import large chunks of money in to the country for little or no effort and all it gets is stick.

Doesn't the hotel employ local people? Doesn't it use local services and utilities which it pays for? So where do you think a big chunk of that $14k ends up? Some people need to think a bit more and be a bit less outraged.

I am currently running three websites www.joekilner.net www.basquechildren.org and www.cottage-in-brittany.co.uk off of a vps link4 that I have hosted at vpslink. The hosting there has been trouble free and basically fantastic, but it's hosted in the states and stuff can be a little sluggish connecting.

So I am looking at getting a UK based provider to replace them which is when I came across this page full of offers on VPS hosting. There are some pretty tasty looking offers there including codes for double ram on plans as cheap as £9 per month (although at the moment I am paying £6 per month which is pretty good!).

Either way it's an excuse to spend a load of time shopping around online...

Mmmmmmm.... sed... tasty....

jk

An interesting little piece here.

It's amazing how often coincidence and correlation are confused which is bad enough in itself, but then correlation is further confused with causation we get wrong headed thinking like the "mobile phones killing bees" stories we were reading a while back.

An interesting point to ponder in light of this is the birthday "paradox". If you have 30 people in a room, what is the probability that two of them will have the same birthday?

Is it 10%?

20%?

No, it's 70%. Why - because although there are only 30 people, there are 30 possible birthdays they could share - so it is like saying "Whose birthday is on 19th August" to a room with 29 people in it, and trying with 30 different dates - it ends up being quite likely that at least one person will say yes.

So what is the lesson in all of this? Our instincts are way off when it comes to working out what is likely when it comes to coincidences in the complex world we live in. Statements like "That can't just be a coincidence!" are often just plain wrong. Either sit down and do the maths or take the advice of someone who has.

jk

I am looking for a good blog on computer graphics or computer vision. Or a good news source on either of these things. Just something that will keep me up to date with the latest news in these fields.

Do such things exist? If you happen to know of any and stumble across my page, please let me know in a comment!

jk