least significant

If you've taken a look at the ckan source code, you'll have come across 'context' as the first parameter in many of the functions. It basically contains all threadlocal information required for a function to execute. It's taken me far too long to understand why they exist and I currently think contexts in their current state are pointless. I'm assuming contexts only contain model, session and user. I get why context exists, it basically comes down to whether you prefer passing parameters for the request/session/user around into functions, so that the parameters define exactly what the function needs to execute. If we do this though, these are 'different' to other parameters which is why they've been seperated into context/data_dict. Your functions do not have to refer to some magical global object to execute. Or whether you are happy using threadlocals, so that if your logic function, calls another function that calls another, that calls ....

At some point at university I borrowed, read a tiny amount of and got fined for the late return of Large Scale C++ software design. I think somewhere at the beginning there was a section on using headers to avoid circular imports. ckan is a large enough project that we can learn a thing or two here. So we have a ckan python style guide , where we used to recommend avoiding from blah import banana style imports and instead recommended import blah.banana as banana Eventually we decided to allow from imports , because the above is ugly, but the docs were updated to include some additional rules about what you can and cannot import to avoid circular imports. ckan.model often gets imported all over the shop and I believe it was the cause of many circular import problems. Stuff like get_action ended up creating stuff to get around it. A simple example of ckan.model ending up everywhere would be importing from ckan.logic import schema schema would contain all the action fu...

We use sqlalchemy in ckan and often we run into problems with DetachedInstanceErrors. For this reason, we normally jam all the data from a model object into a dictionary to avoid getting the DetachedInstanceError. This isn't sqlachemy's fault, it's more than likely that we are just incapable of using it properly. For a while, we kept on running into issues where paster db clean would hang and I spent a while trying to figure out what the hell was going on and what was wrong with the session. As it turned out, there was nothing wrong session really, it was just that we were handling it badly for command line functions. There were basically 2 problems. Not closing transactions that are implicity opened by sqlalchemy Closing and removing a session and then attempting to use the sqlalchemy objects from that session causing DetachedInstanceErrors To demonstrate this, we're going to use do some fiddling with ckan from the ipython command line shell. We'll st...

(e:feb 2015. I originally wrote this years ago when I was looking for a job and 'having a blog' was a question you were asked at interviews. This is nothing revolutionary, hell I'd rewrite the snippet now so it would yield the rows instead of constructing the entire table and returning it. But I felt good coming up with it back in the day.) Handling time series data using python’s builtin datastructures is a bitch. I guess the best way would be to just use a library like pandas , but a colleague came across the way I’d handled it in this code snippet using itertools and functools and asked me to share it with the rest of the team. I’ll just concentrate on the parts which require more explanation. The most complex line to understand is line 24 List comprehensions and python ‘switch’ row = dict([ (option, function_dict[option](series['list'])) for option in table_options ]) The dict function can build dictionaries in a variety of ways, a list pairs in m...