Sunday, 22 January 2012

The River

Yesterday I saw a youtube video pollution in the Mohaka which was a real eye opener.  If you have not seen it, I strongly suggest you click on the link above.
Swimming in a clean mountain river is one of the greatest experiences one can have.

Sadly my local river, the Manawatu is far from pleasant with its greasy algae covered rocks and streams of filamentous gunge. Given the reputation of the Manawatu as the most polluted river in the western world I thought I would in the first instance try and understand what was happening in the Mohaka, and then focus closer to home.

Above the Taharua confluence the Mohaka is clean (that is no visible algal blooms/gunge) with lots of large fish, and clear water.  To my knowledge there is no dispute as to the cause of the poor state of the Taharua river. It is the change to intensive dairy farming.

In a nutshell dairying is not done the same way as it used to be done. Naru island was exported to New Zealand as phosphate rock, which was converted to super-phosphate fertiliser.  This was applied to pasture to encourage clover growth, and the clover supplied the nitrogen to the soil.  The forage was a mix of clover and ryegrass.  However it has long been known that if fast results are required give the pasture nitrogen.

From a classical economics viewpoint giving nitrogen makes sense as the value of the milk produced far outweighs the cost of the nitrogen fertilizer.  Classical economics (through the law of diminishing returns) also tells us how much fertilizer to add, that is until marginal revenue = marginal cost.  The law of diminishing returns says is the first bit of nitrogen we add gets a big response.  As more and more nitrogen is added we get less grass growth for each increment.  If nitrogen costs 50c/kg and milk is worth 1 $/litre we add more nitrogen until adding 2kg of nitrogen gets us and extra litre of milk.

The result of this situation is that New Zealand lowland waterways are in an appalling state.  This is not my opinion but scientific fact.  The question is what can, and what should be done about it? Perhaps I should send an invoice demanding payment for the direct injury to me as the loss of enjoyment of life that I have suffered by their actions.  Perhaps regulation is the answer, but watching on One Plan fiasco continue I doubt it.  Perhaps the easiest is to use economics to undo the damage economics has caused, a significant tax on nitrogen fertiliser, $1000 to $2000 per tonne should be enough.

Substituting this back into the marginal cost = marginal revenue and relying on diminishing rate of return, this should be enough to make farmers a little less heavy handed with their fertiliser.  Now if the price of nitrogen fertiliser goes up to $1.50/kg, then we add nitrogen to the point where 670g of nitrogen gets that extra litre of milk.  This will occur at a lighter fertiliser usage.

The value of environmental destruction is not measured in our current system and the farmer is not asked to pay for it. Instead the farmer is rewarded, not only by healthy profits, but also by an increase in land values.  Let us bring the true costs of nitrogen fertilizer usage back to the farmer, and we might get some better outcomes.

If you did not click on the link at the top of the article, here it is again.  Please  view the clip by clicking on this link.

Wednesday, 4 January 2012

Relativistic Databases

Finally I have a good title for something that I have been thinking about for a long time.

Once there was a brilliant man named Isaac Newton who wrote the physical laws of kinematics, that is how things move in space and time.  Across the universe the clocks that measured how things moved were synchonised, the all showed the same time and they all moved at the same rate.

In the early 20th century that view of the universe was revised with the concept that only the clocks that could see each other could be synchronised, and to stay synchonised they had to tick at different rates and a new relativistic view of the world emerged.

In the same way software engineers have designed software around synchonised clocks in the form of ACID transactions, with locking mechanisms to ensure that data was updated in the "correct" order.  There is only one order, synchonised across the computing universe.  As in the early 20th century physics as the previous theory reached the boundaries of the marvellous achievements of Newtonian physics, and a new theory or paradigm was needed, so to, today the current generation of database designers are having to move on.

As the old notions of global clocks in physics was lost, so it must be in computing as in reality information moves at the speeds of network traffic in much the same way as light moves across the universe.  Things that happen today on the far side of the universe will not be visible here until some future point in history, in the same way data loaded into a computer system on the other side of the planet will take time to get here.

Why is this important? The ACID transaction model requires tight synchronisation that limits scalability and redundancy.  In short it forces computerised databases onto central servers and thereby introduces critical failure points.  The relativistic database is in contrast distributed and fault tolerant.  Why should a shop in Wellington be unable to trade because a server in Christchurch has been hit by an earthquake?

So what will this new computing world look like?  The old notion that programs must process data in order must go.  Data needs to be what I call "temporally commutative".  Let us consider two events happening at the same time on opposite sides of the globe, and let us call these event A and event B.  These events are documented by documents A and B respectively, and these documents are represented as data in a computer network.  In order for there to be consistency the final state S must be the same irrespective of the order at which the data for events A and B arrive.  In fact the final state of the computer system must be uniquely determined by the set of documents it contains irrespective of the order of their arrival.

Achieving this end is about reinterpreting document semantics.  At the place and time where the document was issued it was considered correct and proper to do so.  That it might be considered in this time and place improper is not material as we may are not and can not be if full possession of all the facts, and new facts can and do become available all the time. If data representing a withdrawal of cash over a credit limit arrives, they have got their money so do not dismiss it as invalid.  Rather there may still be a deposit on the way, or an increased credit limit that has not arrived yet.

The property of temporal commutativity allows for any collection of documents only a single final state.  It also considerably reduces the complexity of specifying and testing systems in that it reduces the test cases to n squared for a given document set of n documents to assure temporal commutativity.  If commutativity is not assumed then there are many more cases to consider.  To illustrate consider a set of three numbers a,b and c.  If they are all joined with a + then there is only one answer, a+b+c=a+c+b=b+a+c=b+c+a=c+a+b=c+b+a.   If we were to use a minus operator then some of those mathematical expressions would have had a different answer.

To illustrate let a=1, b=2, c=5.  a-b-c=-6 = a-c-b,  b-a-c=-4=b-c-a and c-a-b = 2 = c-b-a.  Because in general the - operator is non-commutative by definition the order of the terms is significant.

Finally, there are cases where ACID transactions are best, such as selling seats in a stadium for a match.  There is one authoritative perspective which means that seats are not oversold, and as with everything it is all about using the right tool for the job.