New Abstractions for Clojure-HBase

I just pushed Compass Lab’s HBase Client API to my fork of the clojure-hbase library.  The API includes support for table schemas (to auto-encode inputs and outputs) and a constraint language that generates filters and calls for Get and Scan operations.  We decided to integrate with the existing fork to retain access to the excellent admin functionality already implemented there.  We’ll be talking with the original author and see if we’ll merge or split these two API development paths.  In the meantime, you can use our fork of clojure-hbase.


Steps and Flows: Higher-order Composition for Clojure-Hadoop

The clojure-hadoop library is an excellent facility for running Clojure functions within the Hadoop MapReduce framework. At Compass Labs we’ve been using its job abstraction for elements of our production flow and found a number of limitations that motivated us to implement extensions to the base library.  We’ve promoted this for integration into a new release of clojure-hadoop which should issue shortly.

There are still some design and implementation warts in the current release which should be fixed by ourselves or other users in the coming weeks. 

Continue reading “Steps and Flows: Higher-order Composition for Clojure-Hadoop”