Generics based Framework for Composing and Submitting .Net Hadoop MapReduce Jobs

To enable non-Java based MapReduce jobs Hadoop supports a Streaming interface. To better support this interface in .Net I have constructed a composable framework to enable one to submit .Net based MapReduce jobs whilst only writing Mappers and Reducers types.

C#, F# (2.2 MB)
 
 
 
 
 
(5)
2,427 times
Add to favorites
6/14/2013
E-mail Twitter del.icio.us Digg Facebook
Sign in to ask a question


  • Base Type for Mapper
    1 Posts | Last post August 06, 2012
    • Currently the base type for teh Mapper is along the lines of:
      MapperBaseText<V2>
      MapperBaseBinary<V2>
      MapperBaseXml<V2>
      
      I was thinking of changing this to be something like
      MapperBase<V1, V2> where V1 : String, XDocument, Stream
      
      Based on the type the underlying EXE could then be called.