User-defined Functions (UDFs) have a long history of usefulness in SQL-derived languages. While query languages can be rich in their expressiveness, there's just no way they can anticipate all the things a developer wants to do. Thus, the custom UDF has become commonplace in our data manipulation toolbox.
Apache Hive is no different in this respect from other SQL-like languages. Hive allows extensibility via both Hadoop Streaming and compiled Java. However, largely because of the underlying MapReduce paradigm, all Hive UDFs are not created equally. Some UDFs are intended for "map-side" execution, while others are portable and can be run on the "reduce-side." Moreover, UDF behavior via streaming requires that queries be formatted so as to direct script execution where we desire it.
The intricacies of where and how a UDF executes may seem like minutiae, but we would be disappointed time spent coding a cumulative sum UDF only executed on single rows. To that end, I'm going to spend the rest of the week diving into the three primary types of Java-based UDFs in Hive. You can find all of the sample code discussed here.
Hive provides three classes of UDFs that most users are interested in: UDFs, UDTFs, and UDAFs. Broken down simply, the three classes can be explained as such: