A while ago I was playing with the Twitter Streaming API, one of the first things I wanted to do was collect a lot of data for off-line analysis (in Hadoop). I wrote a hacky little utility class called TwitterConsumer.java that did just the trick. Basically you just initialise it with a valid Twitter account (username/password) and give it the URL of the stream you would like to consume, this Read More...
This post has been obsoleted by my update here: Hadoop: Processing ZIP files in Map/Reduce One of the first use-cases I had for playing with Apache Hadoop involved extracting and parsing the contents of thousands of ZIP files. Hadoop doesn’t have a built-in reader for ZIP files, it just sees them as binary blobs. To solve the problem I wrote two small utility classes, ZipFileInputFormat.java and ZipFileRecordReader.java. They extend the default Read More...
A quick and simple Velocimacro to generate HTML Select boxes and its Options; automatically selects the correct option based on the value given. You can place this #macro anywhere in your Velocity templates but I used my global VM_global_library.vm file. Code [code] #macro( generateSelectBox $name $options $value ) #foreach ( $option in $options ) #set( $selected = "" ) #if ( $option == $value ) #set( $selected = "selected='true'" ) Read More...
This is a very simple utility function to produce an indented XML document as a String given a Document. It throws a TransformerException if anything drastic goes wrong. [java] import java.io.*; import org.w3c.dom.Document; import javax.xml.transform.*; import javax.xml.transform.dom.DOMSource; … public static String documentToString( Document document ) throws TransformerException { TransformerFactory tFactory = TransformerFactory.newInstance(); Transformer transformer = tFactory.newTransformer(); DOMSource source = new DOMSource(document); StringWriter sw = new StringWriter(); StreamResult result = new Read More...
I was working on a web-based application recently and wanted a lazy way to get data from a simple ‘value object’ out to the front end without having any hard-coded field names; I came up with this. Basically, this method will take any Object, iterate through its methods looking for any of the format “get<something>” then place the returned value in a new child element of the document. Obviously the Read More...
Builds a JAR file Test.jar and generates a manifest specifying that class Test is the Main-Class. [xml] [/xml]
A better random number generator [java] import java.security.SecureRandom; … public static SecureRandom random = null; static { try { random = SecureRandom.getInstance(“SHA1PRNG”); random.setSeed( random.generateSeed(256) ); } catch( Exception e ) { e.printStackTrace(); } } [/java]
When using JDK 1.4 and above… [java] public static void copyFile(File in, File out) { try { FileChannel sourceChannel = new FileInputStream(in).getChannel(); FileChannel destinationChannel = new FileOutputStream(out).getChannel(); sourceChannel.transferTo(0, sourceChannel.size(), destinationChannel); // or, you can also copy it this way // destinationChannel.transferFrom(sourceChannel, 0, sourceChannel.size()); sourceChannel.close(); destinationChannel.close(); } catch ( Exception e ) { e.printStackTrace(); } } [/java]
Returns true on success, false on failure: [java] public static boolean deleteDir(File dir) { if (dir.isDirectory()) { String[] children = dir.list(); for (int i=0; i<children.length; i++) { boolean success = deleteDir(new File(dir, children[i])); if (!success) { return false; } } } return dir.delete(); } [/java]