Guice, Jersey, and Jackson

August 6, 2011

A couple of years ago, a coworker and I worked on a project that used Jersey, Guice, and Jackson together. At the time, he did all the wiring up of the various bits to make them play nice together. The other day, I wanted to create a new projet for some personal work that used the same stack.

I googled how to setup Guice and Jersey, and I found it surprisingly difficult to actually come up with a working example (besides this one from the incomparable Sonny Gleason). However, Sonny does all his stuff as embedded Jetty in an java application, and I prefer to use war files deployed to a container. So, I went about putting together a simple example, and I’ll pass along the salient points here.

1) you need to tell the servlet to use Guice as the delegate for requests
2) you need to setup Guice to handle the requests
3) profit!!!!

1) convert your web.xml to the following

<?xml version="1.0" encoding="UTF-8"?>
<!-- This web.xml file is not required when using Servlet 3.0 container, 
     see implementation details http://jersey.java.net/nonav/documentation/latest/jax-rs.html#d4e194 -->
<web-app version="2.5" xmlns="http://java.sun.com/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd">
    <listener>
        <listener-class>com.mathew.JerseyServletGuiceModule</listener-class>
    </listener>
    <filter>
        <filter-name>guiceFilter</filter-name>
        <filter-class>com.google.inject.servlet.GuiceFilter</filter-class>
    </filter>
    <filter-mapping>
        <filter-name>guiceFilter</filter-name>
        <url-pattern>/*</url-pattern>
    </filter-mapping>
</web-app>

2) create your Guice servlet module (note the usage of Jackson as the serialization library)

public class JerseyServletGuiceModule extends GuiceServletContextListener{

   @Override
   protected Injector getInjector() {
      return Guice.createInjector(new JerseyServletModule() {
         @Override
         protected void configureServlets() {

            //our persistence layer
            bind(PeristenceManager.class).to(PeristenceManagerImpl.class);

             //We can bind impls for resources
            bind(HandlerA.class).to(HandlerAImpl.class);

             //or we can bind the resource directly
             bind(HandlerB.class);

            /* bind jackson converters for JAXB/JSON serialization */
             bind(MessageBodyReader.class).to(JacksonJsonProvider.class);
             bind(MessageBodyWriter.class).to(JacksonJsonProvider.class);

            // Route all requests through GuiceContainer
            serve("/*").with(GuiceContainer.class, ImmutableMap.of(JSONConfiguration.FEATURE_POJO_MAPPING, "true"));
         }
      });
   }
}

Now you can create your handlers (with constructor injection, etc)

public class HanlderAImpl implements HandlerA {


    @Context
    final UriInfo info;

    final PersistenceManager pm;



    @Inject
    public HandlerAImpl(@Context UriInfo uriInfo, final PersistenceManager pm){

        info = uriInfo;
        this.pm = pm;
    }

    @Override
    @Produces({ MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML })
    public Result send() {       
        //Result is just an simple pojo with jackson annotations
        return new Result();
    }
}


Filtering and Transforming with Guava

August 5, 2011

In ruby, you can do operations on collections really easily. For instance, to convert an array of words to all uppercase, you can simply do

 ["foo1", "foo2", "foo3"].map {|w| w.upcase }

Java on the other hand is lacking in this regard. I set out to find a better way to manage these types of transforms in my java code, and wound up looking at Guava (nee Google Collections). Way back, a coworker (Dain Sundstrom) had shown me some great usage of the filtering abilities of the Google Collections, and I decided to take another look (’cause it’s been a while, and I forget things easily)

First, I defined the transformation I wanted to use

public class Transformers {

    public static Function Upcase = new Function() {
        @Override
        public String apply(final String from) {
            checkNotNull(from);
            return from.toUpperCase();
        }
    } ;
}

Then I actually use it (or define the transformation inline as per the second example)

   public static void main(final String[] args){

        List myList = new ArrayList();
        myList.add("foo1");
        myList.add("foo2");

        //now let's transform them
        List upcasedList = transform(myList, Transformers.Upcase);

        System.out.println(Joiner.on(", ").skipNulls().join(upcasedList));

        //If you want to be ruby/closure like, then try this with an inline def
        List downList =  transform(myList, new Function() {
            @Override
            public String apply(final String from) {
                return from.toLowerCase(); 
            }
        } );

        System.out.println(Joiner.on(", ").skipNulls().join(downList));
}

Simple enough once you get your head around it, but sadly, it’s not nearly as succinct or nice to look at.


Interactive Rails Console

June 4, 2011

One of the features that several of the webapps I’ve worked on have had, is interactive consoles that allow you to enter some form of code, and see that run inside the webapp process itself. At Ning, we used jruby on rails for one of the admin apps, and it was a life saver to be able to execute debugging code on the app, and see exactly how the app itself was interpreting the code. At Apple, we have an interactive Groovy console that serves the same purpose.

So, when I was working on my latest app, I wanted to add the same piece of functionality. Here’s how I did it. It’s straight forward, and not too complicated, but it has a nice feature that I’ve added after my experience with the previous consoles I’ve used. One of the big drawbacks with the other consoles, was that you would end up hitting back and submit over and over as you refined your code. I wanted to minimize that in the spirit of . Here’s how I did it.

Add a new controller to your rails app (in this case, I named it console)

class ConsoleController < ApplicationController
  
  helper :application        
  include ApplicationHelper  
    
  def index
    
  end
  
  
  def execute
    begin               
      @script = params[:script]  
      rendering_start = Time.now    
      @output = eval(@script)   
      @rendering_time = ((Time.now - rendering_start)* 1000.0).to_i
    rescue Exception => ex
        @output =  ex.inspect  + "\n" + ex.backtrace.join("\n") 
    end    
  end
end

Then you’ll need to create the following views
views/console/_console_form.html.erb


<%= form_tag(console_path, :method => "post") do %>
  <%= label_tag(:script, "Script:") %>
	<br/>
  <%= text_area_tag :script , @script || '', :size => "100x20" %>
	<br/>
  <%= submit_tag("Execute!") %>
<% end %>

views/console/index.html

<% content_for :right_bar do %>
  <p></p>
<% end %>

<%= render 'console_form' %>


views/console/execute.html
<!-- This will make the rails renderer send this output fragment to the  'yield :right_bar' block -->
<% content_for :right_bar do %>
  <p><%= "Execution time: #{@rendering_time} milliseconds" %></p>
<% end %>

<%= render 'console_form' %>

<div id=console_output>
	<pre>Script Output:
		<%= @output %>
	</pre>
</div>

You may be wondering what that content_for call is doing. In my app, I have a top bar, bottom bar (actually two of them, a right bar, and a left bar, and a center area that gets the main content). In this case, the right bar is getting the content from that call. It allows me to have a sort of context sensitive menu for each page (while I have a site wide menu on the left hand bar). I won’t muddy the waters by pasting my main layouts and css here, but you get the idea.

Now, the new page should be usable. After each execution, the edit box is prepopulated with the code you just submitted, so making corrections, or changes is very quick and easy. This will also tell you how long the script you just ran actually took (it’s not a hi-res timer, but it’s close enough for government work).

The really nice thing about this, is you can use your active record classes in here (or MongoMapper if you’re one of the cool kids). So, I can easily just run code like

User.find_by_id(5)[:username]

and see what my app will return for that database record.

It should be noted that this console can be very dangerous. There are no controls around deleting or updating data, so you need to be very careful about how you use this, and also make sure that you don’t expose this in any public site, as it’s sure to be abused at some point). I would add specific security checks for things like a specific user account, or Rails.env.development? ). I would probably also look at disabling this in the routes for the deployed app, just to be on the safe side.


Adding fuzzy search capability to Rails 3

June 2, 2011

I wanted to add the ability to do ‘fuzzy’ searches to my rails app (without having to resort to stronger solutions like solr or sphinx). This is more than a simple ‘like’ clause, as you want to be able to match on things that sound similar, or are otherwise ‘very close’.
I came across this blog post

http://unirec.blogspot.com/2007/12/live-fuzzy-search-using-n-grams-in.html

which took me to https://github.com/bmaland/no_fuzz which doesn’t seem to work with rails 3. After looking at the branches of the plugin, I managed to get the plugin installed and running with my users table.

in your rails app directory

Pull the plugin

rails plugin install git://github.com/beno/no_fuzz.git


rails generate no_fuzz User

Then we need to actually create the table to hold our trigrams

rake db:migrate


Now we need to tell our model to populate the trigrams when it’s created, and I added a simple wrapper method for doing the fuzzy searches

class User < ActiveRecord::Base
include NoFuzz
fuzzy :username

def initialize
  super
  User.populate_trigram_index
end

#this returns an array of User objects
def self.fuzzy_search(search_term)
  results = User.fuzzy_find(search_term, 10)
end

  #this returns an active relation object
  def self.search(search_term) 
    if search  
      where('username LIKE ?', "%#{search_term}%")  
    else  
      scoped  
    end  
  end

Now we can use both search methods for our data as shown here

User.fuzzy_find("ma", 10) returns an array of records
User.search("Mat") returns an active record relation

Edit. I wasn’t fully happy with the previous solution, since it wouldn’t play well with the sorting and pagination code that I use elsewhere. So, I ‘fixed’ the no_fuzz plugin to play nice with the rest of the app.

I edited the no_fuzz.rb file to modify the the fuzzy_find method to the following.

    def get_fuzzy_matches(word, limit = 0)
      word = " #{word} "
      trigram_list = (0..word.length-3).collect { |idx| word[idx,3] }
      trigrams = @fuzzy_trigram_model.where(["tg IN (?)", trigram_list])
      trigrams = trigrams.group(@fuzzy_ref_id)
      trigrams = trigrams.order('SUM(score) DESC')
      trigrams = trigrams.includes(belongs_to_association)
      trigrams = trigrams.limit(limit) if limit > 0
      trigrams
    end
    
    #this just returns the ids for the records that match
    def fuzzy_find_ids(word, limit = 0)
      trigrams = get_fuzzy_matches(word, limit)
      trigrams.all.collect do |trigram|
        trigram[belongs_to_association + '_id'] 
      end
    end

    #this returns the actual records that match.
    def fuzzy_find(word, limit = 0)      
      trigrams = get_fuzzy_matches(word, limit)
      trigrams.all.collect do |trigram|
         trigram.send(belongs_to_association)
      end
    end

Then I modified my user model to the following


  #this returns an array of User objects
  def self.fuzzy_search(search)
      results = User.fuzzy_find(search, 10)  
  end
  
  #this returns an active relation object
  def self.search(search) 
    if search  
      user_ids = User.fuzzy_find(search, 10)  
      where(["id IN (?)", user_ids])
    else  
      scoped  
    end  
  end

Finally, in my controller I was able start using this thusly (I’m using Kaminari for my pagination)

 @users = User.search(params[:search]).page(params[:page]).per(10)

Now that the code plays nice with AR, I was able to use it in my ajax’ified search box, so it provides lookahead suggestions as a user is typing (using jquery of course). It also integrates with the dynamic table view I have, to update the tables results based on what’s being searched for.

It should be noted that this has the side effect of possibly having your specified ordering override the ordering returned by no_fuzz, but if you’re only returning a small number of possible matches then it’s not a very big deal.


Process is the enemy

September 20, 2010

I have been fortunate to have been a part of organizations with many different level of process around their software development and deployment. I’ve been at startups (twice now), large companies, and currently the 900 lb gorilla in the valley. All these companies had their own views on process and control of the organization. Some of which I agree with, many of which I don’t.

Process provides the illusion of control by slowing the entire system down, and is proof of a failure in the engineering organization.

The general evolution of process is something like this. Someone in management decides that something needs to be made better. This may be something like ‘preventing the deployment of bugs’, or ‘defining more accurate user requirements’. To make this ‘thing’ better, they come up with a process. All engineers must now pass around a version document and get sign off from stake holders before starting work on a new project to ensure that the requirements are ‘aligned’ with what the user wants, or all software must sit in a QA environment for 1 week before it can be pushed out to production, etc.

These processes may make management feel better about what the organization is doing. It may make them feel much more in control, but the reality is that the organization now has a larger amount of inertia than before, and is moving slower. That is not what any organization really wants, and honestly, it’s not even what management wants.

Here’s a crazy idea. How about enabling your engineers to actually engineer software, instead of filling out wiki nodes with release notes. If there is a problem that needs addressing (and any organization will have more than it’s fair share of problems), then engineer a solution to the problem. This is the key. Manual processes are bad for everyone. Automate the process, and bake it into the organization. For example, write regression tests that must be passed before code can be checked in and deployed (with a corresponding commit hook that prevents checkins that reduce the overall unit test coverage). Give your end users proof of concept software to verify that you are on the right track, then iterate on that software, getting feedback all the time (if this sounds suspiciously like SCRUM, it’s pretty close).

The take away, is that almost any problem has a technical solution that can be automated. All that’s required is managements willingness to spend a little up front cost to prevent a continual drain on the engineers time. Assigning one engineer to implement the technical solution in a month is going to be cheaper in the long run than nickel and diming the entire organization for years to come.

You may be thinking that I am a test driven development zealot, and SCRUM aficionado, but nothing could be further from the truth. I think both of those take a good idea, and apply way too rigid a process to their implementation. Some of that may be the capitalist evolution of the idea, you can’t really sell a three page book on scrum, you need at least 150 pages worth of verbosity to charge $40. But the cynic in my thinks that it’s required to sell the ideas to process happy management. But I digress….

Another side effect of these processes is that no one really wants to do them. That means that people will cut corners every chance they get, and the mere fact that humans are having to do manual actions means that mistakes will be made. If someone forgets to edit a wiki node properly, it’s not a big deal. If, on the other hand, someone pushes the wrong software package out, that is a really big deal.

I honestly believe that the vast majority of processes that I’ve had to deal with, or have had friends gripe about, were completely unnecessary (i.e. they could have been automated). I would be interested to hear of any processes that others have run into that they don’t think they could be automated.


Maven tips and tricks

November 5, 2009

Here’s a couple of maven commands that are handing when it comes to debugging build problems.

If you want to see what deps Maven is pulling in (or hand tweak them), then you can use.

mvn dependency:tree -Dverbose

I usually pipe that to less, however, be aware that maven will print out all deps regardless of whether they are going to get used or not. You will need to look at the end of the output which shows the resolved versions that will be used.

If the issue is one of a class not getting loaded correctly, then you can try this

mvn dependency:copy-dependencies  -DoutputDirectory=...

then you can add them to your classpath manually and run the application, tweaking as needed to get your intended class to load.

The following will load a custom config for your maven build (so you can test out tweaks to your config)

mvn -s ./profile.xml compile


Rails 3 features

November 4, 2009

Yehuda Katz over at Engine Yard has a great post on some of the upcoming changes to rails 3. I’m really excited to see how it ends up. I especially excited to see what performance gains can be achieved with it running on a 1.9 ruby stack. Also of interest is the class level responder addition. That makes the code so much more compact and easy to read.


Setting cookies in NET::HTTP Requests

October 28, 2009

I had a hard time figuring out how to set cookies on a POST request with ruby. The docs are strangely silent on the matter (in fact, the docs for the HTTP::POST are actually empty) .  The docs for http don’t use this form of post. Google was also failing me in my searches, so I’m posting this example here to hopefully help the next poor soul that needs to do this.

Here is the example

  http = Net::HTTP.new(my_hostname, port_number)
  path = '/myform/action'

  # POST
  data =  'field1=#{value1}&field2=#{value2}'

  headers = {
    'Cookie' => 'cookie1=cookie_value1; cookie2=cookie_value2',
    'Referer' => 'referring_host.mydomain.com',
    'Host' => 'referring_host.mydomain.com'
  }

  resp, data = http.post(path, data, headers)
end

The rule of ‘And’

July 23, 2009

I don’t know if someone else has already come up with this, and I’m too lazy to actually use google to look for it, but I’ve come up with a new rule for writing code.

Here’s my thesis. When describing the purpose of a class, you shouldn’t be able to use the word ‘and’.

Here’s the explanation. Any class that you write should have a very small and clearly defined purpose. For example, I was writing a class to wrap the incoming parameters of a rest endpoint.  The problem was, I was trying to do too much with that one class. It was holding the pagination information, the sorting information, and the filtering information.  That’s way too much for one poor class to do.  It started out simple, but as I added the rest of the features from the spec, it got way too bloated. I knew, but tried to force it to work anyway.

In code review, my coworker pointed out that it was bloated, and I should refactor it out.  I knew it already but didn’t want to admit.  With his not so gentle push in the right direction, I spent the next three hours refactoring the code into three seperate classes.  Each class was around 100 lines and served a very defined purpose.  I was able to write unit tests for all three easily to verify that they worked correctly, and as a bonus, I was able to wrap my head around what they were doing much easily.

In the previous bloated object, if someone has asked me for a confidence level of how well it did it’s job across all the possible input domains to it, I’d have said somewhere around 80%. I know it did the right thing in my tests, but my tests are obviously biased.  With the new design, I’d be able to say with a 99% confidence level that each piece does the right thing.  That’s quite an improvement.

Another nice change with the smaller classes, was that it took me all of an hour and a half to get around 90% unit test coverage.  If I’d tried writing unit tests for the heavy version of the object, I’d probably have had to write way too many tests with a lot of overlap and wasted time.

So to recap.  Refactoring my object into three smaller objects resulted in cleaner more managable code.  Better and faster unit test coverage.  The code looked much more pleasing to the eye. Win, win, win.  Ok, the last one isn’t really a win, but it is a nice side effect.


soundex algorithm in C++

July 19, 2009

I couldn’t actually find a C++ version of the soundex algorithm (all the ones I found were C code with a .cpp extension), so I threw one together.
Here it is if anyone is interested.



static char     lookup[] = {
'0',    /* A */
'1',    /* B */
'2',    /* C */
'3',    /* D */
'0',    /* E */
'1',    /* F */
'2',    /* G */
'0',    /* H */
'0',    /* I */
'2',    /* J */
'2',    /* K */
'4',    /* L */
'5',    /* M */
'5',    /* N */
'0',    /* O */
'1',    /* P */
'0',    /* Q */
'6',    /* R */
'2',    /* S */
'3',    /* T */
'0',    /* U */
'1',    /* V */
'0',    /* W */
'2',    /* X */
'0',    /* Y */
'2',    /* Z */
};

std::string computeSoundex(const std::string &amp;input, const int resultLength){

//keep the first character intact
std::string result = input.substr(0,1);

//compute value for each character thereafter
for(int i=1;i&lt;input.length(); i++){
   //skip non-alpha characters
   if(!isalpha(input[i])){
   continue;
}

//uppercase the input value
const char lookupInput = islower(input[i]) ? toupper(input[i]) : input[i];
//lookup it's value
const char *lookupVal = &amp;lookup[lookupInput-'A'];

//make sure this isn't a dupe value
if(result.find(lookupVal, 0) != 0 ){
   result.append(lookupVal);
}
}

//make sure we could actually encode something
if(result.length() &gt;= resultLength){
return result.substr(0,resultLength-1);
}

//In cases of empty strings (or strings with no encodable
characters, return Z000
return "Z000";
}


Follow

Get every new post delivered to your Inbox.