Both my son and I are running Mac OS 10.8.5 with Java 1.6.0_51. The 1.6.4-9.11.0.898 installer ran fine on his MacBook Pro but I got the blank window on my MacBook Air. Despite the OK button being there, hitting return does nothing.
However...
Go to the cow.mods.fml.installer.SimpleInstaller menu and open it, nothing. Pull down to "About..." still nothing. Pull down to "Services..." and boom, the window loads. Even though it loads, it still won't do anything... unless you open the menu to "Services..." again. You have to click the button and then open the menu to get it to execute.
While it did start working, I eventually got this error:
"These libraries failed to download. Try again. org.scala-lang:scala-library:2.10.2, org.scala-lang:scala-compiler:2.10.2"
This is despite the status window saying it had actually downloaded and started unpacking that library.
So, given the different results on two machines with the same Java versions, it seems to be environmental but I haven't figured out what it is yet.