08 April 2009

Running Sinatra apps on Google AppEngine (Java)

Update(14.04.2009): Jruby moved to git today. Please use git instead of svn to checkout the jruby project.

Google today announced Java as a new runtime environment for Google AppEngine. This not only enables developers to use the Java Language to build web applications but also opens the door for a lot of dynamic languages including my current favourite one Ruby. With the help of the Jruby project it is possible to deploy ruby apps in Googles Cloud.

At Bigcurl most applications are written in ruby and a new hosting option which is basicly free is always welcomed. Take a look at some of our internal apps for yourself (www.squidshot.com) and try to spot which ones are hosted in the cloud and which ones are hosted in a traditional data center.

This is a proof of concept which shows that it is possible to run ruby applications and maybe also ruby on rails applications on Google AppEngine. With the help of Ola Bini and the AppEngine Docs for Java , I created a tiny sinatra app (which acts as a placeholder for your much-more-logic-containing-app) and show how to use Google AppEngine as a ruby deployment option.

This was done on Mac OS X 10.5.6.


Jruby

First check out a fresh copy of jruby
git clone git://kenai.com/jruby~main

go into the jruby dir
cd jruby~main

and compile jruby.
ant && ant jar-complete

Lets see if we have the correct version.
bin/jruby -v

The output should be something like:
jruby 1.3.0 (ruby 1.8.6p287) (2009-04-08 r9524) (Java HotSpot(TM) Client VM 1.5.0_16) [i386-java]

Install some gems for our newly created jruby.
PATH-TO-JRUBY/bin/jruby -S gem install rake sinatra warbler


jruby-rack

Get a fresh version from http://github.com/nicksieger/jruby-rack/tree/master

cd jruby-rack
PATH-TO-JRUBY/bin/jruby -S rake SKIP_SPECS=true


We will come back to jruby-rack later.


Create the sinatra app.


Create a new folder "sinatra-app"
cd sinatra-app
touch config.ru app.rb appengine-web.xml
mkdir views public config lib


Fill the files config.ru, config/warble.rb, appengine-web.xml and app.rb with the content from this gist: http://gist.github.com/91801

This is our basic Sinatra app. It will just display a string. But then we know it is working.

For a quick test run:
ruby app.rb

and go to http://localhost:4567/

Now we need to copy some files to the lib dir.
First go to the download section to find the Google App Engine SDK for Java. Version 1.2.0 - 04/07/09 is here.

After the download is finished we need to copy appengine-java-sdk-1.2.0/lib/user/appengine-api-1.0-sdk-1.2.0.jar to the lib dir.

Copy following jar file from jruby-rack to lib folder.
JRUBY-RACK/target/jruby-rack-*.jar.

Next up is jruby itself. Since Ola pointed to the 1000 file limit with AppEngine he wrote a script which splits the jruby-jar into two pieces.

But first copy jruby-complete.jar from PATH-TO-JRUBY/lib/jruby-complete.jar to the lib folder of the sinatra app.

Then run his script in the lib folder and you should have two jar files instead of one. jruby-core.jar and ruby-stdlib.jar. You find the script here.

Now we should be ready to pack our application.
run PATH-TO-JRUBY/bin/jruby -S warble

This should create a tmp folder and a .war file. Since AppEngine needs the exploded war folder and not the .war file, we simply ignore the file

Go to tmp/war/WEB-INF/gems/gems/sinatra-0.9.1.1/lib/sinatra.rb and commend out the last line which is use_in_file_templates!. Somehow this makes problems with the runtime.

Run appengine-java-sdk-1.2.0/bin/dev_appserver.sh tmp/war/ to get a local server up and running for testing stuff.

If you have no error on the console go to http://localhost:8080 and you should see a nice welcome message in form of the string from the app.

Things work fine? Lets deploy.

You need to be signed up as one of the 10.000 developers who have early access to the java runtime. Sign up here if you haven't allready.
If so got to the Google AppEngine, sign in and create a new application.
Copy the application-id you get from there into the appengine-web.xml file and replace YOUR-APPLICATION-ID.

Run this to repopulate with the new id
PATH-TO-JRUBY/bin/jruby -S warble

and then deploy running
appengine-java-sdk-1.2.0/bin/appcfg.sh update tmp/war/

Now got to your-application-id.blogspot.com and it should work. The first request takes quite a while but the following should be fine.

Have fun!

Want to redirect naked domains in App Engine. Take a look here.

33 comments:

dkoontz said...

Thanks for the helpful (and very *fast*) writeup. I know I'll be trying out some JRoR apps here in the near future.

aemadrid said...

Great article. The funny thing is that you _really_ need to wait for that email authorizing you to use the _java_ runtime. If you try to upload your app before that the error you will get is "400 Invalid runtime" instead "Not authorized yet". I also had to comment out the "use_in_file_templates!" for Sinatra to work locally although it might be a Linux problem.

Samuel Goebert said...

@aemadrid
The confirmation email took only a few seconds for me to arrive but you are totaly right on commenting out line 8 in Sinatra. I forgot to mention it but I already corrected the article.

Thanks for keeping an open eye.

Nicolai said...

Very nice. I'll try it as soon as I find some free time.
Would you like to demonstrate it at our next RUG meeting?

aemadrid said...

@Samuel Do you know why it throws that FileNotFound error? Have you tried doing anything more complex?

Samuel Goebert said...

@aemadrid
I forked and patched sinatra on github, this morning. Find the repo here http://github.com/bigcurl/sinatra/tree/master .
I also sent out pull request so hopefully this is in the next version of sinatra.

Anonymous said...

it would be great if you can provide a skeleton sinatra app with all jars and sinatra lib and configs included on github, so people can just fork and start develepoing apps.

basaah said...

Great article!
Got it to work even though I have no jruby experience.
For all those who are - just like me - too lazy to compile: Just download the four jar files here: http://github.com/olabini/yarbl/tree/8681995b548860e2e13c90c4cf030fc682a32f34/lib

One small thing: I had to rename config/warbler.rb to config/warble.rb to make it work.

Samuel Goebert said...

@basaah
Good catch. I updated the gist.

@Anonymous
basaah is right. Ola Bini provides some of the files in his repository. You still need most of the stuff localy compiled to run warbler though.

basaah said...

@Samuel Goebert
Maby you should change it here as well:
Fill the files config.ru, config/warbler, appengine-web.xml and app.rb with the content from this gist: http://gist.github.com/91801

Ola Bini's Bumble also works. Just add the files in the lib folder and add this line to config.ru
require 'lib/bumble'

Now trying out Beeu, but I think it will need some modification to work with sinatra.

basaah said...

I Got some information on to use UserService/Beeu
http://gist.github.com/92566

It's all working perfectly

DominixZ said...

mkdir view public config lib

This line make error in warble it must be

mkdir views public config lib

because in git warble.rb is "views"

Samuel Goebert said...

@DminixZ, basaah
changed!

Anonymous said...

Worked with a small hickup. I needed to run:

appcfg.sh --enable_jar_splitting update tmp/war/

Anonymous said...

Thanks, Samuel!

I have not jruby-complete.jar in /PATH/TO/JRUBY/lib/jruby-complete.jar, just jruby.jar. What i'm doing wrong?

Basaah said...

@Anonymous on April 10, 2009 10:40:00 AM CEST
You probably forgot this line:
ant jar-complete

Anonymous said...

@Basaah
Oh you're right!
Thanks.

aemadrid said...

@Samuel Goebert
Thanks again for the patch for Sinatra.

Bodaniel Jeanes said...

alternative to the first step is of course:

git clone git://github.com/jruby/jruby.git

naum said...

tried following these:

jruby has no juby-complete.jar, it was named jruby.jar and i renamed it before running split-jruby.sh…

after that, no gems folder in /tmp/war/WEB-INF…

but the split-jruby.sh did create jruby-core.jar and ruby-stdlib.jar

Thamster said...
This comment has been removed by the author.
Xeph said...

Thanks for the write-up. Can't get it to run though.

When I launch the server I get this error:

2009-04-18 10:08:21.127 java[7893:80f] [Java CocoaComponent compatibility mode]: Enabled
2009-04-18 10:08:21.127 java[7893:80f] [Java CocoaComponent compatibility mode]: Setting timeout for SWT to 0.100000
Apr 18, 2009 10:08:25 AM com.google.appengine.tools.development.ApiProxyLocalImpl log
SEVERE: [1240042105059000] javax.servlet.ServletContext log: unable to create shared application instance
org.jruby.rack.RackInitializationException: undefined local variable or method `null' for #<Rack::Builder:0xf8a19f>
from file:/Users/xeph/gaej/app/WEB-INF/lib/jruby-rack-0.9.4.jar!/rack/builder.rb:29:in `initialize'
from <script>:2

Any ideas?

meekish said...

Has anyone figured out how to load the GAE development environment in rake or jirb?

Travis Bell said...

@Xeph

Same problem here. You figure that out?

Travis Bell said...

@Xeph,

The problem for me was that I had packed the app once before, not including the proper warble.rb file so there was a left over jruby-rack.jar that wasn't the right one. Deleted the tmp dir, re-packed and boom worked. Just didn't follow the instructions right :D

RafaƂ Sobota said...

"Now got to you-apllication-id.blogspot.com"
-------------------------------^

flavio said...

it workedm thanks
but in my application I would like to use a xml parser, with ruby mri I was using nokogiri, but when I tried to run in google sdk server it broked

Unsupported platform: unknowm-linux from file:.....

what i need to do?
thanks

flavio said...

the stack

SEVERE: [1252016053829000] javax.servlet.ServletContext log: unable to create shared application instance
org.jruby.rack.RackInitializationException: Unsupported platform: unknown-linux
from file:/home/flavio/labs/jrubyrssdip/tmp/war/WEB-INF/lib/ruby-stdlib.jar!/ffi/ffi.rb:71
from file:/home/flavio/labs/jrubyrssdip/tmp/war/WEB-INF/lib/ruby-stdlib.jar!/ffi/ffi.rb:1:in `require'
from file:/home/flavio/labs/jrubyrssdip/tmp/war/WEB-INF/lib/ruby-stdlib.jar!/ffi.rb:1
from file:/home/flavio/labs/jrubyrssdip/tmp/war/WEB-INF/lib/ruby-stdlib.jar!/ffi.rb:9:in `require'
from /home/flavio/labs/jrubyrssdip/tmp/war/WEB-INF/gems/gems/nokogiri-1.3.3-java/lib/nokogiri.rb:9
from /home/flavio/labs/jrubyrssdip/tmp/war/WEB-INF/gems/gems/nokogiri-1.3.3-java/lib/nokogiri.rb:4:in `require'
from script:4
from /home/flavio/labs/jrubyrssdip/tmp/war/WEB-INF/gems/gems/rack-1.0.0/lib/rack/builder.rb:29:in `instance_eval'
from /home/flavio/labs/jrubyrssdip/tmp/war/WEB-INF/gems/gems/rack-1.0.0/lib/rack/builder.rb:29:in `initialize'
from script:2

main said...

@flavio I guess the best option you have is to use a xml lib which is pure ruby like rexml.

John Griffiths said...

Thanks, updated your notes and posted two articles on the subject, awesome stuff!

Got it to work on Snow Leopard, Sinatra, Builder and with naked urls!

http://www.red91.com/2009/09/12/sinatra-on-google-appengine
http://www.red91.com/2009/09/12/naked-urls-on-google-appengine

All the best

FranciscoCabrita said...

Hi,

I have JRuby + Sinatra on GAE up and running without problems.

My question is, it is possible to use some kind of "autoreload" like shotgun? It's too bad restarting dev_appserver.rb everytime I want to see my changes.

Thanks in advance.

Regards
Francisco

killallhumans said...

Great stuff! Except those creepy warnings. I have created a skeleton application with Jruby & Sinatra + GWT frontend, deployable to AppEngine, enjoy: http://github.com/skrat/sinwar

running online at: http://sinwar-demo.appspot.com

Santiago GL said...

Hello, thank for this post. I'm having same problem as reported by naum on April 15, 2009

no gems folder in /tmp/war/WEB-INF…

this is what i've got within WEB-INF ...

tmp/war/WEB-INF$ ls
lib web.xml

Post a Comment