sv rubyconf, day two

Follow us on the collaborative svruby.tumblr.com tumblelog!

[Tor Norbye on IDE support for Ruby]

Nick Sieger, JRuby: Ready for Prime Time

[JRuby, started at 2002 (CRuby 1.6)]

"Applets... remember those? Oooh."

Why Java?

+ HotSpot, Unicode, (Generational / Tunable) Garbage Collection, "java Monitoring & Management Console"

Microbenchmarks: CRuby beats JRuby (interpreted) most of the time, but JRuby (compiled) "starting to kick some butt" (work-in-progress)

Supported? Yes: Hpricot, Camping, etc.
Supported? Maybe: Mongrel, (Super-)RedCloth, ...

JRuby = more compelling for long running applications (due to start-up times)

[WAR builder live demo, deploy mephisto]

Big Bird (scaling twitter), Blaine Cook

Rails Scales (but not out of the box)

600 requests per second. 180 Rails Instances (Mongrel). 1 DB Server (MySQL) + 1 Slave.
30-odd Processes for Misc. Jobs, 8 Sun X4100s.

"March was difficult." (see graphs)

It's Easy, Really.
1. Realize Your Site is Slow
2. Optimize the Database
3. Cache the Hell out of Everything
4. Scale Messaging
5. Deal With Abuse

Don't Be Like Us

• Munin, Nagios
• AWStats, GOOG Analytics
• Exception Notifier / Exception Logger

"It sucks to get seven thousand emails in your inbox in the morning."

"The Next Application I Build is Going to Be Easily Partitionable" - Stewart Butterfield

"Denormalize A Lot." (singlehandedly what saved us)
"Don't Be Stupid: bob.friends.map(&:email)"
"We're so not blocked on our database."
_ActiveRecord nano_. 90% API Requests, Cache Them!

"There are only two hard things in CS: cache invalidation and naming things" -- Phil Karlton

DRb:
+ Stupid Easy (it's kinda slow actually)
+ Reasonably Fast
- Kinda Flaky, Zero Redundancy
- Tightly Coupled

ejabberd <-> jabberclient (drb)

Moved from DRb to Rinda, Shared Queue (TupleSpace). RingyDingy makes it stupid easy.
"O(N) for take(). Sigh."

Twitter, TCP/IP or UDP? "It's definitely UDP."

Options:

• ActiveMQ (Java)
• RabbitMQ (erlang) -- looks promising, no Ruby clients, "wasn't going to write one on Wednesday"
  
• MySQL + Lightweight Locking -- what LiveJournal does
  
• Something Else?

erlang? [What are you doing? Stabbing my eyes out with a fork.] Not a lot of documentation. How do I get this server to start? Developers, I don't know. Doing experiments with [erlang], but afraid of throwing it into production.

Starling: Ruby, will be ported to something faster. 4000 transactional msgs/s. First pass written in 4 hours. Speaks MemCache (set, get). Works great. Use Messages to Invalidate Cache.

Abuse:

The Italians -- "friending everyone on the site... costing us thousands of dollars." They were "inverse spamming themselves."

9000 friends in 24 hours (doesn't scale), delete those users.

Q: JavaScript for relative times on the twitter front pages? Yea, maybe we'll do that.

"We were on a 32-bit database... until Thursday."

Inject database, render time, and server source as <!-- HTML comments --> in pages themselves.

[A number of detailed questions and answers on their architecture, spoofing, scaling, and so on.]

Lunch

Yet Another (the third) [Nameless Search Engine] Developer Talk :)

On Heckle, by Ryan Davis and now Kevin Clark. Compare with Dawson Engler's work.

Ezra Zygmuntowicz (ez), started Engine Yard, blog at brainspl.at.

Mongrel -- "Faster is Possible"

"FastCGI is Poop"

Rails isn't Thread Safe, mutex around dispatch. Mongrel is thread safe.

"Building Mongrel Handlers is easier [than] you think"

":in_front => true to put a custom Mongrel Handler in process with your Rails app and have it intercept"

SecureFile handle example (should use Path.expand_path).

(Aside, see notes on Merb.)

Merb -- "Clean room implementation of ActionPack"
"Thread Safe with confiurable Mutex Locks"
"Rails compatible REST routing"

"Merb... A hacker's framework"

Merb holds mutex lock for much shorter compared to Rails dispatching.

Rails routing is 1800 lines of *very complex* non thread-safe code
Merb's router is 200 lines of complex but *thread safe* code and much more efficient for route matching

Compiles routes to regexen and a case.

"At this point it is not premature anymore ;)"

Merb -- less magicks! than Camping. More familiar to Rails developers.

Used in prod for half a million uploads+downloads every day.

Rubinius, Evan Phoenix

"A Tool of the Future, aka the road to enlightenment"

Mantra of rubinius: If it can be done in ruby, then it shouldn't be in C.

"An idealist is a person who helps other people to be prosperous." --Henry Ford

"If you're not failing every now and again, it's a sign you're not [doing] anything very innovative." --Woody Allen

Project has really sped up since last October.

Goals: 98% first class, 1.8.5 compatible, MRI (Matz Ruby Interpreter) C API compatible.

Timeline? 1.0 by October, 2007 - RubyConf. Near 100% compatible with 1.8.5, able to run Rails 1.2.

2.0, JIT and Optimizers.

Strongtalk? Can't understand it. Written in "C++ that's unable to penetrate my brain."

Currently supports green threads. Contains low level sync / sharing mechanism called Channel (PI Calculus-like channel.) Native Thread support in a MxN scheme (eventually).

"Compiler is a normal class, which opens doors..."

[Description of generational garbage collection strategies used.]

puts "OMG #{a} PONIES!"

Ed: okaybyethanksfortheconference!