Life of Larry

A One and a Two

Objects Serialization in Rails

| Comments

Let’s talk about serializing objects in Rails today.

So, what’s the difference of serialization between in Ruby and in Rails?

In short, Rails uses ActiveSupport::JSON module to deal with JSON instead of using gems directly.

Rails 3.2 & Rails 4.0

So why would we talk about Rails 3.2 version since Rails 4.2.0.rc1 is released and Rails 5.0 development has begun?

Because we still have applications using Rails 3.2, and it’s really such a pain to upgrade apps with so many complicated logics like us.

ActiveSupport::JSON

ActiveSupport::JSON module provides a super simple API composed by two methods:

  • ActiveSupport::JSON.encode(object)
  • ActiveSupport::JSON.decode(string)

ActiveSupport::JSON.encode(object) takes a Ruby object as value and returns a JSON-encoded string. On the opposite, ActiveSupport::JSON.decode(string) takes a JSON-encoded string and returns the corresponding Ruby object.

Here are a few excamples:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
[1] pry(main)> require 'rails'
=> true
[2] pry(main)> Rails.version
=> "3.2.21"
[3] pry(main)> j = ActiveSupport::JSON
=> ActiveSupport::JSON
[4] pry(main)> j.encode(23)
=> "23"
[5] pry(main)> j.encode("A string")
=> "\"A string\""
[6] pry(main)> j.encode({ :color => ["red", "green", "jellow"] })
=> "{\"color\":[\"red\",\"green\",\"jellow\"]}"
[7] pry(main)> j.encode({ :color => ["red", "green", "jellow"], :date => Time.now })
=> "{\"color\":[\"red\",\"green\",\"jellow\"],\"date\":\"2014-11-30T00:02:29+08:00\"}"
[8] pry(main)> j.decode(j.encode({ :color => ["red", "green", "jellow"], :date => Time.now }))
=> {"color"=>["red", "green", "jellow"], "date"=>"2014-11-30T00:02:38+08:00"}

As I mentioned in my previous article, we should implement two more methods to make our custom object serialization work: to_json and self.json_create. Do we have to do that if we are using Rails a.k.a ActiveSupport::JSON?

The common usage of serializing custom objects in Rails is serializing them to a string field stored in database. And we could achieve that with writing our own custom serializer for custom objects. I won’t cover this in this post, maybe in the near future I will do that.

Performance

So, like the previous post, we will pay some attention on the performance of serialization in Rails.

ActiveSupport depends on multi_json gem, which is a simple library that allows you to semalessly provide multiple JSON backends with intelligent defaulting. And ActiveSupport::JSON uses json as its default engine, we could know that by ActiveSupport::JSON.engine # => MultiJson::Adapters::JsonGem.

Then let’s do some benchmarking, eg: setting oj as ActiveSupport’s default engine.

benchmark_active_support_and_json_and_oj.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
require 'rubygems'
require 'bundler/setup'

require 'active_support'
require 'active_support/core_ext/object/to_json'
require 'active_support/json/encoding'
require 'oj'
require 'benchmark/ips'

json_obj = {
  "a" => "Alpha",
  "b" => true,
  "c" => 12345,
  "d" => [true, [false, [-123456789, nil], 3.9676, ["Something else.", false], nil]],
  "e" => {
    "zero"  => nil,
    "one"   => 1,
    "two"   => 2,
    "three" => [3],
    "four"  => [0, 1, 2, 3, 4]
  },
  "f" => nil,
  "h" => {"a"=>{"b"=>{"c"=>{"d"=>{"e"=>{"f"=>{"g"=>nil}}}}}}},
  "i" => [[[[[[[nil]]]]]]]
}

ActiveSupport::JSON.engine = :oj

Benchmark.ips do |x|
  x.report("active_support") { json_obj.to_json }
  x.report("oj") { Oj.dump json_obj }
end

Output:

1
2
3
4
5
6
Calculating -------------------------------------
      active_support   250.000  i/100ms
                  oj    26.304k i/100ms
-------------------------------------------------
      active_support      2.532k (± 4.0%) i/s -     12.750k
                  oj    319.969k (± 5.7%) i/s -      1.605M

And the result may just blow your mind. We already specify our engine to Oj, right? They are supposed to behave like each other . What’s wrong with active_support?

The root cause is ActiveSupport’s implementation problem. In active_support/core_ext/object/to_json.rb file :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# Hack to load json gem first so we can overwrite its to_json.
begin
  require 'json'
rescue LoadError
end

# The JSON gem adds a few modules to Ruby core classes containing :to_json definition, overwriting
# their default behavior. That said, we need to define the basic to_json method in all of them,
# otherwise they will always use to_json gem implementation, which is backwards incompatible in
# several cases (for instance, the JSON implementation for Hash does not work) with inheritance
# and consequently classes as ActiveSupport::OrderedHash cannot be serialized to json.
[Object, Array, FalseClass, Float, Hash, Integer, NilClass, String, TrueClass].each do |klass|
  klass.class_eval do
    # Dumps object in JSON (JavaScript Object Notation). See www.json.org for more info.
    def to_json(options = nil)
      ActiveSupport::JSON.encode(self, options)
    end
  end
end

Did you see the to_json method? It will use ActiveSupport::JSON.encode every time with hard-coded guaranteed.

This monkeypatch was introduced to fix a problem (see comments above the code), but also caused another pain: If you are currently using the JSON/oj/whatever gem adapter with Rails (AS::JSON::Encoding), and you have been calling object#to_json, you are actually using Rail’s pure Ruby JSON encoder. And this explains why the performance suck so much.

Then how to fix this? The solution is quite simple: apply a similar change as the one in active_support/core_ext/object/to_json.rb, monkeypatch to override to_json method again, let it use Oj to do encoding job.

benchmark_with_hard_coded_patch.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
require 'rubygems'
require 'bundler/setup'

require 'active_support'
require 'active_support/core_ext/object/to_json'
require 'active_support/json/encoding'
require 'oj'
require 'benchmark/ips'

json_obj = {
  "a" => "Alpha",
  "b" => true,
  "c" => 12345,
  "d" => [true, [false, [-123_456_789, nil], 3.9676, ["Something else.", false], nil]],
  "e" => {
    "zero"  => nil,
    "one"   => 1,
    "two"   => 2,
    "three" => [3],
    "four"  => [0, 1, 2, 3, 4]
  },
  "f" => nil,
  "h" => {"a"=>{"b"=>{"c"=>{"d"=>{"e"=>{"f"=>{"g"=>nil}}}}}}},
  "i" => [[[[[[[nil]]]]]]]
}

[Object, Array, FalseClass, Float, Hash, Integer, NilClass, String, TrueClass].each do |klass|
  klass.class_eval do
    def to_json(opts = nil)
      Oj.dump(self, opts)
    end
  end
end

Benchmark.ips do |x|
  x.report("active_support") { json_obj.to_json }
  x.report("oj") { Oj.dump json_obj }
end

Output:

1
2
3
4
5
6
Calculating -------------------------------------
      active_support    24.917k i/100ms
                  oj    25.228k i/100ms
-------------------------------------------------
      active_support    289.260k (± 6.9%) i/s -      1.445M
                  oj    300.640k (± 6.4%) i/s -      1.514M

Someone has created a gem called rails-patch-json-encode to fix this error. You could just use it in your app.

Rails 4.1

Rails 4.1 removes multi_json dependency, and the previous patch/gem is unnecessary and will no longer work. Instead, if you want to use oj to deal with JSON, use the oj_mimic_json gem with oj in your Gemfile to have Oj mimic the JSON gem and be used in its place by ActiveSupport JSON handling:

1
2
gem 'oj'
gem 'oj_mimic_json'

And the rest is just easy.

References

Objects Serialization in Ruby

| Comments

Let’s talk about serializing objects in Ruby today.

Built-In Serialization Mechanisms

Ruby has two object serialization mechanisms built into the lauguage. One is what we are very familiar of, YAML(YAML Ain’t Markup Language), which is also human readable format, and the other one is binary format.

YAML Serialization

In Ruby, any objects can be serialized into YAML format. And it’s really easy:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
require 'yaml'

class A
  def initialize(string, number)
    @string = string
    @number = number
  end
end

class B
  def initialize(number, a_object)
    @number   = number
    @a_object = a_object
  end
end

class C
  def initialize(b_object, a_object)
    @b_object = b_object
    @a_object = a_object
  end
end

a = A.new("hello world", 5)
b = B.new(7, a)
c = C.new(b, a)

serialized_object = YAML::dump(c)

puts serialized_object

d = YAML::load(serialized_object)
require 'pry'; binding.pry

And the serialized_object looks like this:

1
2
3
4
5
6
7
--- !ruby/object:C
b_object: !ruby/object:B
  number: 7
  a_object: &1 !ruby/object:A
    string: hello world
    number: 5
a_object: *1

How Did Tenderlove and Others Speed Up Rails?

| Comments

Rails 4.2.0 beta1 was released August 20, 2014. And according to dhh’s release post, and I quote,

a lot of common queries are now no less than twice as fast in Rails 4.2!

So, what did Rails team – or more specifically – tenderlove (Aaron Patterson) do to improve Rails/ActiveRecord so much? Let’s find out through some commits.

Performance Tools

Here are some tools Aaron has used for measuring performance according to his Cascadia Ruby 2014 talk:

You should definitely checkout these tools. It would be very useful in your daily Ruby/Rails development.

Do You Have a Reason to Rejoice Recently?

| Comments

我似乎没有。

这大概源于我平淡无奇没有太多挑战的工作,也可能是因为几乎两点一线的生活,又或许就是我这个人的性格原因。总之,一切都是趋于平静的。当然,这对我来说也未尝不是一件好事。

但我又是一个不安于现状的人,总是渴望着改变。在学校时渴望着去公司历练,大三刚结束的暑假就来了北京,先是腾讯又是百度实习了一整年,也就此开始了北漂;毕业后在百度呆了半年多又觉得大公司的工作太没吸引力,还是跟朋友们一起出去闯闯来得痛快,就想也没想得辞职过上了没有收入的创业生活,多少有些鲁莽,但是也确实是我能干出来的事儿;散伙后就来了现在的公司,期间也想过上真正的 freelance 生活,却还是被 leader 留下,一呆就是两年。

还是我常说的那句话,人短短一辈子,何必在一个地方停留太久?我们总是被各种各样的理由羁绊与束缚,往往被蒙住了双眼,忘记了初心。

Performance Differences in Ruby

| Comments

前几天 @sferik 在 Parley 上说最近在准备一个关于 Performance in Ruby 的 talk,发贴讨论了 Ruby 中哪些写法会造成性能的巨大提升,我顺手整理了下帖子内容并实验了一下。

Benchmark 环境:MacBook Air Mid 2012, Ruby 2.1.1, gem benchmark-ips.

Proc#call versus yield

1
2
3
4
5
6
7
8
9
10
11
12
13
14
require 'benchmark/ips'

def slow(&block)
  block.call
end

def fast
  yield
end

Benchmark.ips do |x|
  x.report("slow") { slow { 1 + 1 } }
  x.report("fast") { fast { 1 + 1 } }
end
1
2
slow   770263.8 (±4.8%) i/s -    3849832 in   5.010201s (5 秒钟可运行 3849832 次)
fast  3985294.7 (±8.7%) i/s -   19751563 in   5.001024s (5 秒钟可运行 19751563 次)

从上面的 benchmark 结果中可以看出两种写法的显著性能差异,这主要是因为第一种写法中要不断的创建 Proc 对象赋给 block 参数导致的。

也无风雨也无晴

| Comments

如果一年前你告诉我,生活将会变成这样,我一定不会相信。

由于上学早的缘故,我通常比同届的同学小一、两岁。很多人对我说过「你还是太年轻」之类的字眼,我听到后往往都是抵触与不屑。也有人说我心理年龄实在太大,超过了我这样年龄的年轻人应有的想法、快乐甚至自由,我也觉得没什么。所以想想,好像大家的想法都是错觉,我也不愿解释什么。

真正让我开始反省自己心理不成熟的一面,是在面临困境的时候,内心与表现的手足无措以及并非发自本心的行动与话语,多少伤害了家人的心。我知道他们不会介意,甚至从未想起,但我深知,我做得是多么得不好。

那段时间,心理上的压力差一点就把我吞噬了。生活中的我也变得越来越敏感,偶尔午夜梦醒,就睁着眼睛发呆。跟女朋友也经常因为这样那样的事情吵架,现在看来都是那么的无关紧要。做什么事情都无精打采,球局饭局酒局能推就推,脸上似乎也常常带着忧愁,想法每天都在变,一切的一切都说明着我心智上的不成熟。

手足无措。

好在这些都过去了,生活中的一切都回到了正轨,我也调整好了自己的心态,想明白了很多问题。没有无奈与抱怨,欣慰于风雨后的平淡。

其实这一切都和我预想的不太一样,我去年写博客还说2013年想要做个自由职业者换个城市生活,上半年原本已经跟老大提了辞职准备全职 freelance 四处游荡,却发现生活中的一切都变得复杂了起来,不由得我再这么做。前几天看 Linsanity,说到林书豪心中:God first, family second, basketball third。对于现在的我来说:Family first, everthing else second.

我希望家人过得快乐,为了这份快乐,我会更加努力。

Some Useful Vim Tips

| Comments

很久以前挖坑说介绍我常用的一些 vim 插件,回头看看那篇文章的日期发现已经是快一年前了,我的 dotfiles 也已经更新过无数次了。

今天不写插件,写写一些简单又能提高效率的 vim tips 吧。

Persistent undo

也就是将 undo 操作持久化,在你重新打开已被关闭/切换的文件时,依然可以通过 u 进行 undo 操作。

1
2
3
4
set undodir=~/.vim/.undo
set undofile
set undolevels=1000
set undoreload=10000

如何重构 Rails 项目

| Comments

Am I doing it wrong?

前天在 GitHub Gist 上看到一个有趣的讨论

事情的起源是 @justinko 因为自己写的一段代码而失去了一份工作合约。那么究竟是什么样的代码「惹火」了雇主呢?

在那样项目中,当用户发表评论后,后台需要做的不仅仅是在数据库中插入一条评论记录,还需要识别出评论的语言、检查是否是垃圾评论、发送邮件以及同步到 Twitter 和 Facebook。@justinko 觉得这些代码既不应该属于 controller 也不应该属于 User model, 也不应该用 ActiveRecordcallbacks 来实现。

于是他新建了一个 PORO (Plain Old Ruby Object) 来处理这些逻辑。代码如下:

post_comment.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
# app/use_cases/post_comment.rb
# Called from the "create" action in a controller

class PostComment
  def initialize(user, entry, attributes)
    @user = user
    @entry = entry
    @attributes = attributes
  end

  def post
    @comment = @user.comments.new
    @comment.assign_attributes(@attributes)
    @comment.entry = @entry
    @comment.save!

    LanguageDetector.new(@comment).set_language
    SpamChecker.new(@comment).check_spam
    CommentMailer.new(@comment).send_mail

    post_to_twitter  if @comment.share_on_twitter?
    post_to_facebook if @comment.share_on_facebook?

    @comment
  end

  private

  def post_to_twitter
    PostToTwitter.new(@user, @comment).post
  end

  def post_to_facebook
    PostToFacebook.new(@user, @comment).action(:comment)
  end
end

Use Pry Instead of IRB

| Comments

IRB 作为原生的 Ruby 交互式解释器,给每个 Rubyist 调试程序时都提供了极大的便利,包括 Rails console 都是构建在 IRB 之上。

但是有一款工具功能更加强大使用更加方便,那就是目前 Ruby 社区中很火的 Pry。现在是时候放弃使用 IRB 了。

Installation

Pry 的安装非常简单,因为它本身就是一个 gem. gem install pry (如果使用 rbenv 管理 Ruby 环境的话,别忘记 rbenv rehash)

然后运行 pry, 就可以像 irb 一样执行 Ruby 代码了。

Usage

0. Code Highlight & Indent

使用 irb 时很头疼的一点就是定义一个类/方法时,代码没有缩进,很容易写了好几行代码然后发现写错了不得不重头再写起。

使用 Pry 则完全不用担心这种问题,换行时会自动缩进,而且支持代码高亮。如下图: