Memory leak

[collimarco]

Even if I already mentioned this problem in #228 saying that I get a "Killed: 9" this is a totally independent problem from the segmentation fault and I think it deserves its own ticket.

Further investigating the "Killed: 9" I discovered (quite obvious) that the ruby process steadily increases memory usage.

The line responsible for this issue is:

@entity = Oj::Doc.open(@line)

In fact by simply substituting it with

@entity = JSON.parse(@line)

I get no problems with memory usage. (The above lines produce different objects, but this is not a problem, since in this demonstration @entity is never used. Also I could have simply removed the line instead of substituting it, but I have done this to prove that is not simply a problem with the garbage collector)

Adding @entity.closeafter the open doesn't solve the issue.

[ohler55]

Once I get the file down loaded I'll take a look. The close should free memory.

On April 18, 2015 1:40:14 PM PDT, Marco Colli notifications@github.com wrote:

Even if I already mentioned this problem in
#228 saying that I get a "Killed:
9" this is a totally independent problem from the segmentation fault
and I think it deserves its own ticket.

Further investigating the "Killed: 9" I discovered (quite obvious) that
the ruby process steadily increases memory usage.

The line responsible for this issue is:

@entity = Oj::Doc.open(@line)

In fact by simply substituting it with

@entity = JSON.parse(@line)

I get no problems with memory usage. (The above lines produce different
objects, but this is not a problem, since in this demonstration
@entity is never used. Also I could have simply removed the line
instead of substituting it, but I have done this to prove that is not
simply a problem with the garbage collector)

Adding @entity.closeafter the open doesn't solve the issue.

---

Reply to this email directly or view it on GitHub:
#229

[collimarco]

To reproduce the issue you can use this script:
https://gist.github.com/collimarco/677e059206854932f1c4#file-gistfile3-rb

[collimarco]

Found out that this causes the issue:

    doc = Oj::Doc.open(line)
    create_item(doc) if fetch(doc, '/type') == 'item'
    doc.close

While this is correct and doesn't waste memory:

    Oj::Doc.open(line) do |doc|
      create_item(doc) if fetch(doc, '/type') == 'item'
    end

[ohler55]

I believe this is fixed a release or two ago.

[collimarco]

@ohler55 Sure, you have fixed it in version 2.12.4 a few days after my message. Thanks a lot! Now this issue can be closed.