Skip to content

Commit

Permalink
Fix memory leak in active record cursor
Browse files Browse the repository at this point in the history
This commit patches the active record cursor implementation so that the
queries emitted by `relation.to_a` in `next_batch` are never cached.
This makes sense because it is a waste to cache results of queries like
`SELECT ... WHERE id > ... LIMIT ...` in an enumerator that is only
moving forward.

The `uncached` interface I make use of in this commit was introduced by
rails/rails#28867 under a similar context.
  • Loading branch information
pedropb committed Mar 18, 2021
1 parent b92d30e commit 218309e
Showing 1 changed file with 3 additions and 1 deletion.
4 changes: 3 additions & 1 deletion lib/job-iteration/active_record_cursor.rb
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,9 @@ def next_batch(batch_size)
relation = relation.where(*conditions)
end

records = relation.to_a
records = relation.uncached do
relation.to_a
end

update_from_record(records.last) unless records.empty?
@reached_end = records.size < batch_size
Expand Down

0 comments on commit 218309e

Please sign in to comment.