Rust: Improve lines-of-code counts.

[geoffw0]

Improve lines-of-code counts:

• remove Module from the count, which was spuriously adding +1 LOC in main.rs.
• add back calls to println! in the tests, since these should be deterministically (if not correctly) extracted now.
• coincidentally the total LOC counted changes after the first commit, but changes back exactly after the second. These changes are not doing nothing!

@aibaars - also note that we have a spuriously located MacroRules (on my_macro.rs line 1) and Struct (on my_struct.rs line 2), which I discovered while debugging this query. I haven't done anything about them, I believe the right fix would be in the extractor.

Obviously types are still not extracted, which is also affecting the counts.

[aibaars]

Please revert this change. The module isn't the problem. The problem are the /* */ comments which syntactically belong to the mod, struct, macrorule, etc. that follows them. Just verify for yourself using the AST viewer.

[geoffw0]

Hmm, I see what you mean now, Module is a module decl not a module. I'm going to have to get used to that.

[geoffw0]

Done.

Is there an easy way to use the AST viewer on a test?

Is there an issue for the location problems (or should I create one)?

[aibaars]

I think the AST viewer works fine on test databases these days. I just built a database running codeql test run --keep-databases, loaded it in vscode, ran from File f select f, click on a file, and right-click "view AST" in that file.

[geoffw0]

That method for tests is a bit of a fiddle unfortunately, but good to know the AST viewer now works for Rust. I had investigated using an adhoc query to print all the AstNode.toString()s associated with each line of each file, which worked - though I didn't get a hierarchy.

Anyway I've created an issue for the locations problems.

[aibaars]

In general using start and end lines of AstNodes to count lines will never work 100%. Currently we're overcounting a bit for items with attached comments. However, we may also change the location of top-level items to be the span of their names only. That gives a much better user-experience (so we highlightithe name of the function instead of the entire function body). Another tricky thing to deal with are multi-line string literals. The only reliable way to calculate LOC is to look at token, which we do not extract (though we might at some point).

A pretty simple way to deal with this problem is to ignore the start line for top-level items. That would under count in cases like

fn
my_function() {...}

but that would be extremely rare.

[aibaars]

Changes look good to me. However, you could implement special handling of top-level items to avoid counting doc comments as a line.