Skip to content

Commit

Permalink
refactor for fixedstruct
Browse files Browse the repository at this point in the history
Refactor `utmpx` for Linux into cross-platform "FixedStruct"
for fixed-size record structs. This includes acct, acct_v3,
lastlog, lastlogx, utmp, and utmpx. This removes crate `uapi`.

Refactor FileType to allow sub-variants.

Add scraped logs from varying platforms.

Update compare-current-and-expected with newly scraped logs.

Add utility utmp-offsets for generating real offsets of fixed-size
structs on any system.

Issue #171
Issue #100
Issue #121
Issue #109
Issue #217
  • Loading branch information
jtmoon79 committed Mar 15, 2024
1 parent 9a23849 commit 98432e5
Show file tree
Hide file tree
Showing 389 changed files with 81,372 additions and 5,066 deletions.
116 changes: 114 additions & 2 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 4 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,12 @@ encoding_rs = "0.8.33"
evtx = { version = "0.8.1", features = ["multithreading"] }
flate2 = "1.0.27"
itertools = "0.11.0"
kinded = "0.3.0"
lazy_static = "1.4.0"
lexical = { version = "6.1.1", features = ["write-integers"] }
lru = "0.12.0"
lzma-rs = "0.3.0"
memoffset = "0.9.0"
mime_guess = "2.0.4"
min-max = "0.1.8"
more-asserts = "0.3.1"
Expand All @@ -55,7 +58,7 @@ phf = { version = "0.11", features = ["macros"] }
rand = "0.8.5"
rangemap = "1.3.0"
regex = "1.10.2"
si_trace_print = "0.3.10"
si_trace_print = "0.3.12"
static_assertions = "1.1.0"
tar = "0.4.40"
termcolor = "1.1.3"
Expand Down
7 changes: 4 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@ Speedily search and merge log messages by datetime.
_Super Speedy Syslog Searcher_ (`s4`) is a command-line tool to search
and merge varying log messages from varying log files, sorted by datetime.
This includes log files that are compressed (`.gz`, `.xz`), or archived (`.tar`),
and binary format logs including utmpx user accounting records (`utmp`, `wtmp`),
and binary format logs including acct/lastlog/utmp accounting records
(`acct`, `pacct`, `lastlog`, `utmp`, `utmpx`, `wtmp`),
systemd journal logs (`.journal`), and Microsoft Event Logs (`.evtx`).
It will parse a variety of formal and ad-hoc log message datetime formats.

Expand Down Expand Up @@ -339,7 +340,7 @@ See the real-world example rationale in section
- [RFC 5424]
- [ISO 8601] \*\*
- [Red Hat Audit Log] files
- binary ["utmpx" user accounting records] files
- binary [user accounting records] files (`acct`, `pacct`, `lastlog`, `utmp`, `utmpx`, `wtmp`)
- binary [Windows Event Log] files
- binary [systemd journal] files with printing options matching [`journalctl`]
- many varying text log messages with ad-hoc datetime formats
Expand All @@ -356,7 +357,7 @@ See the real-world example rationale in section
- Processes invalid UTF-8
- Accepts arbitrarily large files \*\*\*\*

["utmpx" user accounting records]: https://en.wikipedia.org/w/index.php?title=Utmp&oldid=1143684808#utmpx,_wtmpx_and_btmpx
[user accounting records]: https://en.wikipedia.org/w/index.php?title=Utmp&oldid=1143684808#utmpx,_wtmpx_and_btmpx
[RFC 2822]: https://www.rfc-editor.org/rfc/rfc2822#section-3.3
[RFC 3164]: https://www.rfc-editor.org/rfc/rfc3164#section-4.1.2
[RFC 3339]: https://www.rfc-editor.org/rfc/rfc3339#section-5.8
Expand Down
35 changes: 35 additions & 0 deletions logs/Alpine3.17/utmp-offsets-x86_64.out.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
$ (set -eux; uname -a; cc -o utmp-offsets utmp-offsets.c; ./utmp-offsets)
+ uname -a
Linux grape 5.15.107-0-lts #1-Alpine SMP Thu, 13 Apr 2023 17:30:11 +0000 x86_64 Linux
+ cc -o utmp-offsets utmp-offsets.c
+ ./utmp-offsets
lastlog sizeof 296
lastlog.ll_time @ 0 sizeof 8
lastlog.ll_line @ 8 sizeof 32
lastlog.ll_host @ 40 sizeof 256
utmp sizeof 400
utmp.ut_line @ 8 sizeof 32
utmp.ut_name @ 44 sizeof 32
utmp.ut_host @ 76 sizeof 256
utmp.ut_exit @332 sizeof 4
utmp.ut_session @336 sizeof 4
utmp.ut_time @344 sizeof 8
utmp.ut_tv @344 sizeof 16
utmp.ut_tv.tv_sec @344 sizeof 8
utmp.ut_tv.tv_usec@352 sizeof 8
utmp.ut_addr_v6 @360 sizeof 16
utmpx sizeof 400
utmpx.ut_type @ 0 sizeof 2
utmpx.ut_pid @ 4 sizeof 4
utmpx.ut_line @ 8 sizeof 32
utmpx.ut_id @ 40 sizeof 4
utmpx.ut_user @ 44 sizeof 32
utmpx.ut_name @ 44 sizeof 32
utmpx.ut_host @ 76 sizeof 256
utmpx.ut_exit @332 sizeof 4
utmpx.ut_session @336 sizeof 4
utmpx.ut_time @344 sizeof 8
utmpx.ut_tv @344 sizeof 16
utmpx.ut_tv.tv_sec @344 sizeof 8
utmpx.ut_tv.tv_usec@352 sizeof 8
utmpx.ut_addr_v6 @360 sizeof 16
Loading

0 comments on commit 98432e5

Please sign in to comment.