FAQ for pflogsumm - A Log Summarizer/Analyzer for the Postfix MTA Introduction I wouldn't have believed it. What started out mostly as a light- hearted exercise in improving my facility with Perl--with the hope that something useful would come out of it as well--has turned out to be a somewhat popular utility. And as more Admins find out about postfix, and more end up trying pflogsumm, many of the questions, suggestions, and enhancement requests are becoming "frequently asked". So odd as it seems (to me, at any rate), it looks like it's time for a FAQ. Index of pflogsumm Frequently Asked Questions (in no particular order) 1. Project Status 2. "Could You Make" or "Here's A Patch To Make" pflogsumm Do ... 3. Built-In Support for Compressed Logs 4. Processing Multiple Log Files 5. By-domain Listings 6. Reject, Deferred and Bounced Detail Info 7. "Orphaned" (no size) Messages 8. pflogsumm misses/mis-diagnoses/mis-reports, etc. 9. pflogsumm just doesn't work or doesn't report anything 10. Postfix Rejects pflogsumm Reports Because Of Body Checks 11. pflogsumm Reports Double Traffic When Anti-Virus Scanner Used 12. pflogsumm's numbers don't add up 13. Hourly stats for reports run without "-d" option are halved 14. How Do I Get pflogsumm To Email Reports To Me Daily/Weekly/etc.? 15. How Can I View pflogsumm's Reports In My Web Browser? 16. How can I best format my custom reject messages for display in pflogsumm's output? 17. pflogsumm doesn't understand my log file format 18. Why Isn't There Any Mention Of "Monkey Butler" In The FAQ? 19. Translating pflogsumm (Support for Internationalization) 20. Sending Logfile Samples 21. From Where Can I Obtain pflogsumm? 1. Project Status After a 20-year hiatus from pflogsumm development, in 2025 I returned to actively developing it. Since then there have been several Alpha, Beta, and Production releases. 2. "Could You Make" or "Here's A Patch To Make" pflogsumm Do ... To the argument "But it's a patch, all you have to do is...," the answer is: "Not quite." Every time I make a change to pflogsumm I have to run it through a series of regression checks to make sure the change didn't break something. Then there's the commit, documentation, web page update, etc. cycle. I'm particularly unlikely to add code to pflogsumm to account for non-standard Postfix log entries. "Non-standard" being defined as "other than what Wietse's code does." Or additional stats gathering that nobody else has requested and strikes *me* as of limited interest or use. In addition to the development cycle, there's the issue of "code bloat." pflogsumm already takes enough (too much?) time and memory on busy machines with large logs. I'm not prone to make this worse for the sake of these things. See Also: 17. pflogsumm doesn't understand my log file format 3. Built-In Support for Compressed Logs I took a look at this. There is a Perl module (which I downloaded, built, and installed here) to interface to libz, but after considering the changes that would be necessary--and the fact that those changes would require that potential users have to download/build/install libz (and of the correct version) and the additional Perl module, I decided to forego this enhancement. I could just open a pipe within pflogsumm and use zcat/gunzip/gzip. That would depend upon a) them being there [probably a safe bet-- considering the logs somehow got into that format :-), but...] and b) one of these either being in the path or having an environment variable or a script variable or... The thing is, in the latter case there's really no "savings" over simply piping into pflogsumm in the first place. Multiple processes get spawned, pipes opened, etc. either way. It would add a little convenience, is all. So I could do it. And there are a couple of ways I could do it. And my mind is certainly still open on the issue. I'm just not convinced there's a good reason to do it, is all. And I'd like to avoid "creeping over-feature-itis" if I can. My position is *not* set in stone on this issue. In the mean-time: zcat /var/log/maillog.0.gz |pflogsumm or gunzip should do the trick quite nicely for you. If you've a complex situation, for example: your logs aren't rotated exactly at midnight, you might try something like: (zcat /var/log/maillog.0.gz; cat /var/log/maillog) \ |pflogsumm -d yesterday See Also: 4. Processing Multiple Log Files 14. How Do I Get pflogsumm To Email Reports To Me Daily/Weekly/etc.? 4. Processing Multiple Log Files When processing multiple log files (say: an entire weeks worth of logs that are rotated daily), it is important that pflogsumm be fed them in chronological order. For performance and memory conservation reasons, pflogsumm relies on log messages "arriving" in the order in which they were created. If you do something like this: pflogsumm /var/log/maillog* you might not get what you expect! Instead, try something like: pflogsumm `ls -rt /var/log/maillog*` A more complex example, where compressed logs are involved: (zcat `ls -rt /var/log/maillog.*.gz`; cat /var/log/maillog) \ |pflogsumm Obviously, this depends on the file modification times for your logs being reflective of their chronological order. If that can't be trusted, you're gonna have to get ugly. Like in enumerating each file, or as in: (for each in 3 2 1 0; do zcat "/var/log/maillog.$each.gz" done cat /var/log/maillog) |pflogsumm or (somewhat more efficiently--by running zcat only once): (zcat `for ea in 3 2 1 0; do echo "/var/log/maillog.$ea.gz"; done`; cat /var/log/maillog) |pflogsumm [Note: I didn't actually run these. So you would be well-advised to double-check them.] See Also: 3. Built-In Support for Compressed Logs 14. How Do I Get pflogsumm To Email Reports To Me Daily/Weekly/etc.? 5. By-domain Listings I figured on the desire for this one from the start. There are many possibilities: 1) A single report, split by domain 2) An option to limit reporting to a particular domain This issue is kind of tricky. The popularity of Unix amongst SysAdmins is testimony to the beauty of being able to wire- together small, simple tools so that one can generate output to ones taste. Anything I do is likely to make some Admins happy and others wishing I'd done it "the other way". One thought that occurred is to perhaps provide a couple of options that would allow one to limit a particular report to sender=regular_expression and/or recipient=regular_expression The problem with this solution is that an Admin desiring to emit custom reports for multiple domains would have to re-process the same log multiple times--once for each desired domain. So I'm still thinking about this one. 6. Reject, Deferred and Bounced Detail Info I've actually only received one query about this so far, but there are bound to be more. So... The "detailed" information in the "Reject", "Deferred" and "Bounced" reports is a compromise. Just take a stroll through your postfix logs some day and observe the variation in how the "reason" for a particular reject, defer, or bounce is reported. Without putting a lot of static comparisons for each-and-every case into the analyzer, I have absolutely no hope of doing this very well. Emitting the entire "reason" is not good, either. The entire "reason" string can be very long. Depending on what somebody is using to display pflogsumm's output, the lines may well wrap-- producing output that is no more readable than just grepping the logs. And anything more I do to this end may soon be rendered moot. After Wietse gets most of the more important functional stuff out of the way, Postfix logging is going to be completely re-written. (Oh boy, won't that be fun!) I'm hoping I'll be able to get some input into the process so the formatting is more amenable to automated processing. Wietse has indicated that such would be the case. Also, please note my primary objective behind pflogsumm (besides the entertainment value): "just enough detail to give the administrator a ``heads up'' for potential trouble spots." It's not *supposed* to do away with manual inspection entirely. For those that really want all that extra detail in the log summary reports, specify the "--verbose-msg-detail" switch. See Also: 20. Sending Logfile Samples 7. "Orphaned" (no size) Messages The Problem: Message size is reported only by the queue manager. The message may be delivered long-enough after the (last) qmgr log entry that the information is not in the log(s) processed by a particular run of pflogsumm. The Result: "Orphaned" messages. These are reported by pflogsumm as "Messages with no size data." This, of course, throws off "Recipients by message size" and the total for "bytes delivered." ("bytes in messages" in earlier versions.) The Solution: "Memorize" message sizes by queue i.d. Easy in theory. Difficult in practice. At least at the moment. You see, if pflogsumm's going to "memorize" message sizes, it has to have some definitive way to know when to delete a no- longer-needed reference. Otherwise the memory file will just grow forever. As with the "Reject, Deferred and Bounced Detail Info" issue above, I'm hoping to get some input into future changes in logging issues. In any event: maybe whatever comes out of the logging redesign will provide a solution. As of pflogsumm version 1.0.12, the "Messages with no size data" report can be turned off. 8. pflogsumm misses/mis-diagnoses/mis-reports, etc. Are you processing your log files in chronological order? See item "4: "Processing Multiple Log Files". pflogsumm is being developed by me on my rather small-scale server at home. There are only two users on the system. And I do no mail-forwarding. So the log samples I have to work with are commensurately limited. If there's something that pflogsumm is not doing, or not doing right, let me know what it is, what you think it ought to do, and send me a representative sample of *real* log entries with which to work. See Also: 9. pflogsumm just doesn't work or doesn't report anything 12. pflogsumm's numbers don't add up 17. pflogsumm doesn't understand my log file format 20. Sending Logfile Samples 9. pflogsumm just doesn't work or doesn't report anything Did you *download* pflogsumm as opposed to grabbing it by "copy-and-paste" from a browser? Copy-and-paste can result in lines being unintentionally wrapped and hard-tabs being converted to spaces. This will break pflogsumm. Also, I've received a couple of reports by people downloading pflogsumm with Lynx that the download has long lines wrapped. Naturally, this breaks pflogsumm. See Also: 8. pflogsumm misses/mis-diagnoses/mis-reports, etc. 17. pflogsumm doesn't understand my log file format 10. Postfix Rejects pflogsumm Reports Because Of Body Checks You configure Postfix to do body checks, Postfix does its thing, pflogsumm reports it and Postfix catches the same string in the pflogsumm report. There are several solutions to this. Wolfgang Zeikat contributed this: #!/usr/bin/perl use MIME::Lite; ### Create a new message: $msg = MIME::Lite->new( From => 'your@send.er', To => 'your@recipie.nt', # Cc => 'some@other.com, some@more.com', Subject => 'pflogsumm', Date => `date`, Type => 'text/plain', Encoding => 'base64', Path =>'/tmp/pflogg', ); $msg->send; Where "/tmp/pflogg" is the output of pflogsumm. This puts pflogsumm's output in a base64 MIME attachment. In a follow-up to a thread in the postfix-users mailing list, Ralf Hildebrandt noted: "mpack does the same thing." The canonical FTP site for mpack is ftp.andrew.cmu.edu:pub/mpack/ The solution I came up with is to modify the body_checks statements to ignore the strings when they're in a pflogsumm report, as follows: Bounce anything with 6 or more "$"s in a row... /\${6,}/ REJECT Which, of course, catches the line in the pflogsumm report too. So... /^(?!\s+[0-9]+\s+).*?\${6,}/ REJECT which reads "anything with 6 or more '$'s in a row that is not a line beginning with one or more whitespace characters, followed by one or more digits, followed by one or more whitespace characters." (This is using PCRE's, btw.) Note that my solution will be more computationally expensive, by a *long* way, than encoding pflogsumm's output into a format that body_checks won't catch. Robert L Mathews suggested the following solution /^ {6,11}[[:digit:]]{1,6}[ km] / OK Placed at the beginning of a body_checks file, this will "pre-approve" lines in pflogsumm's output that might otherwise get caught. That's a POSIX regexp version. A PCRE version of the same thing would be: /^ {6,11}\d{1,6}[ km] / OK 11. pflogsumm Reports Double Traffic When Anti-Virus Scanner Used Sadly, there's absolutely nothing I can do about this :-(. The problem arises because of the way in which anti-virus scanning is handled by Postfix. Basically, Postfix "delivers" each email to the anti-virus scanner and the anti-virus scanner re-sends it through Postfix. So each email really is received twice and sent/delivered twice. And yes, I tried. I really, really tried. If I recall correctly, I spent some two days mucking-about with this problem. Actually thought I had it once or twice. But the results inevitably failed regression testing. At the end of this, and with some more careful thought, I realized it just wasn't possible. If you think you can prove me wrong, please do so. I'd be quite pleased to be proven wrong on this one. Note: As of 2026-08-16 I'm not certain the FAQ entry is still true. 12. pflogsumm's numbers don't add up pflogsumm reports more "delivered" than "received" Naturally. A single email message can have multiple recipients. pflogsumm reports more "rejected" than "received" Why doesn't delivered + deferred + bounce + rejected = received? Some rejects (header and body checks, for example) happen in "cleanup," after alias lists are expanded. Thus a single received message will be rejected multiple times: once for each recipient. The "size=" fields, multiplied by their "nrcpt=" fields, when added-up yields a total higher than pflogsumm's "bytes delivered" total. pflogsumm doesn't count something delivered until it actually *is* delivered. Nrcpt only suggests the number of intended recipients, not how many are actually deliverable. Only if there were no bounces, rejects, defers or other undeliverables for everything that was received would a calculation such as that above yield the proper value. pflogsumm's "% rejected" doesn't add up The "percent rejected" and "percent discarded" figures are only approximations. They are calculated as follows (example is for "percent rejected"): percent rejected = (rejected / (delivered + rejected + discarded)) * 100 Given the issues discussed above, this is really the best that can be hoped-for, IMO. I consistently see more "delivered" than "received." How is that possible? Any message that's got multiple recipients in the "To:," "Cc:," and "Bcc:" fields will result in a single "received" with multiple "delivered"s, as well as, possibly, multiple "rejects" (or reject warnings, discards or holds), depending on where in Postfix' processing the rule was that resulted in the reject, etc. See Also: 8. pflogsumm misses/mis-diagnoses/mis-reports, etc. 13. Hourly stats for reports run without "-d" option are halved Scenario: On day #1 of a fresh logfile, you run pflogsumm with "-d today" and the next day you run it with no "-d" option. The "Per-Hour Traffic" statistics are approximately halved. How can this be? Note that when you run pflogsumm on a logfile that contains multi-day logfile entries, pflogsumm automatically changes the per-hour stats to daily traffic averages. If there's even *one* logfile entry from another day, all of the per-hour stats will be divided by two. Unless you rotate logfiles *precisely* at midnight--and it's unlikely you can guarantee that happening--there's no way to prevent this. 14. How Do I Get pflogsumm To Email Reports To Me Daily/Weekly/etc.? Excuse me? You're running a mailserver and you don't know how to use cron to run things on a scheduled basis and pipe the output to something that'll email it to you? Oh. My. Lord. *sigh* Here's my crontab entries: 10 0 * * * /usr/local/sbin/pflogsumm -d yesterday /var/log/syslog \ 2>&1 |/usr/bin/mailx -s "`uname -n` daily mail stats" postmaster 10 4 * * 0 /usr/local/sbin/pflogsumm /var/log/syslog.0 \ 2>&1 |/usr/bin/mailx -s "`uname -n` weekly mail stats" postmaster (Those are actually each a single line. I line-broke them [and signified that with the "\"s] for readability.) The first generates stats for the previous day and is run *after* midnight. The second is run against the previous week's entire log. (I rotate my logs weekly.) If you rotate your logs on a different schedule, want monthly reports, etc., I leave it as an exercise to you, the reader, to figure out how to concatenate several logs to stdout and feed that to pflogsumm. See Also: 3. Built-In Support for Compressed Logs 4. Processing Multiple Log Files The Unix manual pages for "cron," "crontab," "cat," "zcat," "gzip," "gunzip," "mail," "mailx," etc. 15. How Can I View pflogsumm's Reports In My Web Browser? Just direct pflogsumm's output to a file, call it "something.txt" or whatever, and look at it with your browser :). If you want to get fancy, create a post-processing shell script that'll create a date-tagged file, like "mailstats-20030216.txt". It's easy. 16. How can I best format my custom reject messages for display in pflogsumm's output? Reject reason strings found in the mail log will be truncated at the first comma (","), colon (":") or semi-colon (";"). If you want a "clause" in your reject message to appear in pflogsumm's output, without having to specify --verbose-msg-detail, use a punctuation mark other than one of those three, such as a dash ("-"). 17. pflogsumm doesn't understand my log file format I've received several requests to modify pflogsumm's log file format regular expression matching to accommodate "non-standard" log file formats. I'm not inclined to honour such requests. The regexp that identifies Postfix' log file entries is nearly incomprehensible as it is. If your log file format has extra fields (e.g.: FreeBSD syslogd with "-v -v" specified), or, as in one case (metalog), is lacking fields, and you insist on doing things that way, I recommend you code-up a little pre-filter to mung the format into a standard one. See Also: 8. pflogsumm misses/mis-diagnoses/mis-reports, etc. 9. pflogsumm just doesn't work or doesn't report anything 18. Why Isn't There Any Mention Of "Monkey Butler" In The FAQ? An on-line acquaintance asked if I'd put the phrase "monkey butler" in the FAQ. The answer is no. pflogsumm is used by some rather large corporations. There are credibility issues. Sorry. :) 19. Translating pflogsumm (Support for Internationalization) Unfortunately, pflogsumm doesn't currently have i18n support. Prospective translators are urged to translate *only* the stable Production versions. Beta and Alpha versions can sometimes change rapidly. If you do translate pflogsumm, let me know and I'll put a link to it on pflogsumm's main web page. 20. Sending Logfile Samples Here's the deal with whatever you may send me in the way of log samples: . Obfuscate them if you want. But take care not to alter them in such a manner that they're not accurate wrt the "realism" of the data, make sure the field formatting is not altered, and that the order of the log entries is not altered. . The world is an unsafe place for your data, no matter where it might reside. But I'll do my level best to ensure that your data does not fall into the hands of others. . If you want, I'll PGP-encrypt the data when it's not in use. . You can PGP-encrypt it when you send it to me if you're concerned. My PGP public key can be found on my Web site and at the PGP public key servers. . If you want, I'll delete the sample data when the work is done. But I would *like* to keep it around for future regression-testing. It's your call. Let me know. 21. From Where Can I Obtain pflogsumm? https://athena.LinxNet.com/postfix_contrib.html Created: 15 Feb., 1999 / Last updated: 15 Aug., 2026